From arnd.baecker at web.de Tue Mar 2 03:59:01 2004 From: arnd.baecker at web.de (Arnd Baecker) Date: Tue, 2 Mar 2004 09:59:01 +0100 (CET) Subject: [SciPy-dev] misprint & system_info.py problem Message-ID: Hi, I just checked out the CVS version (scipy.__version__, '0.2.1_253.3734' ) and found two small issues a) >>> help("scipy.xplt") File "/opt/python/lib/python2.3/site-packages/scipy/optimize/optimize.py", line 268 if (c<=0) or (i%3)==2): ^ SyntaxError: invalid syntax Removing the additional bracket fixes the problem. b) python scipy_core/scipy_distutils/system_info.py gives the following error (shortened, full output at the end): python scipy_core/scipy_distutils/system_info.py atlas_blas_info: [...] atlas_blas_threads_info: [...] atlas_info: [...] system_info.atlas_info [...] atlas_threads_info: [...] blas_info: [...] blas_opt_info: Traceback (most recent call last): File "scipy_core/scipy_distutils/system_info.py", line 1190, in ? show_all() File "scipy_core/scipy_distutils/system_info.py", line 1187, in show_all r = c.get_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 279, in get_info self.calc_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 930, in calc_info atlas_version = get_atlas_version(**version_info) File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 771, in get_atlas_version from scipy_distutils.core import Extension, setup ImportError: No module named scipy_distutils.core This happens before doing a scipy installation, i.e. if there are no scipy* in site-packages. After the installation `python scipy_core/scipy_distutils/system_info.py` works fine. This may have been like this for quite a while as normally I don't remove the scipy* directories in site-packages. Many thanks, Arnd Full output: ----------- python scipy_core/scipy_distutils/system_info.py atlas_blas_info: ( library_dirs = /opt/python/lib:/usr/local/lib:/usr/lib ) ( paths: /opt/python/lib/libf77blas.a ) ( paths: /opt/python/lib/libcblas.a ) ( paths: /opt/python/lib/libatlas.a ) system_info.atlas_blas_info ( include_dirs = /usr/local/include:/usr/include:/opt/python/include ) ( paths: /opt/python/lib/cblas.h ) FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/opt/python/lib'] language = c include_dirs = ['/opt/python/lib'] atlas_blas_threads_info: ( library_dirs = /opt/python/lib:/usr/local/lib:/usr/lib ) ( paths: /opt/python/lib/libatlas.a ) system_info.atlas_blas_threads_info NOT AVAILABLE atlas_info: ( library_dirs = /opt/python/lib:/usr/local/lib:/usr/lib ) ( paths: /opt/python/lib/libf77blas.a ) ( paths: /opt/python/lib/libcblas.a ) ( paths: /opt/python/lib/libatlas.a ) ( paths: /opt/python/lib/liblapack.a ) system_info.atlas_info ( include_dirs = /usr/local/include:/usr/include:/opt/python/include ) ( paths: /opt/python/lib/cblas.h ) FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['/opt/python/lib'] language = f77 include_dirs = ['/opt/python/lib'] atlas_threads_info: ( library_dirs = /opt/python/lib:/usr/local/lib:/usr/lib ) ( paths: /opt/python/lib/libatlas.a ) system_info.atlas_threads_info NOT AVAILABLE blas_info: ( library_dirs = /opt/python/lib:/usr/local/lib:/usr/lib ) ( paths: /usr/lib/libblas.so ) FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 blas_opt_info: Traceback (most recent call last): File "scipy_core/scipy_distutils/system_info.py", line 1190, in ? show_all() File "scipy_core/scipy_distutils/system_info.py", line 1187, in show_all r = c.get_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 279, in get_info self.calc_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 930, in calc_info atlas_version = get_atlas_version(**version_info) File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 771, in get_atlas_version from scipy_distutils.core import Extension, setup ImportError: No module named scipy_distutils.core From pearu at scipy.org Tue Mar 2 06:15:38 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 2 Mar 2004 05:15:38 -0600 (CST) Subject: [SciPy-dev] misprint & system_info.py problem In-Reply-To: Message-ID: On Tue, 2 Mar 2004, Arnd Baecker wrote: > I just checked out the CVS version (scipy.__version__, '0.2.1_253.3734' ) > and found two small issues > > a) >>> help("scipy.xplt") > File > "/opt/python/lib/python2.3/site-packages/scipy/optimize/optimize.py", line > 268 > if (c<=0) or (i%3)==2): > ^ > SyntaxError: invalid syntax Travis has fixed it in CVS. > Removing the additional bracket fixes the problem. > > b) python scipy_core/scipy_distutils/system_info.py > gives the following error (shortened, full output at the end): > ImportError: No module named scipy_distutils.core Fixed in CVS. Thanks, Pearu From charles.harris at sdl.usu.edu Wed Mar 3 16:44:21 2004 From: charles.harris at sdl.usu.edu (Chuck Harris) Date: Wed, 3 Mar 2004 14:44:21 -0700 Subject: [SciPy-dev] testing Message-ID: <1885D1238A4FFE40B113CEB26F87872B0667D5@cobra.usurf.usu.edu> From charles.harris at sdl.usu.edu Wed Mar 3 16:45:44 2004 From: charles.harris at sdl.usu.edu (Chuck Harris) Date: Wed, 3 Mar 2004 14:45:44 -0700 Subject: [SciPy-dev] FW: RFC: Chebychev polynomials Message-ID: <1885D1238A4FFE40B113CEB26F87872B0667D6@cobra.usurf.usu.edu> Hi all, I have written a small set of routines for dealing with Chebychev polynomials. I would like suggestions for improvements, extra functions to include etc. I use these functions often myself, and thought they might be useful to include in scipy. Where they might go is an open question. Perhaps a package of polynomial routines that also contained divided differences, Lagrange interpolation, Newton interpolation, a modern root finding routine, a routine for orthogonal polynomials over a given set of sample points, etc. I have written some of these and wonder what the general interest is. A question about power series. There are routines in scipy-base for manipulating power series, but the coefficients are ordered from high degree to low degree. This is okay for polynomials, maybe, but doesn't seem the most natural order when dealing with the initial entries in an infinite series. Is there any settled view on this? My own routine that converts Chebychev series to power series orders the coefficients from low degree to high degree and the Chebychev coefficients themselves are ordered in the same way. I have a few routines that might go into a misc package, specifically routines for equivalence relations, very useful for clustering, and a generator for permutations of a list. Any interest? The contouring functions in xplt have also proved useful as stand alone functions. It might be nice to break these out into their own module. I think it might be nice to have separate functions for various spline operations also : B-splines, splines in tension, not-a-knot end conditions, natural splines, etc. The current single function call is hard to understand in line, and in any case, python it the natural language in which to produce such a monstrosity. I have attached the Chebychev module. Feedback welcome. Chuck -------------- next part -------------- A non-text attachment was scrubbed... Name: Chebychev1.py Type: application/octet-stream Size: 19653 bytes Desc: Chebychev1.py URL: From arnd.baecker at web.de Fri Mar 5 08:05:02 2004 From: arnd.baecker at web.de (Arnd Baecker) Date: Fri, 5 Mar 2004 14:05:02 +0100 (CET) Subject: [SciPy-dev] distutils, weave dependency of scipy/xplt/Mplot.py Message-ID: Hi, while tracking down a problem with scipy on the latest quantian 0.4.9.4 (based on Knoppix) we came across the following: A) weave dependency of scipy/xplt/Mplot.py In scipy/xplt/Mplot.py the function _getdir() uses weave to get a directory that the user has write access to. So commenting out the line weave = ppimport('weave'); _level_docs(weave) in scipy/__init__.py is not enough to remove any dependency on weave. Would it be possible to write a _getdir() which does not make use of weave ? Presumably this is not that critical, but in the quantian case python2.3-dev was not installed which contains distutils. This brings me to my second question: B) It seems that without distutils an `import weave` fails. Are distutils really necessary to _use_ weave ? (At least for scipy itself distutils seems only necessary to run scipy.test()) Best, Arnd From pearu at scipy.org Fri Mar 5 08:23:33 2004 From: pearu at scipy.org (Pearu Peterson) Date: Fri, 5 Mar 2004 07:23:33 -0600 (CST) Subject: [SciPy-dev] distutils, weave dependency of scipy/xplt/Mplot.py In-Reply-To: Message-ID: On Fri, 5 Mar 2004, Arnd Baecker wrote: > Hi, > > while tracking down a problem with scipy on the latest > quantian 0.4.9.4 (based on Knoppix) > we came across the following: > > A) weave dependency of scipy/xplt/Mplot.py > > In scipy/xplt/Mplot.py the function _getdir() > uses weave to get a directory that the user > has write access to. > > So commenting out the line > weave = ppimport('weave'); _level_docs(weave) > in scipy/__init__.py > is not enough to remove any dependency on weave. > Would it be possible to write a _getdir() which > does not make use of weave ? A quick look at the xplt code makes also me to wonder whether weave dependency is necessary there but I am not familiar with xplt details. Travis? > Presumably this is not that critical, but in the quantian > case python2.3-dev was not installed which contains distutils. > > This brings me to my second question: > > B) It seems that without distutils an `import weave` fails. > Are distutils really necessary to _use_ weave ? > (At least for scipy itself distutils seems only > necessary to run scipy.test()) I think the answer is yes. For one reason, do grep distutils *.py */*.py in weave/ directory ;) Pearu From oliphant at ee.byu.edu Fri Mar 5 10:53:50 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 05 Mar 2004 09:53:50 -0600 Subject: [SciPy-dev] distutils, weave dependency of scipy/xplt/Mplot.py In-Reply-To: References: Message-ID: <4048A28E.5030803@ee.byu.edu> Pearu Peterson wrote: > > >>A) weave dependency of scipy/xplt/Mplot.py >> >> In scipy/xplt/Mplot.py the function _getdir() >> uses weave to get a directory that the user >> has write access to. >> >> So commenting out the line >> weave = ppimport('weave'); _level_docs(weave) >> in scipy/__init__.py >> is not enough to remove any dependency on weave. >> Would it be possible to write a _getdir() which >> does not make use of weave ? >> >> Sure it's possible so I just did it (current CVS updated). It's only the utilties of weave that were used. Specifically, the utilities in weave/catalog.py I just copied those functions over to Mplot.py Perhaps we should refactor several of the file utility routines from weave into scipy_base/fileutils.py or something -Travis O. From j_r_fonseca at yahoo.co.uk Fri Mar 5 16:49:21 2004 From: j_r_fonseca at yahoo.co.uk (=?iso-8859-1?Q?Jos=E9?= Fonseca) Date: Fri, 5 Mar 2004 21:49:21 +0000 (UTC) Subject: [SciPy-dev] latest scipy CVS fails to build with python-2.1 Message-ID: When trying to rebuild debian packages for lates CVS I noticed it fails to build with python-2.1. When "byte compiling" scipy_core python complains: ---- /usr/lib/python2.1/site-packages/scipy_distutils/system_info.py:770: SyntaxWarning: local name 'magic' in 'get_atlas_version' shadows use of 'magic' as global in nested scope 'atlas_version_c' def get_atlas_version(**config): ---- And when trying to build scipy it fails with: ---- building extension "atlas_version" sources creating build creating build/src global name 'magic' is not defined FOUND: include_dirs = ['/usr/include'] language = c library_dirs = ['/usr/lib'] libraries = ['f77blas', 'cblas', 'atlas'] define_macros = [('NO_ATLAS_INFO', 2)] Traceback (most recent call last): File "setup.py", line 125, in ? setup_package(ignore_packages) File "setup.py", line 98, in setup_package parent_path=local_path) File "setup.py", line 59, in get_packages config = setup_module.configuration(*args) File "/home/jfonseca/projects/debian/scipy/SciPy-0.2.1_253.3737/Lib/integrate/setup_integrate.py", line 23, in configuration linpack_lite = ('linpack_lite_src',{'sources':local_glob('linpack_lite','*.f')}) File "/home/jfonseca/projects/debian/scipy/SciPy-0.2.1_253.3737/Lib/integrate/setup_integrate.py", line 17, in local_glob return glob.glob(os.path.join(*((local_path,)+names))) NameError: global name 'local_path' is not defined ---- This must have been caused by some recent change in scipy_distutils which violates python 2.1 syntax. Python 2.2 and 2.3 build fine. Note that if there is no interest to support python 2.1 in scipy, I can drop those packages just as well. BTW, the reason scipy is not yet on debian official repository is that no sponsor replied my request... I'll incorporate some fixes resulting from the user feedback I received and try again. Regards, Jose Fonseca From gillet at scripps.edu Wed Mar 10 23:08:11 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Wed, 10 Mar 2004 20:08:11 -0800 Subject: [SciPy-dev] problem building scipy on win32 Message-ID: <404FE62B.2080502@scripps.edu> Hi, I am trying to build SciPy-0.2.0_alpha_200.4161. I did follow the note from Pearu Peterson (http://cens.ioc.ee/~pearu/scipy/BUILD_WIN32.html) I am not using the cvs version, but I did use the cvs version of scipy_core for installing it. Then I cd in SciPy-0.2.0_alpha_200.4161 and type msys.py python setup.py build. I am now getting the following error: /building 'scipy_base.fastumath' extension Traceback (most recent call last): File "setup.py", line 111, in ? setup_package() File "setup.py", line 101, in setup_package url = "http://www.scipy.org", File "scipy_core\scipy_distutils\core.py", line 42, in setup return old_setup(**new_attr) File "c:\Python23\lib\distutils\core.py", line 149, in setup dist.run_commands() File "c:\Python23\lib\distutils\dist.py", line 907, in run_commands self.run_command(cmd) File "c:\Python23\lib\distutils\dist.py", line 927, in run_command cmd_obj.run() File "c:\Python23\lib\distutils\command\build.py", line 107, in run self.run_command(cmd_name) File "c:\Python23\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "c:\Python23\lib\distutils\dist.py", line 927, in run_command cmd_obj.run() File "c:\Python23\lib\distutils\command\build_ext.py", line 269, in run self.build_extensions() File "c:\Python23\lib\distutils\command\build_ext.py", line 395, in build_extensions self.build_extension(ext) File "scipy_core\scipy_distutils\command\build_ext.py", line 116, in build_extension res = old_build_ext.build_extension(self,ext) File "c:\Python23\lib\distutils\command\build_ext.py", line 492, in build_extension target_lang=language) File "c:\Python23\lib\distutils\ccompiler.py", line 843, in link_shared_object extra_preargs, extra_postargs, build_temp, target_lang) TypeError: link() takes exactly 13 arguments (14 given) Any Idea. I need to use that version of Scipy as the GA module seems not to work on later version, that version was installed on Linux and works fine for us. Thanks Alex /-- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 / / From pearu at scipy.org Thu Mar 11 13:25:29 2004 From: pearu at scipy.org (Pearu Peterson) Date: Thu, 11 Mar 2004 12:25:29 -0600 (CST) Subject: [SciPy-dev] problem building scipy on win32 In-Reply-To: <404FE62B.2080502@scripps.edu> Message-ID: On Wed, 10 Mar 2004, Alexandre Gillet wrote: > Hi, > I am trying to build SciPy-0.2.0_alpha_200.4161. > I did follow the note from Pearu Peterson > (http://cens.ioc.ee/~pearu/scipy/BUILD_WIN32.html) > I am not using the cvs version, but I did use the cvs version of > scipy_core for installing it. > Then I cd in SciPy-0.2.0_alpha_200.4161 and type msys.py python setup.py > build. > > I am now getting the following error: > > /building 'scipy_base.fastumath' extension > Traceback (most recent call last): > File "setup.py", line 111, in ? > setup_package() > File "setup.py", line 101, in setup_package > url = "http://www.scipy.org", > File "scipy_core\scipy_distutils\core.py", line 42, in setup > return old_setup(**new_attr) > File "c:\Python23\lib\distutils\core.py", line 149, in setup > dist.run_commands() > File "c:\Python23\lib\distutils\dist.py", line 907, in run_commands > self.run_command(cmd) > File "c:\Python23\lib\distutils\dist.py", line 927, in run_command > cmd_obj.run() > File "c:\Python23\lib\distutils\command\build.py", line 107, in run > self.run_command(cmd_name) > File "c:\Python23\lib\distutils\cmd.py", line 333, in run_command > self.distribution.run_command(command) > File "c:\Python23\lib\distutils\dist.py", line 927, in run_command > cmd_obj.run() > File "c:\Python23\lib\distutils\command\build_ext.py", line 269, in run > self.build_extensions() > File "c:\Python23\lib\distutils\command\build_ext.py", line 395, in > build_extensions > self.build_extension(ext) > File "scipy_core\scipy_distutils\command\build_ext.py", line 116, in > build_extension > res = old_build_ext.build_extension(self,ext) > File "c:\Python23\lib\distutils\command\build_ext.py", line 492, in > build_extension > target_lang=language) > File "c:\Python23\lib\distutils\ccompiler.py", line 843, in > link_shared_object > extra_preargs, extra_postargs, build_temp, target_lang) > TypeError: link() takes exactly 13 arguments (14 given) That error is due to the bug in Python 2.3 distutils. scipy_distutils implements a workaround to this issue but the workaround is effective only when setup scripts use scipy_distutils, the setup scripts in SciPy-0.2.0_alpha_200.4161 apparently do not do that. > Any Idea. > I need to use that version of Scipy as the GA module seems not to work > on later version, that version was installed on Linux and works fine for us. It might be easier to fix whatever issues you have with the recent GA module rather than trying to get SciPy-0.2.0_alpha_200.4161 working. What problems exactly do you experience with the GA module? Pearu From gazzar at email.com Tue Mar 16 08:19:38 2004 From: gazzar at email.com (Gary Ruben) Date: Tue, 16 Mar 2004 23:19:38 +1000 Subject: [SciPy-dev] value+error type (long) Message-ID: <20040316131938.2D620153608@ws3-1.us4.outblaze.com> I recently became fed up with performing error calculations on experimental data and decided to make a Python data type which I could use with Numeric. Now I can enter data with error along with the errors as in the following example and all the error calculations are automatically carried through for me. I thought there might be some interest amongst others for inclusion of this in the scipy distribution, although you might prefer it to be in C. I provide it as food for thought, but let me know if there is interest in including it as is and I can look at making it robust by adding testcases etc. and some nicer helper functions for constructors with relative errors etc. Note, the helper functions for extracting the limits would be better implemented as properties of an array type subclassed from the Numeric array type. I gather Numarray would allow this, but I couldn't think of a way to do it in Numeric and I wanted to maintain compatibility. Similarly, I couldn't think of a nicer way to apply Ufuncs across the prime value and error extrema. It would be nice to not have to call ApplyUfuncToErr() to do this. Here's a sample of usage of the class, followed by the class itself. Note I've used Gnuplot.py in this but you'll get the idea by looking at the code: -- R_AB.py: from Numeric import * import Gnuplot, Gnuplot.funcutils from ErrorVal import * from scipy.interpolate import splrep,splev # manometer readings [mmHg] upperPressure = ArrayOfErr([909., 802., 677., 585., 560., 548.], 1.0) lowerPressure = ArrayOfErr([144., 246., 378., 469., 493., 505.], 1.0) pressureDiff = upperPressure - lowerPressure # Temperatures & Pressures from lookup table lowerTablePressures = array([760. , 540. , 290. , 110. , 60. , 40. ]) # [mmHg] lowerTableTemps = array([4.210, 3.867, 3.332, 2.685, 2.373, 2.194]) # [K] upperTablePressures = array([780. , 560. , 300. , 120. , 70. , 50. ]) # [mmHg] upperTableTemps = array([4.238, 3.902, 3.358, 2.735, 2.447, 2.290]) # [K] # Linearly interpolate table values to produce temperatures temps = ( lowerTableTemps + (pressureDiff - lowerTablePressures) * (upperTableTemps - lowerTableTemps) / (upperTablePressures - lowerTablePressures) ) # Voltages across R_AB [V] V_RStandard = ArrayOfErr([2.016, 2.016, 2.020, 2.017, 2.021, 2.019], 0.001) R = 100.2 # standard resistor value [Ohm] I = V_RStandard / R # current [A] # Voltages across Germanium resistor [V] V_RAB = array([Err(6.167, 0.002), Err(6.168, 0.002), Err(6.170, 0.002), Err(6.160, 0.002), Err(6.153, 0.02), Err(5.894, 0.01)], PyObject) R_AB = V_RAB / I logRAB = ApplyUfuncToErr(R_AB, log10) # This means log10(R_AB) sqrtLogROnT = (logRAB/temps)**0.5 g = Gnuplot.Gnuplot(debug=1) g.title('Allen-Bradley model log(R)=A+B[log(R)/T]^1/2') g.ylabel('log(R)') g.xlabel('[log(R)/T]^1/2') g('set bar 1') # set bar end length g('set grid') # spline fit x1 = PrimeVals(sqrtLogROnT)[0] x2 = PrimeVals(sqrtLogROnT)[5] splineX = arange(x1,x2+.005,.01) exactSpline = splrep(PrimeVals(sqrtLogROnT), PrimeVals(logRAB), s=0) exactSplineY = splev(splineX, exactSpline) g.plot(\ Gnuplot.Data(PrimeVals(sqrtLogROnT), PrimeVals(logRAB), inline=0, with='linesp lt 1 pt 0'), Gnuplot.Data(splineX, exactSplineY, inline=0, with='linesp lt 2 pt 0'), # exact spline Gnuplot.Data(PrimeVals(sqrtLogROnT), # x-points PrimeVals(logRAB), # y-points MinVals(sqrtLogROnT), MaxVals(sqrtLogROnT), # x-axis error bars MinVals(logRAB), MaxVals(logRAB), # y-axis error bars inline=0, with='xyerrorbars -1 0'), ) raw_input('Gary says copy to clipboard now\n') -- ErrorVal.py: from Numeric import * class Err: def __init__(self, value, posErr=0., negErr=None): self.value = float(value) self.posErr = float(posErr) if negErr: self.negErr = float(negErr) else: self.negErr = float(posErr) def __repr__(self): return "Err(%s,%s,%s)" % (self.value, self.posErr, self.negErr) def __str__(self): return "%s +%s/-%s" % (self.value, self.posErr, self.negErr) def __abs__(self): value = abs(self.value) if value == self.value: posErr = self.posErr negErr = self.negErr else: negErr = self.posErr posErr = self.negErr return Err(value, posErr, negErr) def __neg__(self): return Err(-self.value, self.negErr, self.posErr) def __pos__(self): return Err(self.value, self.posErr, self.negErr) def __add__(self, other): value = self.value + other.value posErr = self.posErr + other.posErr negErr = self.negErr + other.negErr return Err(value, posErr, negErr) def __radd__(self, other): value = self.value + other.value posErr = self.posErr + other.posErr negErr = self.negErr + other.negErr return Err(value, posErr, negErr) def __sub__(self, other): value = self.value - other.value posErr = self.posErr + other.negErr negErr = self.negErr + other.posErr return Err(value, posErr, negErr) def __rsub__(self, other): value = other.value - self.value posErr = other.posErr + self.negErr negErr = other.negErr + self.posErr return Err(value, posErr, negErr) def __mul__(self, other): value = self.value * other.value posErr = value * ((self.posErr/self.value) + (other.posErr/other.value)) negErr = value * ((self.negErr/self.value) + (other.negErr/other.value)) return Err(value, posErr, negErr) def __rmul__(self, other): value = self.value * other.value posErr = value * ((self.posErr/self.value) + (other.posErr/other.value)) negErr = value * ((self.negErr/self.value) + (other.negErr/other.value)) return Err(value, posErr, negErr) def __div__(self, other): value = self.value / other.value posErr = value * ((self.posErr/self.value) + (other.posErr/other.value)) negErr = value * ((self.negErr/self.value) + (other.negErr/other.value)) return Err(value, posErr, negErr) def __rdiv__(self, other): value = other.value / self.value posErr = value * ((self.posErr/self.value) + (other.posErr/other.value)) negErr = value * ((self.negErr/self.value) + (other.negErr/other.value)) return Err(value, posErr, negErr) def __pow__(self, y): value = self.value**y.value posErr = (self.value + self.posErr)**(y.value + y.posErr) - value negErr = value - (self.value - self.negErr)**(y.value - y.negErr) return Err(value, posErr, negErr) def __rpow__(self, y): value = y.value**self.value posErr = (y.value + y.posErr)**(self.value + self.posErr) - value negErr = value - (y.value - y.negErr)**(self.value - self.negErr) return Err(value, posErr, negErr) def __coerce__(self, other): if isinstance(other, Err): return self, other try: return self, Err(float(other)) except ValueError: pass def ApplyFunc(self, func): value = func(self.value) posErr = func(self.value + self.posErr) - value negErr = value - func(self.value - self.negErr) return Err(value, posErr, negErr) def ArrayOfErr(errArray, posErr=0., negErr=None): ''' Builds a Numeric array of Err() objects given an array and a +ve or both a +ve and -ve errorval Example: egErr = ArrayOfErr([909., 802., 677., 585., 560., 548.], 1.0) eg2Err = ArrayOfErr([909., 802., 677., 585., 560., 548.], 1.0, 1.5) ''' valArray = asarray(errArray) arrayLength = valArray.shape[0] errs = repeat(array([Err(0.0, posErr, negErr)], PyObject), arrayLength) return valArray + errs def PrimeVals(errArray): ''' Extract a Numeric array with just the prime value of the Err() objects ''' li = [] for i in errArray: li.append(i.value) return asarray(li) def PosErrs(errArray): ''' Return a Numeric array with the posErr values of the Err() objects ''' li = [] for i in errArray: li.append(i.posErr) return asarray(li) def MaxVals(errArray): ''' Return a Numeric array with the prime+posErr values of the Err() objects ''' li = [] for i in errArray: li.append(i.value + i.posErr) return asarray(li) def NegErrs(errArray): ''' Return a Numeric array with the negErr values of the Err() objects ''' li = [] for i in errArray: li.append(i.negErr) return asarray(li) def MinVals(errArray): ''' Return a Numeric array with the prime-negArr values of the Err() objects ''' li = [] for i in errArray: li.append(i.value - i.negErr) return asarray(li) def ErrToFloatArrays(errArray): ''' Return a Numeric array with 3 rows: row 1: the prime value of the Err() objects row 2: the prime+posErr values of the Err() objects row 3: the prime-negErr values of the Err() objects ''' r1 = PrimeVals(errArray) r2 = MaxVals(errArray) r3 = MinVals(errArray) return array([r1, r2, r3]) def FloatToErrArrays(floatArray): ''' Convert a Numeric array of the form produced by ErrToFloatArrays, which contains 3 rows, to an errArray ''' r1 = floatArray[0] r2 = floatArray[1]-r1 r3 = r1-floatArray[2] return asarray(map(Err,r1,r2,r3), PyObject) def ApplyUfuncToErr(errArray, func): ''' Example: a = ArrayOfErr([99., 82., 67., 58., 50., 54.], 1.0) b = ApplyUfuncToErr(a, log) # This means log(a) ''' a = ErrToFloatArrays(errArray) b = func(a) c = FloatToErrArrays(b) return c -- -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm From gillet at scripps.edu Tue Mar 16 18:33:41 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Tue, 16 Mar 2004 15:33:41 -0800 Subject: [SciPy-dev] problem building scipy on win32 Message-ID: <40578ED5.3040502@scripps.edu> > > >n Wed, 10 Mar 2004, Alexandre Gillet wrote: > >>/ Hi, >/>/ I am trying to build SciPy-0.2.0_alpha_200.4161. >/>/ I did follow the note from Pearu Peterson >/>/ (http://cens.ioc.ee/~pearu/scipy/BUILD_WIN32.html ) >/>/ I am not using the cvs version, but I did use the cvs version of >/>/ scipy_core for installing it. >/>/ Then I cd in SciPy-0.2.0_alpha_200.4161 and type msys.py python setup.py >/>/ build. >/>/ >/>/ I am now getting the following error: >/>/ >/>/ /building 'scipy_base.fastumath' extension >/>/ Traceback (most recent call last): >/>/ File "setup.py", line 111, in ? >/>/ setup_package() >/>/ File "setup.py", line 101, in setup_package >/>/ url = "http://www.scipy.org", >/>/ File "scipy_core\scipy_distutils\core.py", line 42, in setup >/>/ return old_setup(**new_attr) >/>/ File "c:\Python23\lib\distutils\core.py", line 149, in setup >/>/ dist.run_commands() >/>/ File "c:\Python23\lib\distutils\dist.py", line 907, in run_commands >/>/ self.run_command(cmd) >/>/ File "c:\Python23\lib\distutils\dist.py", line 927, in run_command >/>/ cmd_obj.run() >/>/ File "c:\Python23\lib\distutils\command\build.py", line 107, in run >/>/ self.run_command(cmd_name) >/>/ File "c:\Python23\lib\distutils\cmd.py", line 333, in run_command >/>/ self.distribution.run_command(command) >/>/ File "c:\Python23\lib\distutils\dist.py", line 927, in run_command >/>/ cmd_obj.run() >/>/ File "c:\Python23\lib\distutils\command\build_ext.py", line 269, in run >/>/ self.build_extensions() >/>/ File "c:\Python23\lib\distutils\command\build_ext.py", line 395, in >/>/ build_extensions >/>/ self.build_extension(ext) >/>/ File "scipy_core\scipy_distutils\command\build_ext.py", line 116, in >/>/ build_extension >/>/ res = old_build_ext.build_extension(self,ext) >/>/ File "c:\Python23\lib\distutils\command\build_ext.py", line 492, in >/>/ build_extension >/>/ target_lang=language) >/>/ File "c:\Python23\lib\distutils\ccompiler.py", line 843, in >/>/ link_shared_object >/>/ extra_preargs, extra_postargs, build_temp, target_lang) >/>/ TypeError: link() takes exactly 13 arguments (14 given) >/ >That error is due to the bug in Python 2.3 distutils. scipy_distutils >implements a workaround to this issue but the workaround is effective only >when setup scripts use scipy_distutils, the setup scripts in >SciPy-0.2.0_alpha_200.4161 apparently do not do that. > >>/ Any Idea. >/>/ I need to use that version of Scipy as the GA module seems not to work >/>/ on later version, that version was installed on Linux and works fine for us. >/ >It might be easier to fix whatever issues you have with the recent GA >module rather than trying to get SciPy-0.2.0_alpha_200.4161 working. What >problems exactly do you experience with the GA module? > >Pearu > Thanks for the answer, I am only trying to install Scipy and I am not using the GA module,I will ask the person in my lab who used the GA to provide me with the exact problem. And I will get back to you as soon as I have the information. Alex -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 From pearu at scipy.org Wed Mar 17 01:50:51 2004 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 17 Mar 2004 00:50:51 -0600 (CST) Subject: [SciPy-dev] problem building scipy on win32 In-Reply-To: <40578ED5.3040502@scripps.edu> Message-ID: On Tue, 16 Mar 2004, Alexandre Gillet wrote: > Thanks for the answer, I am only trying to install Scipy and I am not > using the GA module,I will ask the person in my lab who used the GA to > provide me with the exact problem. And I will get back to you as soon as > I have the information. In case you don't need GA then you can disable building it in setup.py file by adding 'ga' to the `ignore_packages` list. Regards, Pearu From arnd.baecker at web.de Wed Mar 17 07:15:54 2004 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed, 17 Mar 2004 13:15:54 +0100 (CET) Subject: [SciPy-dev] system_info.py, blas_opt_info broken ? Message-ID: Hi, I just downloaded the CVS version and doing cd scipy_core/scipy_distutils python system_info.py gives blas_opt_info: Traceback (most recent call last): File "system_info.py", line 1263, in ? show_all() File "system_info.py", line 1260, in show_all r = c.get_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 283, in get_info self.calc_info() File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 934, in calc_info atlas_version = get_atlas_version(**version_info) File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", line 775, in get_atlas_version from core import Extension, setup File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/core.py", line 5, in ? from scipy_distutils.dist import Distribution ImportError: No module named scipy_distutils.dist The same happens for lapack_opt_info. One more: can one safely ignore ====================================================================== FAIL: check_real (scipy_base.type_check.test_type_check.test_imag) ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/python/lib/python2.3/site-packages/scipy_base/tests/test_type_check.py", line 31, in check_real assert_array_equal(0,imag(y)) File "/opt/python/lib/python2.3/site-packages/scipy_test/testing.py", line 641, in assert_array_equal assert len(shape(x))==len(shape(y)) and \ AssertionError: Arrays are not equal (shapes (), (10,) mismatch): ? Many thanks, Arnd From pearu at scipy.org Wed Mar 17 07:38:06 2004 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 17 Mar 2004 06:38:06 -0600 (CST) Subject: [SciPy-dev] system_info.py, blas_opt_info broken ? In-Reply-To: Message-ID: On Wed, 17 Mar 2004, Arnd Baecker wrote: > I just downloaded the CVS version and doing > cd scipy_core/scipy_distutils > python system_info.py > gives > > blas_opt_info: > Traceback (most recent call last): > File "system_info.py", line 1263, in ? > show_all() > File "system_info.py", line 1260, in show_all > r = c.get_info() > File > "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", > line 283, in get_info > self.calc_info() > File > "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", > line 934, in calc_info > atlas_version = get_atlas_version(**version_info) > File > "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/system_info.py", > line 775, in get_atlas_version > from core import Extension, setup > File "/tmp/baecker/CompileDir/scipy/scipy_core/scipy_distutils/core.py", > line 5, in ? > from scipy_distutils.dist import Distribution > ImportError: No module named scipy_distutils.dist At the moment it is not possible to execute python system_info.py until scipy_distutils is installed first. The reason is that system_info tries to compile an extension module to determine the version of ATLAS, but do to that, it needs scipy_distutils. One option would be to disable ATLAS version detection if scipy_distutils is not installed, but then again the output of `python system_info.py` would not reflect the real state of the system. So, install scipy_core first (or put it's location to PYTHONPATH) and then run system_info.py. > The same happens for lapack_opt_info. > > One more: can one safely ignore > ====================================================================== > FAIL: check_real (scipy_base.type_check.test_type_check.test_imag) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/opt/python/lib/python2.3/site-packages/scipy_base/tests/test_type_check.py", > line 31, in check_real > assert_array_equal(0,imag(y)) > File "/opt/python/lib/python2.3/site-packages/scipy_test/testing.py", > line 641, in assert_array_equal > assert len(shape(x))==len(shape(y)) and \ > AssertionError: > Arrays are not equal (shapes (), (10,) mismatch): > > ? Yes, you can ignore it. It's solely an issue of testing hooks and will be fixed as soon as I get a moment for it. Pearu From arnd.baecker at web.de Fri Mar 19 06:46:11 2004 From: arnd.baecker at web.de (Arnd Baecker) Date: Fri, 19 Mar 2004 12:46:11 +0100 (CET) Subject: [SciPy-dev] help and delayed import (once more) Message-ID: Hi, I hope I am not annoying you with this, but there is one open issue with the delayed importer: Starting in a fresh python session >>> help("scipy.linalg") gives Help on instance of _ModuleLoader in scipy: scipy.linalg = Whereas >>> import scipy >>> help("scipy.linalg") is okay. Though in most cases the user will already have done an import of scipy or one of its modules, the above behaviour might be confusing. (Is it also problematic for doc-string extracting tools ?) There is one more thing I would like to say: _Many_ thanks to you, Pearu and Travis, for all your help and fixes! Best Arnd From arnd.baecker at web.de Fri Mar 19 06:47:36 2004 From: arnd.baecker at web.de (Arnd Baecker) Date: Fri, 19 Mar 2004 12:47:36 +0100 (CET) Subject: [SciPy-dev] scipy.xplt keywords / ghelp Message-ID: Hi, this is a follow-up to the discussion on ghelp and the keywors for the xplt commands. - In order to have the keyword help available in a doc-string I have added these for the course at the end of info_xplt.py If you want to include this in the general distribution I can send you the corresponding text (which was taken from http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/pygist_html/node57.html and formatted properly. - Side remark: ghelp has not been added back into scipy.xplt. Personally I think this good, because there are already to many different help commands. - the help to scipy.xplt starts with """All Gist functions are also available as xplt. xplt.ghelp('plsys')""" I think the "xplt.ghelp('plsys')" is left over and should be removed. Best, Arnd From hoel at gl-group.com Mon Mar 22 03:42:41 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Mon, 22 Mar 2004 09:42:41 +0100 Subject: [SciPy-dev] bug in scipy distutils / Scipy_core-0.2.2_alpha_104.1640 Message-ID: Hello, I installed Scipy_core-0.2.2_alpha_104.1640. Now trying to build a swig package I get: File "/usr/local/gltools/python/Python-2.3/linux/lib/python2.3/site-packages/scipy_distutils/command/build_src.py", line 311, in swig_sources assert name==ext_name[1:],'mismatch of extension names: '\ AssertionError: mismatch of extension names: lib/Metis/metis.i provides 'Metis_Wrapper' but expected 'etis_Wrapper' on the lines: // File : metis.i %module Metis_Wrapper changing line 311 of build_src.py to assert name==ext_name,'mismatch of extension names: '\ works for me Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Tue Mar 23 11:06:08 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Tue, 23 Mar 2004 17:06:08 +0100 Subject: [SciPy-dev] Further extensions for "fcompiler.py"? Message-ID: Hello, Some of my Fortran source files for an extension module library require pre processing. Ifc for Linux uses fpp for files with the extension ".F" or ".F90" instead of ".f" and ".f90". Therefore I would like to find the list "src_extensions" extended by these extensions. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Wed Mar 24 11:11:08 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Wed, 24 Mar 2004 17:11:08 +0100 Subject: [SciPy-dev] scipy_distutils under cygwin using DF Message-ID: Hello, I try to compile some python extension libraries under WinXP/cygwin. The fortran codes need DF, no chance for g77. But I get: >make test PYTHONPATH="w:\\hoel\\work\\GLPy\\lib;w:\\hoel\\work\\GLPy\\build\\lib.win32-2.3;" python setup.py config_fc --fcompiler=compaqv build_src build_clib build_ext ... customize CompaqVisualFCompiler customize CompaqVisualFCompiler using build_clib splitcmdline('DF /what') ******************************************************************************** scipy_distutils.compaqfcompiler.CompaqVisualFCompiler version_cmd = ['DF', '/what'] compiler_fix = ['DF', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/n ames:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fa st', '/threads'] ranlib = None archiver = ['C:\\Programme\\Microsoft Visual Studio\\VC98\\BIN\\lib.exe', '/OUT:'] compiler_f77 = ['DF', '/f77rtl', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len _arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math _library:fast', '/threads'] linker_so = ['DF'] compiler_f90 = ['DF', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lower case', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/thr eads'] version = None libraries = [] library_dirs = [] object_switch = '/object:' compile_switch = '/compile_only ' include_dirs = [] ******************************************************************************** building 'fortmisc' library compling Fortran sources DF(f77) options: '/f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase/assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads' DF(f90) options: '/nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads' DF(fix) options: '/fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads' compile options: '-Ilib\numfort -c' DF:f77: lib\numfort\NB.F DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c "/compile_only " lib\numfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o Microsoft: error: Unknown switch: '/compile_only ' error: Command "DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c "/compile_only " lib\numfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o" failed with exit status 1 make: *** [build] Error 1 changing "compile_switch" in fcompiler.py from '/compile_only ' to '/compile_only' solves this problem, but now I get: DF:f77: lib\numfort\NB.F DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:undersc ore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /compile_only lib\n umfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o Visual lib\numfort\NB.F f90: Severe: No such file or directory ... file is 'Visual' error: Command "DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /co mpile_only lib\numfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o" failed with exit status 1 make: *** [build] Error 1 I have no idea where the "Visual" is coming from. Is there anyone who can help me? Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at scipy.org Wed Mar 24 14:38:50 2004 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 24 Mar 2004 13:38:50 -0600 (CST) Subject: [SciPy-dev] scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Wed, 24 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > Hello, > > I try to compile some python extension libraries under > WinXP/cygwin. The fortran codes need DF, no chance for g77. But I get: > > changing "compile_switch" in fcompiler.py from '/compile_only ' to > '/compile_only' solves this problem, but now I get: Ok, thanks for this information. This is now also in CVS. > DF:f77: lib\numfort\NB.F > DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:undersc > ore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /compile_only lib\n > umfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o > Visual > lib\numfort\NB.F > f90: Severe: No such file or directory > ... file is 'Visual' > error: Command "DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase > /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /co > mpile_only lib\numfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o" failed with exit status 1 > > make: *** [build] Error 1 > > I have no idea where the "Visual" is coming from. Is there anyone who > can help me? Yes, the current scipy_distutils.exec_command had issues on Windows when a the full path of an executable (like of DF) contains spaces. This is now fixed in CVS. Pearu From hoel at gl-group.com Thu Mar 25 08:41:52 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Thu, 25 Mar 2004 14:41:52 +0100 Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: (Pearu Peterson's message of "Wed, 24 Mar 2004 13:38:50 -0600 (CST)") References: Message-ID: Pearu Peterson writes: > On Wed, 24 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> Hello, >> >> I try to compile some python extension libraries under >> WinXP/cygwin. The fortran codes need DF, no chance for g77. But I get: >> >> changing "compile_switch" in fcompiler.py from '/compile_only ' to >> '/compile_only' solves this problem, but now I get: > > Ok, thanks for this information. This is now also in CVS. You are welcome. > > >> DF:f77: lib\numfort\NB.F >> DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase /assume:undersc >> ore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /compile_only lib\n >> umfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o >> Visual >> lib\numfort\NB.F >> f90: Severe: No such file or directory >> ... file is 'Visual' >> error: Command "DF /f77rtl /fixed /nologo /MD /WX /iface=(cref,nomixed_str_len_arg) /names:lowercase >> /assume:underscore /Ox /fast /optimize:5 /unroll:0 /math_library:fast /threads -Ilib\numfort -c /co >> mpile_only lib\numfort\NB.F /object:build\temp.win32-2.3\lib\numfort\NB.o" failed with exit status 1 >> >> make: *** [build] Error 1 >> >> I have no idea where the "Visual" is coming from. Is there anyone who >> can help me? > > Yes, the current scipy_distutils.exec_command had issues on Windows when a > the full path of an executable (like of DF) contains spaces. This is now > fixed in CVS. Thanks a lot, the fortran compilation works now, but! I can't compile C source files anymore: ******************************************************************************** scipy_distutils.compaqfcompiler.CompaqVisualFCompiler version_cmd = ['DF', '/what'] compiler_fix = ['DF', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] ranlib = None archiver = ['C:\\Programme\\Microsoft Visual Studio\\VC98\\BIN\\lib.exe', '/OUT:'] compiler_f77 = ['DF', '/f77rtl', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] linker_so = ['DF'] compiler_f90 = ['DF', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] version = LooseVersion ('6.1') libraries = [] library_dirs = ['c:\\Python23\\libs', 'c:\\Python23\\PCBuild', 'build\\temp.win32-2.3'] object_switch = '/object:' compile_switch = '/compile_only' include_dirs = ['c:\\Python23\\include', 'c:\\Python23\\PC'] ******************************************************************************** building 'SXFPyBase' extension compling C sources C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include-Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj" failed with exit status 1 Is this a related problem, than can be solved equally fast? Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at cens.ioc.ee Thu Mar 25 10:29:29 2004 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 25 Mar 2004 17:29:29 +0200 (EET) Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Thu, 25 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > Thanks a lot, the fortran compilation works now, but! I can't compile > C source files anymore: > > ******************************************************************************** > scipy_distutils.compaqfcompiler.CompaqVisualFCompiler > version_cmd = ['DF', '/what'] > compiler_fix = ['DF', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] > ranlib = None > archiver = ['C:\\Programme\\Microsoft Visual Studio\\VC98\\BIN\\lib.exe', '/OUT:'] > compiler_f77 = ['DF', '/f77rtl', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] > linker_so = ['DF'] > compiler_f90 = ['DF', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] > version = LooseVersion ('6.1') > libraries = [] > library_dirs = ['c:\\Python23\\libs', 'c:\\Python23\\PCBuild', 'build\\temp.win32-2.3'] > object_switch = '/object:' > compile_switch = '/compile_only' > include_dirs = ['c:\\Python23\\include', 'c:\\Python23\\PC'] > ******************************************************************************** > building 'SXFPyBase' extension > compling C sources > C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include-Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj > Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" > Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist > "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj > > error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj" failed with exit status 1 > > Is this a related problem, than can be solved equally fast? Yes, it appears that path to a C compiler is double quoted. This is now fixed in CVS. Thanks, Pearu From hoel at gl-group.com Fri Mar 26 04:02:07 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Fri, 26 Mar 2004 10:02:07 +0100 Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: (Pearu Peterson's message of "Thu, 25 Mar 2004 17:29:29 +0200 (EET)") References: Message-ID: Pearu Peterson writes: > On Thu, 25 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> Thanks a lot, the fortran compilation works now, but! I can't compile >> C source files anymore: ... >> Is this a related problem, than can be solved equally fast? > > Yes, it appears that path to a C compiler is double quoted. This is now > fixed in CVS. I did a "cvs update" in scipy and "python setup.py install" in scipy_core but still get ******************************************************************************** scipy_distutils.compaqfcompiler.CompaqVisualFCompiler version_cmd = ['DF', '/what'] compiler_fix = ['DF', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] ranlib = None archiver = ['C:\\Programme\\Microsoft Visual Studio\\VC98\\BIN\\lib.exe', '/OUT:'] compiler_f77 = ['DF', '/f77rtl', '/fixed', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] linker_so = ['DF'] compiler_f90 = ['DF', '/nologo', '/MD', '/WX', '/iface=(cref,nomixed_str_len_arg)', '/names:lowercase', '/assume:underscore', '/Ox', '/fast', '/optimize:5', '/unroll:0', '/math_library:fast', '/threads'] version = None libraries = [] library_dirs = ['c:\\Python23\\libs', 'c:\\Python23\\PCBuild', 'build\\temp.win32-2.3'] object_switch = '/object:' compile_switch = '/compile_only' include_dirs = ['c:\\Python23\\include', 'c:\\Python23\\PC'] ******************************************************************************** building 'SXFPyBase' extension compling C sources C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj" failed with exit status 1 make: *** [build] Error 1 My cygwin does find the DF executable: hoel at PC021358 ~/work/GLPy/test $ DF /what Compaq Visual Fortran Optimizing Compiler Version 6.1 (Update A) Copyright 2000 Compaq Computer Corp. All rights reserved. Compaq Visual Fortran 6.1-970-42A1L c:\Programme\Microsoft Visual Studio\DF98\bin\decfor90.exe hoel at PC021358 ~/work/GLPy/test $ which DF /cygdrive/c/Programme/Microsoft Visual Studio/DF98/bin/DF -- Mit freundlichen Gr??en Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Fri Mar 26 04:15:02 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Fri, 26 Mar 2004 10:15:02 +0100 Subject: [SciPy-dev] scipy distutils from CVS does not build code for fortran libraries Message-ID: Hello, I tried to compile a project of ours under solaris5. I gave PYTHONPATH="/home/hoel/work/GLPy/lib:/home/hoel/work/GLPy/build/lib.solaris-2.8-sun4u-2.2:" python setup_Engmodel.py config_fc --fcompiler=sun build_src build_clib build_ext ... customize SunFCompiler customize SunFCompiler using build_clib ******************************************************************************** scipy_distutils.sunfcompiler.SunFCompiler archiver = ['ar', '-cr'] version_cmd = ['f90', '-V'] compiler_f77 = ['f90', '-f77', '-ftrap=%none', '-xcode=pic32'] linker_so = ['f90', '-Bdynamic', '-G'] compiler_fix = ['f90', '-fixed', '-xcode=pic32'] ranlib = ['ranlib'] compiler_f90 = ['f90', '-xcode=pic32'] version = None libraries = ['fsu', 'sunmath', 'mvec', 'f77compat'] library_dirs = [] object_switch = '-o ' compile_switch = '-c' include_dirs = [] ******************************************************************************** building 'engforcegen' library compling Fortran sources creating build.Engmodel/temp.solaris-2.8-sun4u-2.2 creating build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib creating build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen f90:f77: lib/engforcegen/engforcelib.f f90:f77: lib/engforcegen/totalforces.f f90:f77: lib/engforcegen/foucoefftable.f f90:f77: lib/engforcegen/esrheader.f f90:f77: lib/engforcegen/pphase.f f90:f77: lib/engforcegen/excnodes.f f90:f77: lib/engforcegen/fourier.f f90:f77: lib/engforcegen/cylphase.f f90:f77: lib/engforcegen/esrdata.f f90:f77: lib/engforcegen/normangle.f f90:f77: lib/engforcegen/cylforces.f ar: adding 11 object files to build.Engmodel/temp.solaris-2.8-sun4u-2.2/libengforcegen.a error: Command "ar -cr build.Engmodel/temp.solaris-2.8-sun4u-2.2/libengforcegen.a build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/engforcelib.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylphase.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/esrdata.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/esrheader.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/excnodes.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/foucoefftable.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/fourier.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/pphase.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/totalforces.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/normangle.o" failed with exit status 1 all the ".o" files are missing. Just another problem with "exec_command"? Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Fri Mar 26 04:56:46 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Fri, 26 Mar 2004 10:56:46 +0100 Subject: [SciPy-dev] Log problem in exec_command.py Message-ID: Hello, In an attempt to debug my Solaris compile problem I added from scipy_distutils.log import set_verbosity set_verbosity(2) to my setup.py file. Now I get ... File "/usr/local/gltools/python/Python-2.2.1/sol2/lib/python2.2/site-packages/scipy_distutils/exec_command.py", line 186, in exec_command ','.join(['%s=%r'%kv for kv in env.items()]))) File "", line 23, in debug File "", line 16, in _log TypeError: not enough arguments for format string The problem is one of the arguments to the sun FORTRAN compiler. log gets exec_command(['f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/engforcegen/engforcelib.f', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/engforcelib.o'],) The problem seems to be in the '-ftrap=%none'. log() does string replacement. My solution is to change lines 186/7 to log.debug(('exec_command(%r,%s)' % (command,\ ','.join(['%s=%r'%kv for kv in env.items()]))).replace("%", "%%")) and lines 386/7 to log.debug(('Running %s(%s,%r,%r,os.environ)' \ % (spawn_command.__name__,os.P_WAIT,argv[0],argv)).replace("%", "%%")) Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Fri Mar 26 06:18:53 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Fri, 26 Mar 2004 12:18:53 +0100 Subject: [SciPy-dev] Re: scipy distutils from CVS does not build code for fortran libraries In-Reply-To: (Berthold =?iso-8859-1?q?H=F6llmann's?= message of "Fri, 26 Mar 2004 10:15:02 +0100") References: Message-ID: Berthold H?llmann writes: > Hello, > > I tried to compile a project of ours under solaris5. I gave > ... > all the ".o" files are missing. Just another problem with "exec_command"? what version of the sun FORTRAN compiler does the "config_fc --fcompiler=sun" switch expect. We have f90 -V f90: Sun WorkShop 6 update 2 Fortran 95 6.2 Patch 111690-07 2002/06/26 Usage: f90 [ options ] files. Use 'f90 -flags' for details installed. It does not know the "-f77" switch. It turns out, my problem is the "/bin/bash" wrapper around the f90 call. Why is exec_command.py running spawnvpe(0,'/bin/sh',['/bin/sh', '-c', 'f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/numfort/NB.F', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/numfort/NB.o'],os.environ) instead of spawnvpe(0,'f90',['f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/numfort/NB.F', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/numfort/NB.o'],os.environ) ? My problem can be shown with hoel at donau:test /bin/sh -c "f90 -f77 -ftrap=%none -xcode=pic32 -c lib/engforcegen/cylforces.f -o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o" f90: Warning: Option -f77 passed to ld, if ld is invoked, ignored otherwise ERROR: Cannot open source file "lib/engforcegen/cylforces.f". hoel at donau:test /bin/sh -c f90 -f77 -ftrap=%none -xcode=pic32 -c lib/engforcegen/cylforces.f -o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o Usage: f90 [ options ] files. Use 'f90 -flags' for details hoel at donau:test echo $? 0 "-c" requires the command to be called as string intead of a tuple/list of arguments. I changed lines 348-351 to: if type(command) is type([]): argv = [sh,'-c', " ".join(command)] else: argv = [sh,'-c',command] and it now seems to work for me. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at scipy.org Fri Mar 26 10:45:19 2004 From: pearu at scipy.org (Pearu Peterson) Date: Fri, 26 Mar 2004 09:45:19 -0600 (CST) Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Fri, 26 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > I did a "cvs update" in scipy and "python setup.py install" in > scipy_core but still get > > compling C sources > C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj > Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" > Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist > "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj > > error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\lib\SXFPyBase\SXFPyBasemodule.c /Fobuild\temp.win32-2.3\Release\build\src\lib\SXFPyBase\SXFPyBasemodule.obj" failed with exit status 1 > make: *** [build] Error 1 Hmm, not sure what is going on here. Could you try running the exec_command.py tests? Just execute python exec_command.py in scipy_distutils/ directory. There is a function test_svn() that I used to test exec_command on commands having spaces in their paths (`svn` just happened to be such a one in my system). You can play with python -c "from exec_command import *;exec_command(['DF','/what'])" and the code starting at line #397 in exec_command.py to find out what is going on. > My cygwin does find the DF executable: > > hoel at PC021358 ~/work/GLPy/test > $ DF /what > Compaq Visual Fortran Optimizing Compiler Version 6.1 (Update A) > Copyright 2000 Compaq Computer Corp. All rights reserved. > > Compaq Visual Fortran 6.1-970-42A1L > c:\Programme\Microsoft Visual Studio\DF98\bin\decfor90.exe > > hoel at PC021358 ~/work/GLPy/test > $ which DF > /cygdrive/c/Programme/Microsoft Visual Studio/DF98/bin/DF That's interesting. Try to replace the following line return exe in find_executable() with return os.path.realpath(exe) HTH, Pearu From pearu at scipy.org Fri Mar 26 10:53:58 2004 From: pearu at scipy.org (Pearu Peterson) Date: Fri, 26 Mar 2004 09:53:58 -0600 (CST) Subject: [SciPy-dev] scipy distutils from CVS does not build code for fortran libraries In-Reply-To: Message-ID: On Fri, 26 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > Hello, > > I tried to compile a project of ours under solaris5. I gave > > PYTHONPATH="/home/hoel/work/GLPy/lib:/home/hoel/work/GLPy/build/lib.solaris-2.8-sun4u-2.2:" python setup_Engmodel.py config_fc --fcompiler=sun build_src build_clib build_ext > ... > customize SunFCompiler > customize SunFCompiler using build_clib > ******************************************************************************** > scipy_distutils.sunfcompiler.SunFCompiler > archiver = ['ar', '-cr'] > version_cmd = ['f90', '-V'] > compiler_f77 = ['f90', '-f77', '-ftrap=%none', '-xcode=pic32'] > linker_so = ['f90', '-Bdynamic', '-G'] > compiler_fix = ['f90', '-fixed', '-xcode=pic32'] > ranlib = ['ranlib'] > compiler_f90 = ['f90', '-xcode=pic32'] > version = None > libraries = ['fsu', 'sunmath', 'mvec', 'f77compat'] > library_dirs = [] > object_switch = '-o ' > compile_switch = '-c' > include_dirs = [] > ******************************************************************************** > building 'engforcegen' library > compling Fortran sources > creating build.Engmodel/temp.solaris-2.8-sun4u-2.2 > creating build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib > creating build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen > f90:f77: lib/engforcegen/engforcelib.f > f90:f77: lib/engforcegen/totalforces.f > f90:f77: lib/engforcegen/foucoefftable.f > f90:f77: lib/engforcegen/esrheader.f > f90:f77: lib/engforcegen/pphase.f > f90:f77: lib/engforcegen/excnodes.f > f90:f77: lib/engforcegen/fourier.f > f90:f77: lib/engforcegen/cylphase.f > f90:f77: lib/engforcegen/esrdata.f > f90:f77: lib/engforcegen/normangle.f > f90:f77: lib/engforcegen/cylforces.f > ar: adding 11 object files to build.Engmodel/temp.solaris-2.8-sun4u-2.2/libengforcegen.a > error: Command "ar -cr build.Engmodel/temp.solaris-2.8-sun4u-2.2/libengforcegen.a build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/engforcelib.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylphase.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/esrdata.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/esrheader.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/excnodes.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/foucoefftable.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/fourier.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/pphase.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/totalforces.o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/normangle.o" failed with exit status 1 > > all the ".o" files are missing. Just another problem with "exec_command"? No, I would not think so. On linux building scipy works fine with a recent CVS copy. So, could you find out where these ".o" files end up? Btw, using PYTHONPATH=.... python setup_Engmodel.py config_fc --fcompiler=sun build should work just fine (I mean instead of 'build_src ...'). Pearu From pearu at scipy.org Fri Mar 26 11:19:34 2004 From: pearu at scipy.org (Pearu Peterson) Date: Fri, 26 Mar 2004 10:19:34 -0600 (CST) Subject: [SciPy-dev] Log problem in exec_command.py In-Reply-To: Message-ID: On Fri, 26 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > Hello, > > In an attempt to debug my Solaris compile problem I added > > from scipy_distutils.log import set_verbosity > set_verbosity(2) > > to my setup.py file. Now I get > > ... > File "/usr/local/gltools/python/Python-2.2.1/sol2/lib/python2.2/site-packages/scipy_distutils/exec_command.py", line 186, in exec_command > ','.join(['%s=%r'%kv for kv in env.items()]))) > File "", line 23, in debug > File "", line 16, in _log > TypeError: not enough arguments for format string > > The problem is one of the arguments to the sun FORTRAN compiler. log gets > > exec_command(['f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/engforcegen/engforcelib.f', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/engforcelib.o'],) > > The problem seems to be in the '-ftrap=%none'. log() does string > replacement. My solution is to change lines 186/7 to > > log.debug(('exec_command(%r,%s)' % (command,\ > ','.join(['%s=%r'%kv for kv in env.items()]))).replace("%", "%%")) > > and lines 386/7 to > > log.debug(('Running %s(%s,%r,%r,os.environ)' \ > % (spawn_command.__name__,os.P_WAIT,argv[0],argv)).replace("%", "%%")) Thanks, I have applied a patch to log.py that fixes this bug. Pearu From pearu at scipy.org Fri Mar 26 14:00:55 2004 From: pearu at scipy.org (Pearu Peterson) Date: Fri, 26 Mar 2004 13:00:55 -0600 (CST) Subject: [SciPy-dev] Re: scipy distutils from CVS does not build code for fortran libraries In-Reply-To: Message-ID: On Fri, 26 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > what version of the sun FORTRAN compiler does the > > "config_fc --fcompiler=sun" > > switch expect. We have > > f90 -V > f90: Sun WorkShop 6 update 2 Fortran 95 6.2 Patch 111690-07 2002/06/26 > Usage: f90 [ options ] files. Use 'f90 -flags' for details > > installed. It does not know the "-f77" switch. The sunfcompiler.py is probably tested only for 7.1 version: $ f90 -V f90: Sun Fortran 95 7.1 2003/03/12 Usage: f90 [ options ] files. Use 'f90 -flags' for details I have fixed (hopefully) sunfcompiler.py in CVS for Sun WorkShop 6. > It turns out, my problem is the "/bin/bash" wrapper around the f90 > call. Why is exec_command.py running > > spawnvpe(0,'/bin/sh',['/bin/sh', '-c', 'f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/numfort/NB.F', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/numfort/NB.o'],os.environ) > > instead of > > spawnvpe(0,'f90',['f90', '-f77', '-ftrap=%none', '-xcode=pic32', '-c', 'lib/numfort/NB.F', '-o', 'build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/numfort/NB.o'],os.environ) > > ? My problem can be shown with > > hoel at donau:test /bin/sh -c "f90 -f77 -ftrap=%none -xcode=pic32 -c lib/engforcegen/cylforces.f -o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o" > f90: Warning: Option -f77 passed to ld, if ld is invoked, ignored otherwise > ERROR: Cannot open source file "lib/engforcegen/cylforces.f". > hoel at donau:test /bin/sh -c f90 -f77 -ftrap=%none -xcode=pic32 -c lib/engforcegen/cylforces.f -o build.Engmodel/temp.solaris-2.8-sun4u-2.2/lib/engforcegen/cylforces.o > Usage: f90 [ options ] files. Use 'f90 -flags' for details > hoel at donau:test echo $? > 0 > > "-c" requires the command to be called as string intead of a > tuple/list of arguments. Strange, `man bash` allows `bash -c string arg0 arg1 ..` > I changed lines 348-351 to: > > if type(command) is type([]): > argv = [sh,'-c', " ".join(command)] > else: > argv = [sh,'-c',command] > > and it now seems to work for me. This is now in scipy CVS as well. Thanks, Pearu From hoel at gl-group.com Mon Mar 29 04:10:31 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Mon, 29 Mar 2004 11:10:31 +0200 Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: (Pearu Peterson's message of "Fri, 26 Mar 2004 09:45:19 -0600 (CST)") References: Message-ID: Pearu Peterson writes: > On Fri, 26 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> I did a "cvs update" in scipy and "python setup.py install" in >> scipy_core but still get >> >> make: *** [build] Error 1 > > Hmm, not sure what is going on here. Could you try running the > exec_command.py tests? Just execute > > python exec_command.py hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core/scipy_distutils $ python exec_command.py splitcmdline('a b cc') splitcmdline('a') splitcmdline('a " b cc"') splitcmdline('"a bcc" -h') splitcmdline('"\\"a \\" bcc" -h') splitcmdline(" 'a bcc' -h") splitcmdline("'\\'a \\' bcc' -h") Using cygwin echo in win32 environment is not supported splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('echo path=%path%') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "raise \'Ignore me.\'"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(\'0\');sys.stderr.write(\'1\');sys.stderr.write(\'2\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.exit(15)"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "print \'Heipa\'"') ok Using cygwin echo in win32 environment is not supported splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('echo path=%path%') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "raise \'Ignore me.\'"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(\'0\');sys.stderr.write(\'1\');sys.stderr.write(\'2\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.exit(15)"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "print \'Heipa\'"') ok splitcmdline('c:\\Python23\\PYTHON.EXE -c "print \'Ignore the following IOError:\',open(\'tmp0b66sb\',\'r\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "print open(\'tmp0b66sb\',\'r\').read()"') ok splitcmdline('c:\\Python23\\PYTHON.EXE -c "print \'Ignore the following IOError:\',open(\'tmp0zwgk-\',\'r\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "print open(\'tmp0zwgk-\',\'r\').read()"') ok Could not locate executable svn Executable svn does not exist > > in scipy_distutils/ directory. There is a function test_svn() that I used > to test exec_command on commands having spaces in their paths (`svn` just > happened to be such a one in my system). You can play with > > python -c "from exec_command import *;exec_command(['DF','/what'])" hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core/scipy_distutils $ python -c "from exec_command import * ; exec_command(['DF','/what'])" hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core/scipy_distutils $ python -c "from exec_command import * ; print exec_command(['DF','/what'])" (999, '[Errno 22] Invalid argument: c:\\Programme\\Microsoft Visual Studio\\DF98\\bin\\DF.exe') > > and the code starting at line #397 in exec_command.py to find out what is > going on. > >> My cygwin does find the DF executable: >> >> hoel at PC021358 ~/work/GLPy/test >> $ DF /what >> Compaq Visual Fortran Optimizing Compiler Version 6.1 (Update A) >> Copyright 2000 Compaq Computer Corp. All rights reserved. >> >> Compaq Visual Fortran 6.1-970-42A1L >> c:\Programme\Microsoft Visual Studio\DF98\bin\decfor90.exe >> >> hoel at PC021358 ~/work/GLPy/test >> $ which DF >> /cygdrive/c/Programme/Microsoft Visual Studio/DF98/bin/DF > > That's interesting. Try to replace the following line > > return exe > > in find_executable() with > > return os.path.realpath(exe) after some fiddeling, the following does work for me under Win XP, cygwin, using the Enthought Python. cygwin python does return "posix" for "os.name" so the 'argv = [os.environ['COMSPEC'],'/C']+argv' part could be OK: Index: scipy_core/scipy_distutils/exec_command.py =================================================================== RCS file: /home/cvsroot/world/scipy_core/scipy_distutils/exec_command.py,v retrieving revision 1.17 diff -r1.17 exec_command.py 150c150 < return f_ext --- > return os.path.realpath(f_ext) 153c153 < return exe --- > return os.path.realpath(exe) 364c364 < argv[0] = find_executable(argv[0]) --- > argv[0] = quote_arg(find_executable(argv[0])) 367,370c367,370 < if os.name in ['nt','dos']: < # argv[0] might be internal command < argv = [os.environ['COMSPEC'],'/C']+argv < using_command = 1 --- > if os.name in ['nt','dos']: > # argv[0] might be internal command > argv = [os.environ['COMSPEC'],'/C']+argv > using_command = 1 397,398d396 < argv0 = quote_arg(argv[0]) < 400c398 < status = spawn_command(os.P_WAIT,argv0,argv,os.environ) --- > status = spawn_command(os.P_WAIT,argv[0],argv,os.environ) This version also works for me on Solaris and Linux with Python 2.2.1, and distutils from 2.3.3 replacing the original one. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Mon Mar 29 07:47:56 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Mon, 29 Mar 2004 14:47:56 +0200 Subject: [SciPy-dev] Another issue with scipy_distutils and FORTRAN files Message-ID: Hello, Our FORTRAN code often uses "include" statements. The FORTRAN compiler we use (DF 6.1, ifc 7.1, WorkShop 6 update 2 Fortran) recognize the -I flag for pointing to an include directory. I seems scipy_distutils have lost the ability to regognize the "include_dirs" argument to "Extensions". The compiler does not find the include files anymore. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at scipy.org Mon Mar 29 10:59:44 2004 From: pearu at scipy.org (Pearu Peterson) Date: Mon, 29 Mar 2004 09:59:44 -0600 (CST) Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Mon, 29 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > after some fiddeling, the following does work for me under Win XP, > cygwin, using the Enthought Python. cygwin python does return "posix" > for "os.name" so the 'argv = [os.environ['COMSPEC'],'/C']+argv' part > could be OK: > > Index: scipy_core/scipy_distutils/exec_command.py > =================================================================== > RCS file: /home/cvsroot/world/scipy_core/scipy_distutils/exec_command.py,v > retrieving revision 1.17 > diff -r1.17 exec_command.py Thanks for the patch. I have applied it with few minor changes to CVS. Let me know if my changes did not work. Thanks! Pearu From pearu at scipy.org Mon Mar 29 12:04:22 2004 From: pearu at scipy.org (Pearu Peterson) Date: Mon, 29 Mar 2004 11:04:22 -0600 (CST) Subject: [SciPy-dev] Another issue with scipy_distutils and FORTRAN files In-Reply-To: Message-ID: On Mon, 29 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > Our FORTRAN code often uses "include" statements. The FORTRAN compiler > we use (DF 6.1, ifc 7.1, WorkShop 6 update 2 Fortran) recognize the > -I flag for pointing to an include directory. I seems scipy_distutils > have lost the ability to regognize the "include_dirs" argument to > "Extensions". The compiler does not find the include files anymore. Hmm, I just checked and I could not reproduce this: paths in the include_dirs keyword argument of an Extension class are shown in `compile options:`. Try for example cd scipy_distutils/tests/f2py_f90_ext python setup.py config_fc --fcompiler=intel build after updating/installing scipy_core and f2py from CVS. However, f2py was not able to find the include files. This is now fixed in f2py CVS. Regards, Pearu From hoel at gl-group.com Tue Mar 30 05:41:22 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Tue, 30 Mar 2004 12:41:22 +0200 Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: (Pearu Peterson's message of "Mon, 29 Mar 2004 09:59:44 -0600 (CST)") References: Message-ID: Pearu Peterson writes: > On Mon, 29 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> after some fiddeling, the following does work for me under Win XP, >> cygwin, using the Enthought Python. cygwin python does return "posix" >> for "os.name" so the 'argv = [os.environ['COMSPEC'],'/C']+argv' part >> could be OK: >> >> Index: scipy_core/scipy_distutils/exec_command.py >> =================================================================== >> RCS file: /home/cvsroot/world/scipy_core/scipy_distutils/exec_command.py,v >> retrieving revision 1.17 >> diff -r1.17 exec_command.py > > Thanks for the patch. I have applied it with few minor changes to CVS. > Let me know if my changes did not work. Dear Pearu, There are still problems (a problem) with the patches. I was already im mine, but I had'nt found the time to report them, sorry. The version is working for Linux and Solaris but trying to build scipy_core under WinXP I get: ---snip--- ... compling C sources creating build\temp.win32-2.3 creating build\temp.win32-2.3\Release creating build\temp.win32-2.3\Release\scipy_base C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DUSE_MCONF_LITE_LE -Ic:\Python23\include -Ic:\Python23\PC /Tcscipy_base\fastumathmodule.c /Fobuild\temp.win32-2.3\Release\scipy_base\fastumathmodule.obj Could not locate executable h:\work\CVS\scipy\scipy_core\"C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" Executable h:\work\CVS\scipy\scipy_core\"C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DUSE_MCONF_LITE_LE -Ic:\Python23\include -Ic:\Python23\PC /Tcscipy_base\fastumathmodule.c /Fobuild\temp.win32-2.3\Release\scipy_base\fastumathmodule.obj" failed with exit status 1 ---snip--- "h:\work\CVS\scipy\scipy_core\" is where I checked out scipy from CVS. Further "python scipy_distutils/exec_command.py" fails. Line numbers are not exactely those from CVS because of additional debugging code. ---snip--- s,o=exec_command('echo path=%path%') log.debug((s,o)) assert s==0 and o!='',(s,o) s,o=exec_command('%s -c "import sys;sys.stderr.write(sys.platform)"' \ % pythonexe) log.debug((s,o)) assert s==0 and o=='win32',(s,o) ---snip--- With this I get: ---snip--- $ python scipy_distutils/exec_command.py ... Running spawnve(0,'C:\\WINDOWS\\system32\\cmd.exe',['C:\\WINDOWS\\system32\\cmd.exe', '/C', 'C:\\cygwin\\bin\\echo.exe', 'path=%path%'],os.environ) _update_environment(...) (0, 'path=c:\\Programme\\......\\bin') exec_command('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"',) Retaining cwd: h:\work\CVS\scipy\scipy_core _preserve_environment([]) _update_environment(...) _exec_command(...) splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"') splitcmdline -> ['c:\\Python23\\PYTHON.EXE', '-c', '"import sys;sys.stderr.write(sys.platform)"'] find_executable('c:\\Python23\\PYTHON.EXE') Found executable c:\Python23\PYTHON.EXE Running spawnve(0,'C:\\WINDOWS\\system32\\cmd.exe',['C:\\WINDOWS\\system32\\cmd.exe', '/C', 'c:\\Python23\\PYTHON.EXE', '-c', '"import sys;sys.stderr.write(sys.platform)"'],os.environ) _update_environment(...) (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') Traceback (most recent call last): File "scipy_distutils/exec_command.py", line 599, in ? test(use_tee=0) File "scipy_distutils/exec_command.py", line 500, in test_nt assert s==0 and o=='win32',(s,o) AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') ---snip--- The problem is, the test writes to stderr, but a result on stderr leads "_exec_command" to generate the "... FAILED ..." message which is not handled by the assert statement. Changing the assert to ---snip--- assert s==0 and o[-5:]=='win32',(s,o) ---snip--- fixes this problem and leads to the next one: ---snip--- exec_command('c:\\Python23\\PYTHON.EXE -c "raise \'Ignore me.\'"',) Retaining cwd: h:\work\CVS\scipy\scipy_core _preserve_environment([]) _update_environment(...) _exec_command(...) splitcmdline('c:\\Python23\\PYTHON.EXE -c "raise \'Ignore me.\'"') splitcmdline -> ['c:\\Python23\\PYTHON.EXE', '-c', '"raise \'Ignore me.\'"'] find_executable('c:\\Python23\\PYTHON.EXE') Found executable c:\Python23\PYTHON.EXE Running spawnve(0,'C:\\WINDOWS\\system32\\cmd.exe',['C:\\WINDOWS\\system32\\cmd.exe', '/C', 'c:\\Python23\\PYTHON.EXE', '-c', '"raise \'Ignore me.\'"'],os.environ) _update_environment(...) (1, '') Traceback (most recent call last): File "scipy_distutils/exec_command.py", line 600, in ? test(use_tee=0) File "scipy_distutils/exec_command.py", line 504, in test_nt assert s==1 and o,(s,o) AssertionError: (1, '') ---snip--- I have no idea what the problem is here, because: ---snip--- C:\WINDOWS\system32>C:\\WINDOWS\\system32\\cmd.exe /C c:\\Python23\\PYTHON.EXE -c "raise 'Ignore me.'" Traceback (most recent call last): File "", line 1, in ? Ignore me. ---snip--- Testing works on Linux, but on Solaris I get: ---snip--- ... exec_command('echo $AAA',) Retaining cwd: /home/hoel/work/CVS/scipy/scipy_core _preserve_environment([]) _update_environment(...) _exec_command(...) splitcmdline('echo $AAA') splitcmdline -> ['echo', '$AAA'] Running spawnvpe(0,'echo',['echo', '$AAA'],os.environ) _update_environment(...) Traceback (most recent call last): File "scipy_distutils/exec_command.py", line 600, in ? test(use_tee=0) File "scipy_distutils/exec_command.py", line 523, in test_posix assert s==0 and o=='',(s,o) AssertionError: (0, '$AAA') ---snip--- which I also don't understand ---snip--- hoel at donau:scipy_core echo $AAA hoel at donau:scipy_core /bin/sh $ echo $AAA $ ---snip--- I'll try to find some fixes for these problems. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Tue Mar 30 05:42:04 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Tue, 30 Mar 2004 12:42:04 +0200 Subject: [SciPy-dev] Re: Another issue with scipy_distutils and FORTRAN files In-Reply-To: (Pearu Peterson's message of "Mon, 29 Mar 2004 11:04:22 -0600 (CST)") References: Message-ID: Pearu Peterson writes: > On Mon, 29 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> Our FORTRAN code often uses "include" statements. The FORTRAN compiler >> we use (DF 6.1, ifc 7.1, WorkShop 6 update 2 Fortran) recognize the >> -I flag for pointing to an include directory. I seems scipy_distutils >> have lost the ability to regognize the "include_dirs" argument to >> "Extensions". The compiler does not find the include files anymore. > > Hmm, I just checked and I could not reproduce this: paths in > the include_dirs keyword argument of an Extension class are shown > in `compile options:`. Try for example > > cd scipy_distutils/tests/f2py_f90_ext > python setup.py config_fc --fcompiler=intel build > > after updating/installing scipy_core and f2py from CVS. > > However, f2py was not able to find the include files. This is now fixed > in f2py CVS. I have no idea what went wrong here, but now it works, Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Tue Mar 30 10:02:24 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Tue, 30 Mar 2004 17:02:24 +0200 Subject: [SciPy-dev] Re: Another issue with scipy_distutils and FORTRAN files In-Reply-To: (Berthold =?iso-8859-1?q?H=F6llmann's?= message of "Tue, 30 Mar 2004 12:42:04 +0200") References: Message-ID: Berthold H?llmann writes: > Pearu Peterson writes: > >> On Mon, 29 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: >> >>> Our FORTRAN code often uses "include" statements. The FORTRAN compiler >>> we use (DF 6.1, ifc 7.1, WorkShop 6 update 2 Fortran) recognize the >>> -I flag for pointing to an include directory. I seems scipy_distutils >>> have lost the ability to regognize the "include_dirs" argument to >>> "Extensions". The compiler does not find the include files anymore. >> >> Hmm, I just checked and I could not reproduce this: paths in >> the include_dirs keyword argument of an Extension class are shown >> in `compile options:`. Try for example >> >> cd scipy_distutils/tests/f2py_f90_ext >> python setup.py config_fc --fcompiler=intel build >> >> after updating/installing scipy_core and f2py from CVS. >> >> However, f2py was not able to find the include files. This is now fixed >> in f2py CVS. > > I have no idea what went wrong here, but now it works, Hello, It turned out, the problem is how to tell include directiories when generating a library. We use ---snip--- ... flibs = [('fortmisc', {'sources': lib_src, 'include_dirs': [os.path.join("lib", "numfort")], 'macros':[]},),] ... setup(... libraries=flibs, ...) ... ---snip--- and setting set_verbosity(3) gives ---snip--- ... Running os.system('( /usr/software/fitools/opt/intel/compiler70_l_fc_pc_7.1.038/ia32/bin/ifc -72 -w90 -w95 -KPIC -cm -O3 -unroll -tpp5 -xM -c lib/numfort/NB.F -o build/temp.linux-i686-2.2/lib/numfort/NB.o ; echo $? > /tmp/@6992.7 ) 2>&1 | tee /tmp/@6992.6 ') ... ---snip--- I guess the the correct fix lies in _compiler in fcompiler.py. The line ---snip--- command = compiler + cc_args + s_args + o_args + extra_postargs ---snip--- should be ---snip--- command = compiler + cc_args + s_args + o_args + extra_postargs + pp_opts ---snip--- (at least this works for me). Of course "NB.F" only compiles with another (helpfull) patch to fcompiler.f. I changed ---snip--- language_map = {'.f':'f77', '.for':'f77', '.F':'f77', '.ftn':'f77', '.f77':'f77', '.f90':'f90', '.F90':'f90', '.f95':'f90'} ---snip--- and ---snip--- src_extensions = ['.for','.ftn','.f77','.f','.f90','.f95','.F','.F90'] ---snip--- Probably 'language_map' shold reflect that '.F' and '.F90' files should be preprocessed using fpp, and fpp must be explicitly switche on on some platforms. Kind regards Berthold H?llmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at scipy.org Tue Mar 30 11:33:13 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 30 Mar 2004 10:33:13 -0600 (CST) Subject: [SciPy-dev] Re: Another issue with scipy_distutils and FORTRAN files In-Reply-To: Message-ID: Hi Berthold, On Tue, 30 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > It turned out, the problem is how to tell include directiories when > generating a library. We use > > ---snip--- > ... > flibs = [('fortmisc', > {'sources': lib_src, > 'include_dirs': [os.path.join("lib", "numfort")], > 'macros':[]},),] > ... > setup(... > libraries=flibs, > ...) > ... > ---snip--- This usage of specifying libraries is not recommended and some of the corresponding features were removed indentionally (but at the moment I am not sure why the above does not work..), though the standard distutils might support this. The reason comes from the experience of building scipy: the above approach would cause 'fortmisc' to be linked with all extension modules that a package contains and this may cause undesired effects, for example, when subpackages define different libriaries with the same name, etc. The recommended way of specifying libriaries is to add them explicitely to extension modules that actually need them: flibs = [('fortmisc', {'sources': ..., 'include_dirs': ...})] ext = Extension('extname', ... libraires = [..]+flibs, ...) setup(..., ext_modules = [ext,..], ...) > I guess the the correct fix lies in _compiler in fcompiler.py. The line > > ---snip--- > command = compiler + cc_args + s_args + o_args + extra_postargs > ---snip--- > > should be > > ---snip--- > command = compiler + cc_args + s_args + o_args + extra_postargs + pp_opts > ---snip--- > > (at least this works for me). Let me know if the above approach of specifying libraires is acceptable for your application and then I'll postpone tracking down the "correct" fix to the problem. I am just hesitating commiting patches that I don't understand fully.. pp_opts should probably used only as preprocessor arguments but scipy_distutils does not have preprocessor support yet. > Of course "NB.F" only compiles with > another (helpfull) patch to fcompiler.f. I changed > > ---snip--- > language_map = {'.f':'f77', > '.for':'f77', > '.F':'f77', > '.ftn':'f77', > '.f77':'f77', > '.f90':'f90', > '.F90':'f90', > '.f95':'f90'} > ---snip--- > > and > > ---snip--- > src_extensions = ['.for','.ftn','.f77','.f','.f90','.f95','.F','.F90'] > ---snip--- This is now in CVS. > Probably 'language_map' shold reflect that '.F' and '.F90' files > should be preprocessed using fpp, and fpp must be explicitly switche > on on some platforms. Yes, I agree. Thanks, Pearu From pearu at scipy.org Tue Mar 30 12:01:08 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 30 Mar 2004 11:01:08 -0600 (CST) Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Tue, 30 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > There are still problems (a problem) with the patches. I was already > im mine, but I had'nt found the time to report them, sorry. The > version is working for Linux and Solaris but trying to build > scipy_core under WinXP I get: > > ---snip--- > ... > compling C sources > creating build\temp.win32-2.3 > creating build\temp.win32-2.3\Release > creating build\temp.win32-2.3\Release\scipy_base > C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DUSE_MCONF_LITE_LE -Ic:\Python23\include -Ic:\Python23\PC /Tcscipy_base\fastumathmodule.c /Fobuild\temp.win32-2.3\Release\scipy_base\fastumathmodule.obj > Could not locate executable h:\work\CVS\scipy\scipy_core\"C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" > Executable h:\work\CVS\scipy\scipy_core\"C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist > > error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DUSE_MCONF_LITE_LE -Ic:\Python23\include -Ic:\Python23\PC /Tcscipy_base\fastumathmodule.c /Fobuild\temp.win32-2.3\Release\scipy_base\fastumathmodule.obj" failed with exit status 1 > ---snip--- > > "h:\work\CVS\scipy\scipy_core\" is where I checked out scipy from > CVS. Further "python scipy_distutils/exec_command.py" fails. Line > numbers are not exactely those from CVS because of additional > debugging code. It seems that os.path.realpath is messing up things. For example, In [4]: os.path.realpath('c:/adad/adsa') Out[4]: '/home/pearu/c:/adad/adsa' In [5]: os.path.realpath('/adad/adsa') Out[5]: '/adad/adsa' Could you test on WinXP what is the result of the following expression os.path.realpath(r'C:\Programme') when python was fired up from the H: disk? Hmm, Python docs say that os.path.realpath is available only for Unix. This could provide a fix. > ---snip--- > s,o=exec_command('echo path=%path%') > log.debug((s,o)) > assert s==0 and o!='',(s,o) > > s,o=exec_command('%s -c "import sys;sys.stderr.write(sys.platform)"' \ > % pythonexe) > log.debug((s,o)) > assert s==0 and o=='win32',(s,o) > ---snip--- > > With this I get: > > ---snip--- > $ python scipy_distutils/exec_command.py > ... > AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') > ---snip--- > > The problem is, the test writes to stderr, but a result on stderr > leads "_exec_command" to generate the "... FAILED ..." I think I have fixed that yesterday. > AssertionError: (0, '$AAA') > ---snip--- > > which I also don't understand Ok, I could reproduce this on Sun and this is now fixed in CVS. Thanks, Pearu From hoel at gl-group.com Wed Mar 31 05:44:47 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Wed, 31 Mar 2004 12:44:47 +0200 Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: (Pearu Peterson's message of "Tue, 30 Mar 2004 11:01:08 -0600 (CST)") References: Message-ID: Pearu Peterson writes: Hallo Pearu, Thank you foru your prompt replay. > On Tue, 30 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > ... > > Could you test on WinXP what is the result of the following expression > > os.path.realpath(r'C:\Programme') I just checke out the latest CVS, and now get: ---snip--- hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core $ pwd /home/hoel/unix/work/CVS/scipy/scipy_core hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core $ ls -l /home/hoel/unix lrwxrwxrwx 1 hoel Kein 94 Oct 27 14:52 /home/hoel/unix -> /cygdrive/h hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core $ python Enthought Edition build 1028 Python 2.3 (#46, Aug 11 2003, 09:34:05) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import os.path >>> os.path.realpath(r'C:\Programme') 'C:\\Programme' >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio") 'C:\\Programme\\Microsoft Visual Studio' >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio\\") 'C:\\Programme\\Microsoft Visual Studio' >>> os.path.realpath(r'"C:\Programme\Microsoft Visual Studio\\"') 'h:\\work\\CVS\\scipy\\scipy_core\\"C:\\Programme\\Microsoft Visual Studio\\"' >>> ^Z $ ---snip--- > > when python was fired up from the H: disk? > So the problem seems to be, the executable name got quoted to early (I think this was my fault, I moved "quote_arg" to an too early position, but on I think now it is called again at a wrong position. What I get now under WinXP is ---snip--- customize MSVCCompiler customize MSVCCompiler using build_clib 0 customize CompaqVisualFCompiler customize CompaqVisualFCompiler using build_clib splitcmdline('DF /what') running build_ext No module named msvccompiler customize MSVCCompiler customize MSVCCompiler using build_ext customize CompaqVisualFCompiler customize CompaqVisualFCompiler using build_ext splitcmdline('DF /what') building 'SXFPyBase' extension compling C sources C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include-Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj" failed with exit status 1 make: *** [build] Error 1 ---snip--- The line after "compling C sources" looks wrong (should'nt the word "compling" in command/build_clib.py be replaced by "compiling" at 4 instances?) but it comes from CCompiler_spawn and the cl.exe part goes properly quoted to exec_command. Maybe the "_nt_quote_args" and the part leading to "log.info" should be exchanged for clarity in CCompiler_spawn. ---snip--- # Using customized CCompiler.spawn. def CCompiler_spawn(self, cmd, display=None): if type(cmd) is type([]) and os.name == 'nt': cmd = _nt_quote_args(cmd) if display is None: display = cmd if type(display) is type([]): display = ' '.join(display) log.info(display) s,o = exec_command(cmd) if s: if type(cmd) is type([]): cmd = ' '.join(cmd) print o raise DistutilsExecError,\ 'Command "%s" failed with exit status %d' % (cmd, s) CCompiler.spawn = new.instancemethod(CCompiler_spawn,None,CCompiler) ---snip--- Back to my problem: It came out to the definition of the include directory '-I"f:\DATA\FBE\ESC\Devel\include"'. The execution via "cmd.exe" does not like the quotes (") around the include path, so I removed them, just to stumble over the next problem: "-If:\DATA\FBE\ESC\Devel\include" as an option to DF is interpreted as "/iface=\DATA\FBE\ESC\Devel\include" and leads to an syntax error in DF. Luckily for me, "f:" is a network directory so I can write "\\Glf\Vol1\...", but I guess a general solution should be found. > Hmm, Python docs say that os.path.realpath is available only for Unix. > This could provide a fix. > >> ---snip--- >> s,o=exec_command('echo path=%path%') >> log.debug((s,o)) >> assert s==0 and o!='',(s,o) >> >> s,o=exec_command('%s -c "import sys;sys.stderr.write(sys.platform)"' \ >> % pythonexe) >> log.debug((s,o)) >> assert s==0 and o=='win32',(s,o) >> ---snip--- >> >> With this I get: >> >> ---snip--- >> $ python scipy_distutils/exec_command.py >> ... >> AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') >> ---snip--- >> >> The problem is, the test writes to stderr, but a result on stderr >> leads "_exec_command" to generate the "... FAILED ..." > > I think I have fixed that yesterday. > > >> AssertionError: (0, '$AAA') >> ---snip--- >> >> which I also don't understand > > Ok, I could reproduce this on Sun and this is now fixed in CVS. python scipy_distutils/exec_command.py OK on Linux and Solaris (except for the failing svn test, but I have no svn installed, so this is expected ;-). But on WinXP I still get ---snip--- hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core $ python scipy_distutils/exec_command.py splitcmdline('a b cc') splitcmdline('a') splitcmdline('a " b cc"') splitcmdline('"a bcc" -h') splitcmdline('"\\"a \\" bcc" -h') splitcmdline(" 'a bcc' -h") splitcmdline("'\\'a \\' bcc' -h") Using cygwin echo in win32 environment is not supported splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') splitcmdline('echo path=%path%') splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"') Traceback (most recent call last): File "scipy_distutils/exec_command.py", line 601, in ? test(use_tee=0) File "scipy_distutils/exec_command.py", line 502, in test_nt assert s==0 and o=='win32',(s,o) AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') ---snip--- maybe line 502 in exec_command.py could be changed to ---snip--- assert s==0 and o[-5:]=='win32',(s,o) ---snip--- Thanks Berthold -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From hoel at gl-group.com Wed Mar 31 07:11:38 2004 From: hoel at gl-group.com (=?ISO-8859-15?Q?Berthold_H=F6llmann?=) Date: Wed, 31 Mar 2004 14:11:38 +0200 Subject: [SciPy-dev] Re: Another issue with scipy_distutils and FORTRAN files In-Reply-To: (Pearu Peterson's message of "Tue, 30 Mar 2004 10:33:13 -0600 (CST)") References: Message-ID: Pearu Peterson writes: > Hi Berthold, > > On Tue, 30 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > >> It turned out, the problem is how to tell include directiories when >> generating a library. We use >> >> ---snip--- >> ... >> flibs = [('fortmisc', >> {'sources': lib_src, >> 'include_dirs': [os.path.join("lib", "numfort")], >> 'macros':[]},),] >> ... >> setup(... >> libraries=flibs, >> ...) >> ... >> ---snip--- > > This usage of specifying libraries is not recommended and some of the > corresponding features were removed indentionally (but at the moment I am > not sure why the above does not work..), though the standard distutils > might support this. The reason comes from the experience of building > scipy: the above approach would cause 'fortmisc' to be linked with > all extension modules that a package contains and this may cause undesired > effects, for example, when subpackages define different libriaries > with the same name, etc. The recommended way of specifying libriaries is > to add them explicitely to extension modules that actually need them: > > flibs = [('fortmisc', {'sources': ..., 'include_dirs': ...})] > > ext = Extension('extname', > ... > libraires = [..]+flibs, > ...) > > setup(..., > ext_modules = [ext,..], > ...) That is OK for me, I did not know this is possible. But to my experience the 'libraries' argument to 'setup' only tells distutils what libraries to build, if you want to link it to a specific extension you also have to give it's name in the 'Extension(...libraires' option. I have at least one library that is shared amongst some extensions. It is a dummy library to make distutils use the FORTRAN compiler for linking the extension because although all code belonging to the extension is C code, some of the external libraries used are FORTRAN code, and it is much easier to use FORTRAN compiler to link C code than the other way round (at least in my experience). > >> I guess the the correct fix lies in _compiler in fcompiler.py. The line >> >> ---snip--- >> command = compiler + cc_args + s_args + o_args + extra_postargs >> ---snip--- >> >> should be >> >> ---snip--- >> command = compiler + cc_args + s_args + o_args + extra_postargs + pp_opts >> ---snip--- >> >> (at least this works for me). > > Let me know if the above approach of specifying libraires is acceptable > for your application and then I'll postpone tracking down the "correct" > fix to the problem. I am just hesitating commiting patches that I don't > understand fully.. pp_opts should probably used only as preprocessor > arguments but scipy_distutils does not have preprocessor support > yet. There is usually some "internal" preprocessing support in FORTRAN code: using the 'include "some.inp"' command. Some FORTRAN compiler such as Compaq or SunFortran are looking into the direcory where the source file comes from for those include file, but unfortunately the Intel Compiler (at least under Linux) does not. So even without real preprocessing support at least the 'include_dirs' part of 'pp_opts' is needed for the FORTRAN compiler. For the four FORTRAN compiler I have access to, using the preprocessor is just another compiler option: g77: -x f77-cpp-input ifc: -fpp Sun: -fpp compaq: /fpp So maybe implemention is easy? '_compile' gets only pp_opts as an argument so I used it in building the command line, but of course one could split 'pp_opts' again in 'macros', and 'include_dirs' and give those to _compile. > >> Of course "NB.F" only compiles with >> another (helpfull) patch to fcompiler.f. I changed >> >> ---snip--- >> language_map = {'.f':'f77', >> '.for':'f77', >> '.F':'f77', >> '.ftn':'f77', >> '.f77':'f77', >> '.f90':'f90', >> '.F90':'f90', >> '.f95':'f90'} >> ---snip--- >> >> and >> >> ---snip--- >> src_extensions = ['.for','.ftn','.f77','.f','.f90','.f95','.F','.F90'] >> ---snip--- > > This is now in CVS. > >> Probably 'language_map' shold reflect that '.F' and '.F90' files >> should be preprocessed using fpp, and fpp must be explicitly switche >> on on some platforms. > > Yes, I agree. > > Thanks, > Pearu Thanks, Berthold -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel at gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc. From pearu at scipy.org Wed Mar 31 14:25:37 2004 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 31 Mar 2004 13:25:37 -0600 (CST) Subject: [SciPy-dev] Re: scipy_distutils under cygwin using DF In-Reply-To: Message-ID: On Wed, 31 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > I just checke out the latest CVS, and now get: > > ---snip--- > hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core > $ pwd > /home/hoel/unix/work/CVS/scipy/scipy_core > > hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core > $ ls -l /home/hoel/unix > lrwxrwxrwx 1 hoel Kein 94 Oct 27 14:52 /home/hoel/unix -> /cygdrive/h > > hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core > $ python > Enthought Edition build 1028 > Python 2.3 (#46, Aug 11 2003, 09:34:05) [MSC v.1200 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import os.path > >>> os.path.realpath(r'C:\Programme') > 'C:\\Programme' > >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio") > 'C:\\Programme\\Microsoft Visual Studio' > >>> os.path.realpath(r"C:\Programme\Microsoft Visual Studio\\") > 'C:\\Programme\\Microsoft Visual Studio' > >>> os.path.realpath(r'"C:\Programme\Microsoft Visual Studio\\"') > 'h:\\work\\CVS\\scipy\\scipy_core\\"C:\\Programme\\Microsoft Visual Studio\\"' > >>> ^Z > $ > ---snip--- > > > > > when python was fired up from the H: disk? > > > > So the problem seems to be, the executable name got quoted to early (I > think this was my fault, I moved "quote_arg" to an too early position, > but on I think now it is called again at a wrong position. What I get > now under WinXP is > > ---snip--- > customize MSVCCompiler > customize MSVCCompiler using build_clib > 0 > customize CompaqVisualFCompiler > customize CompaqVisualFCompiler using build_clib > splitcmdline('DF /what') > running build_ext > No module named msvccompiler > customize MSVCCompiler > customize MSVCCompiler using build_ext > customize CompaqVisualFCompiler > customize CompaqVisualFCompiler using build_ext > splitcmdline('DF /what') > building 'SXFPyBase' extension > compling C sources > C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include-Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj > Could not locate executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" > Executable "C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" does not exist > > error: Command ""C:\Programme\Microsoft Visual Studio\VC98\BIN\cl.exe" /c /nologo /Ox /MD /W3 /GX /DNDEBUG -DDEBUG -DVISUAL_CPLUSPLUS -Ilib\SXFPyBase -I"f:\DATA\FBE\ESC\Devel\include" -Ibuild\src -Ic:\Python23\include -Ic:\Python23\PC /Tcbuild\src\fortranobject.c /Fobuild\temp.win32-2.3\Release\build\src\fortranobject.obj" failed with exit status 1 > make: *** [build] Error 1 > ---snip--- > > The line after "compling C sources" looks wrong (should'nt the word > "compling" in command/build_clib.py be replaced by "compiling" at 4 > instances?) but it comes from CCompiler_spawn and the cl.exe part goes > properly quoted to exec_command. Maybe the "_nt_quote_args" and the > part leading to "log.info" should be exchanged for clarity in > CCompiler_spawn. > > ---snip--- > # Using customized CCompiler.spawn. > def CCompiler_spawn(self, cmd, display=None): > if type(cmd) is type([]) and os.name == 'nt': > cmd = _nt_quote_args(cmd) > if display is None: > display = cmd > if type(display) is type([]): display = ' '.join(display) > log.info(display) > s,o = exec_command(cmd) > if s: > if type(cmd) is type([]): > cmd = ' '.join(cmd) > print o > raise DistutilsExecError,\ > 'Command "%s" failed with exit status %d' % (cmd, s) > CCompiler.spawn = new.instancemethod(CCompiler_spawn,None,CCompiler) > ---snip--- I fixed this issue by removing quotes in find_executable function. It is in CVS now and seems to work, I was able to build scipy_core with MSVC 6.0 compiler! > Back to my problem: It came out to the definition of the include > directory '-I"f:\DATA\FBE\ESC\Devel\include"'. The execution via > "cmd.exe" does not like the quotes (") around the include path, so I > removed them, just to stumble over the next problem: > > "-If:\DATA\FBE\ESC\Devel\include" > > as an option to DF is interpreted as > "/iface=\DATA\FBE\ESC\Devel\include" and leads to an syntax error in > DF. Luckily for me, "f:" is a network directory so I can write > "\\Glf\Vol1\...", but I guess a general solution should be found. > > > Hmm, Python docs say that os.path.realpath is available only for Unix. > > This could provide a fix. > > > >> ---snip--- > >> s,o=exec_command('echo path=%path%') > >> log.debug((s,o)) > >> assert s==0 and o!='',(s,o) > >> > >> s,o=exec_command('%s -c "import sys;sys.stderr.write(sys.platform)"' \ > >> % pythonexe) > >> log.debug((s,o)) > >> assert s==0 and o=='win32',(s,o) > >> ---snip--- > >> > >> With this I get: > >> > >> ---snip--- > >> $ python scipy_distutils/exec_command.py > >> ... > >> AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') > >> ---snip--- > >> > >> The problem is, the test writes to stderr, but a result on stderr > >> leads "_exec_command" to generate the "... FAILED ..." > > > > I think I have fixed that yesterday. > > > > > >> AssertionError: (0, '$AAA') > >> ---snip--- > >> > >> which I also don't understand > > > > Ok, I could reproduce this on Sun and this is now fixed in CVS. > > python scipy_distutils/exec_command.py > > OK on Linux and Solaris (except for the failing svn test, but I have > no svn installed, so this is expected ;-). > > But on WinXP I still get > > ---snip--- > hoel at PC021358 ~/unix/work/CVS/scipy/scipy_core > $ python scipy_distutils/exec_command.py > splitcmdline('a b cc') > splitcmdline('a') > splitcmdline('a " b cc"') > splitcmdline('"a bcc" -h') > splitcmdline('"\\"a \\" bcc" -h') > splitcmdline(" 'a bcc' -h") > splitcmdline("'\\'a \\' bcc' -h") > Using cygwin echo in win32 environment is not supported > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\',\'\')"') > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'AAA\')"') > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import os;print os.environ.get(\'BBB\',\'\')"') > splitcmdline('echo path=%path%') > splitcmdline('c:\\Python23\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"') > Traceback (most recent call last): > File "scipy_distutils/exec_command.py", line 601, in ? > test(use_tee=0) > File "scipy_distutils/exec_command.py", line 502, in test_nt > assert s==0 and o=='win32',(s,o) > AssertionError: (0, 'COMMAND \'c:\\\\Python23\\\\PYTHON.EXE -c "import sys;sys.stderr.write(sys.platform)"\' FAILED: win32') > ---snip--- > > maybe line 502 in exec_command.py could be changed to > > ---snip--- > assert s==0 and o[-5:]=='win32',(s,o) > ---snip--- No, that would be cheating;-) Recent CVS seems to fix this issue as well. Regards, Pearu From pearu at scipy.org Wed Mar 31 15:38:04 2004 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 31 Mar 2004 14:38:04 -0600 (CST) Subject: [SciPy-dev] Re: Another issue with scipy_distutils and FORTRAN files In-Reply-To: Message-ID: On Wed, 31 Mar 2004, [ISO-8859-15] Berthold H?llmann wrote: > There is usually some "internal" preprocessing support in FORTRAN > code: using the 'include "some.inp"' command. Some FORTRAN compiler > such as Compaq or SunFortran are looking into the direcory where the > source file comes from for those include file, but unfortunately the > Intel Compiler (at least under Linux) does not. So even without real > preprocessing support at least the 'include_dirs' part of 'pp_opts' is > needed for the FORTRAN compiler. For the four FORTRAN compiler I have > access to, using the preprocessor is just another compiler option: > > g77: -x f77-cpp-input > ifc: -fpp > Sun: -fpp > compaq: /fpp > > So maybe implemention is easy? > > '_compile' gets only pp_opts as an argument so I used it in building > the command line, but of course one could split 'pp_opts' again in > 'macros', and 'include_dirs' and give those to _compile. I'll put fixing this into my TODO list. Pearu