From m.hadfield@niwa.cri.nz Tue May 1 18:19:04 2001 From: m.hadfield@niwa.cri.nz (Mark Hadfield) Date: Tue May 1 17:19:04 2001 Subject: [Distutils] Building Cygwin-Python extension modules - export strategies Message-ID: <005f01c0d284$4face1e0$d938a8c0@Hadfield> I have posted the following on comp.lang.python. At Jason Tishler's suggestion I am now also sending it to this SIG. In keeping with the dignity of this forum I have cut some of the excessively flippant bits.... On Friday I posted a message to comp.lang.idl-pvwave entitled "Building Numeric package on Cygwin-Python". I reported that the Numpy module library could not be built with Cygwin Python 2.1 using the distutils-based setup script. With the help of people who responded to this message, and to another one on the Cygwin mailing list, I can now build Numpy. But to get it to work out-of-the-box requires changes either to the C code or to distutils (see below). I would like to trigger a discussion on this list and ultimately a ruling from the Python gods about the one-true-Pythonic-way to write and install Python extension modules for the Cygwin version of Python. To recap, I have the Numpy source distribution, Numeric-20.0.0b2.tar.gz (but I think these considerations apply to any Python module library that uses C code) and I have Python 2.1 installed by Cygwin's setup program (but the results should be the same if I had built it myself using the Cygwin gcc compiler). I unpack the source and cd to the directory created. According to the "Installing Python Modules" manual, I should be able to build & install this by typing python setup.py build python setup.py install However the build step fails at the first link command, with messages about undefined symbols. The problem is (and I'm a bit hazy here) that to build a DLL on the Windows platform, the compiler needs a list of symbols to be exported. The above command invokes a Unix-like build process that doesn't know about this. There are two solutions: 1. (Thanks to Robert Yodlowski): In each C file foomodule.c, replace void initfoo () with DL_EXPORT(void) initfoo () where DL_EXPORT is a preprocessor macro. 2. (Thanks to Norman Vine): Invoke the setup script's "cygwin" compiler option: python setup.py build compiler=cygwin python setup.py install after first applying a patch (after sig) that modifies a couple of distutils files. The compiler then creates and uses the necessary .def files to specify the exports. (Note that the Numeric source does include several .def files but these are ignored.) This is all fine, and it's not that much of a hardship to make one or other of these changes. But obviously it would be good if Cygwin Python users could build extension modules out-of-the-box, so I would like to see one of these changes (or something like them) incorporated into Python and its documentation. So which one? Jason Tishler and Norman Vine have told me that these things have been discussed before on python-dev and distutils-sig but never carried through to a resolution Option 1 does have the blessing of Tim Peters http://mail.python.org/pipermail/python-dev/2001-February/013292.html The downside is that it requires changes (albeit minor) to the C code of the Numeric package and, I presume, most other Python extension packages. Not to mention modifications to the "Extending and Embedding" manual. Option 2 was raised by Jason Tishler on distutils-sig http://mail.python.org/pipermail/distutils-sig/2001-February/002003.html but there wasn't any enthusiasm for it so he dropped it (to paraphrase Jason's words). If this is to be applied I would prefer that it not be necessary to specify the "cygwin" compiler option explicitly. (According to the "Installing Python Modules" manual this should only be necessary when using Cygwin to build extensions for native win32 Python.) Yes, I know I can achieve this myself via the distutils configuration file. On the newsgroup Jason has pointed out that Option 1 works with the traditional Misc/Makefile.pre.in, whereas Option 2 does not. Opinions? --- Mark Hadfield m.hadfield@niwa.cri.nz http://katipo.niwa.cri.nz/~hadfield National Institute for Water and Atmospheric Research ? cygwinccompiler.py.patch ? diffs ? my.diffs cvs server: Diffing . Index: cygwinccompiler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/cygwinccompiler.py,v retrieving revision 1.12 diff -r1.12 cygwinccompiler.py 99,103c99,103 < self.set_executables(compiler='gcc -mcygwin -O -Wall', < compiler_so='gcc -mcygwin -mdll -O -Wall', < linker_exe='gcc -mcygwin', < linker_so=('%s -mcygwin -mdll -static' % < self.linker_dll)) --- > # self.set_executables(compiler='gcc -mcygwin -O -Wall', > # compiler_so='gcc -mcygwin -mdll -O -Wall', > # linker_exe='gcc -mcygwin', > # linker_so=('%s -mcygwin -mdll -static' % > # self.linker_dll)) Index: sysconfig.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v retrieving revision 1.34 diff -r1.34 sysconfig.py 120c120,121 < if compiler.compiler_type == "unix": --- > # if compiler.compiler_type == "unix": > if compiler.compiler_type in ["unix", "cygwin"]: cvs server: Diffing command From scleary@jerviswebb.com Wed May 9 15:37:01 2001 From: scleary@jerviswebb.com (scleary@jerviswebb.com) Date: Wed May 9 14:37:01 2001 Subject: [Distutils] Distributing modules with dependencies Message-ID: <22602E7E3B92D411969B0090273CC28B1D1E32@cecexchange.jerviswebb.com> I see that between PEP241 and 243, Py is moving towards a central module repository. My question is: has any thought been given to the idea of having one application/library be dependent on another library, and for the download/install to be done as part of setup.py? For example, I'm writing a simple application that uses wxPython -- which not everyone has installed. During installation, I can right now in setup.py abort if wxPython isn't found -- but it would be much nicer to users (IMO) if it can download & install wxPython silently. This is more complicated than just that; I'd like to see: . The ability for users (special command-line option?) to download all dependent packages to a local directory first. . The ability for distutils to substitute newer versions in place of older -- this is a lot more tricky, since we *don't* want it to do that for non-backwards-compatible differences. I'd like to see the basic functionality anyway, even if the two features above aren't done until later. The reason being that my intended user base are not programmers or even serious computer users at all. In fact, if there was any way to package Python itself with my app, I'd like to know about it (is there?). I just don't want to see us in the future run into what Linux did a little while ago -- oh, your update for package x requires an update to package y, so after you download and install the update to package y, you find it requires an update to package z, ... -Steve From gpk@bell-labs.com Mon May 14 11:43:00 2001 From: gpk@bell-labs.com (Greg Kochanski) Date: Mon May 14 10:43:00 2001 Subject: [Distutils] No support for .pyo files Message-ID: <3AFFEEA9.B39B51F5@bell-labs.com> Python has a -O flag that turns on some compiler optimizations. Python then looks for (and produces) .pyo files instead of .pyc files. Distutils doesn't seem to provide any support for this. What can be done? From R.Liebscher@gmx.de Mon May 14 11:54:02 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Mon May 14 10:54:02 2001 Subject: [Distutils] No support for .pyo files References: <3AFFEEA9.B39B51F5@bell-labs.com> Message-ID: <3AFFEF9B.355F9E8C@gmx.de> Greg Kochanski wrote: > > Python has a -O flag that turns on some compiler > optimizations. Python then looks for (and produces) > .pyo files instead of .pyc files. > > Distutils doesn't seem to provide any support for this. > What can be done? > By using "python setup.py install -O1" you can tell distutils to create pyo files. ------------------------------------------------------------------------------ rob@cvs/distutils >python setup.py install --help ... Options for 'install' command: ... --compile (-c) compile .py to .pyc [default] --no-compile don't compile .py files --optimize (-O) also compile with optimization: -O1 for "python -O", -O2 for "python -OO", and -O0 to disable [default: -O0] ... ------------------------------------------------------------------------------ Kind regards Rene Liebscher From mal@lemburg.com Mon May 14 11:55:22 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon May 14 10:55:22 2001 Subject: [Distutils] No support for .pyo files References: <3AFFEEA9.B39B51F5@bell-labs.com> Message-ID: <3AFFF187.F791BBD8@lemburg.com> Greg Kochanski wrote: > > Python has a -O flag that turns on some compiler > optimizations. Python then looks for (and produces) > .pyo files instead of .pyc files. > > Distutils doesn't seem to provide any support for this. > What can be done? Distutils does have support to compile .pyo files. You just have to specify -O or -O2 as option to the build_py command. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From hinsen@cnrs-orleans.fr Wed May 16 14:54:01 2001 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 16 13:54:01 2001 Subject: [Distutils] Installing non-program files into the lib directory Message-ID: <200105161752.TAA26791@chinon.cnrs-orleans.fr> I have a package which includes some non-program files (in fact scripts, but which are run by exec_file) that have to be installed in a specific subpackage folder. Is there any simple way to do this with distutils, i.e. one which doesn't require me to reimplement several methods? Library installation ignores everything which is not .py, whereas data file installation doesn't let me access the lib directory. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From pete@visionart.com Wed May 16 15:18:01 2001 From: pete@visionart.com (Pete Shinners) Date: Wed May 16 14:18:01 2001 Subject: [Distutils] Installing non-program files into the lib directory References: <200105161752.TAA26791@chinon.cnrs-orleans.fr> Message-ID: <004001c0de34$b953b3b0$f43f93cd@visionart.com> > I have a package which includes some non-program files (in fact > scripts, but which are run by exec_file) that have to be installed in > a specific subpackage folder. Is there any simple way to do this with > distutils, i.e. one which doesn't require me to reimplement several > methods? Library installation ignores everything which is not .py, > whereas data file installation doesn't let me access the lib > directory. yeah, i have figured this out. i have the exact same setup, a couple resources that need to be in the same directory as the .PY files that use them. i have a "lib" directory with all the files i want in my final directory. here's the relevant snips from my setup.py file to get them all installed nicely note that it took me quite awhile to get everything to this stage, but now it is working beautifully. you'll see the main secret is creating my own "install_data" command that actually puts files where the installation is going, instead of pretty much random locations on different platforms. hope this helps! ----------------------------------------------------------------- #note, this is cut and pasted together. just the relevent parts #of my setup.py file. DESCRIPTION = """Pygame is a Python wrapper module for the SDL multimedia library. It contains python functions and classes that will allow you to use SDL's support for playing cdroms, audio and video output, and keyboard, mouse and joystick input. Pygame also includes support for the Numerical Python extension.""" METADATA = { "name": "pygame", "version": "1.0.1", "license": "LGPL", "url": "http://pygame.seul.org", "author": "Pete Shinners", "author_email": "pygame@seul.org", "description": "Python Game Development Package", "long_description": DESCRIPTION, } import os, glob, sys import distutils.sysconfig from distutils.command.install_data import install_data #get us to the correct directory path = os.path.split(os.path.abspath(sys.argv[0]))[0] os.chdir(path) #add non .py files in lib directory data_files = [] for f in glob.glob(os.path.join('lib', '*')): if not f.endswith('.py') and os.path.isfile(f): data_files.append(f) #data installer with improved intelligence over distutils #data files are copied into the project directory instead #of willy-nilly class smart_install_data(install_data): def run(self): #need to change self.install_dir to the library dir install_cmd = self.get_finalized_command('install') self.install_dir = getattr(install_cmd, 'install_lib') return install_data.run(self) #finally, #call distutils with all needed info PACKAGEDATA = { "cmdclass": {'install_data': smart_install_data}, "packages": ['pygame'], "package_dir": {'pygame': 'lib'}, "data_files": [['pygame', data_files]], } PACKAGEDATA.update(METADATA) apply(setup, [], PACKAGEDATA) From hinsen@cnrs-orleans.fr Wed May 16 15:47:01 2001 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 16 14:47:01 2001 Subject: [Distutils] Installing non-program files into the lib directory In-Reply-To: <004001c0de34$b953b3b0$f43f93cd@visionart.com> (pete@visionart.com) References: <200105161752.TAA26791@chinon.cnrs-orleans.fr> <004001c0de34$b953b3b0$f43f93cd@visionart.com> Message-ID: <200105161846.UAA27218@chinon.cnrs-orleans.fr> > you'll see the main secret is creating my own "install_data" > command that actually puts files where the installation is > going, instead of pretty much random locations on different > platforms. Right, that does it for me as well, thanks a lot! I was thinking about a much more complicated solution. I just don't know enough about distutils internals to be able to come up with yours - I think more examples would be welcome. When I finally get my MMTK setup file working, I'll gladly submit it as an example - at currently 205 lines it is probably one of the more complicated ones. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From mal@lemburg.com Thu May 17 12:36:03 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu May 17 11:36:03 2001 Subject: [Distutils] Building external libs with distutils Message-ID: <3B03EF8A.93043D9D@lemburg.com> I was just wondering whether anybody has already tackled the job of using distutils to build external libs. I am thinking of the following setup here: * you have a source tree in ./libsrc * building the lib is done in the usual Unix way: cd ./libsrc; ./configure; make * now copy te resulting .a or .so to your Python package target build dir * continue with the normal distutils procedure to build the Python wrapper Is this possible in a fairly portable way (at least with respect to Unix platforms) ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From robin@jessikat.fsnet.co.uk Thu May 17 14:20:01 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Thu May 17 13:20:01 2001 Subject: [Distutils] Building external libs with distutils In-Reply-To: <3B03EF8A.93043D9D@lemburg.com> References: <3B03EF8A.93043D9D@lemburg.com> Message-ID: In article <3B03EF8A.93043D9D@lemburg.com>, M.-A. Lemburg writes >I was just wondering whether anybody has already tackled the >job of using distutils to build external libs. I am thinking >of the following setup here: > >* you have a source tree in ./libsrc >* building the lib is done in the usual Unix way: > cd ./libsrc; ./configure; make >* now copy te resulting .a or .so to your Python package > target build dir >* continue with the normal distutils procedure to build the > Python wrapper > >Is this possible in a fairly portable way (at least with >respect to Unix platforms) ? > well I have used the standard lib building scheme, but that doesn't involve the configure make framework. All the flags were set in setup.py. Which is why the AIX build of my bitmap renderer went wrong; ie it didn't know about endianness which is presumably something that the configure script would normally figure out. -- Robin Becker From mal@lemburg.com Thu May 17 14:47:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu May 17 13:47:01 2001 Subject: [Distutils] Building external libs with distutils References: <3B03EF8A.93043D9D@lemburg.com> Message-ID: <3B040E54.3C9D2861@lemburg.com> Robin Becker wrote: > > In article <3B03EF8A.93043D9D@lemburg.com>, M.-A. Lemburg > writes > >I was just wondering whether anybody has already tackled the > >job of using distutils to build external libs. I am thinking > >of the following setup here: > > > >* you have a source tree in ./libsrc > >* building the lib is done in the usual Unix way: > > cd ./libsrc; ./configure; make > >* now copy te resulting .a or .so to your Python package > > target build dir > >* continue with the normal distutils procedure to build the > > Python wrapper > > > >Is this possible in a fairly portable way (at least with > >respect to Unix platforms) ? > > > well I have used the standard lib building scheme, but that doesn't > involve the configure make framework. All the flags were set in > setup.py. Which is why the AIX build of my bitmap renderer went wrong; > ie it didn't know about endianness which is presumably something that > the configure script would normally figure out. Right... I think we really need to get distutils to know about the standard Unix autoconf-based build process in order to include building external libs right along with the Python extensions. I'm currently looking at having iODBC built directly by the mxODBC setup -- this would allow shipping a single archive for the whole machinery and greatly simplify the installation process. This is also a prerequisite for auto-building in a CPAN like way. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From davida@pobox.com Thu May 24 04:59:01 2001 From: davida@pobox.com (David Arnold) Date: Thu May 24 03:59:01 2001 Subject: [Distutils] replacing *.in from autoconf-land Message-ID: <200105240757.f4O7vHs14132@xevious.dstc.monash.edu.au> i'm in the middle of moving a package from using autoconf to distutils, and i'm having trouble finding a good way to do substitution of configured/discovered options into a template file. in autoconf, i have a config.py.in file, which contains lines of the form default_url = "@default_url@" at configure time, the @foo@ values are replaced with real content, from --with-default-url options to configure, and various other discovered stuff. the package is pure Python, so i have been able to just run install, which in turn ran build, and i don't really want to add a config step unless i have to. i have played with sub-classing the build command, and adding a file copy and substitute step into the run command, but ... has anyone else done something like this? any suggestions? d From mal@lemburg.com Thu May 24 07:14:02 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu May 24 06:14:02 2001 Subject: [Distutils] replacing *.in from autoconf-land References: <200105240757.f4O7vHs14132@xevious.dstc.monash.edu.au> Message-ID: <3B0CDEB7.BE3EC5D2@lemburg.com> David Arnold wrote: > > i'm in the middle of moving a package from using autoconf to > distutils, and i'm having trouble finding a good way to do > substitution of configured/discovered options into a template file. > > in autoconf, i have a config.py.in file, which contains lines of the > form > > default_url = "@default_url@" > > at configure time, the @foo@ values are replaced with real content, > from --with-default-url options to configure, and various other > discovered stuff. > > the package is pure Python, so i have been able to just run install, > which in turn ran build, and i don't really want to add a config step > unless i have to. > > i have played with sub-classing the build command, and adding a file > copy and substitute step into the run command, but ... > > has anyone else done something like this? any suggestions? Python 2.1 uses the approach you describe for setup.cfg: it has a setup.cfg.in which then gets updated by configure to form the final setup.cfg file. Would this be what you need ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From shredwheat@mediaone.net Sat May 26 03:48:01 2001 From: shredwheat@mediaone.net (Pete Shinners) Date: Sat May 26 02:48:01 2001 Subject: [Distutils] macos support (weak?) Message-ID: <003c01c0e5af$81f61700$0200a8c0@petebox> i've got some people doing MacOS testing for my extension package. currently support is great under windows and unix. i was under the impression python2.1 shipped with some distutils code that was ready to go for macos? but from the reports i hear this is not the case? unfortunately, i've got not mac to test this on, so it comes to you all second hand. i'm hoping there's a mac user or two in here who can lend some knowledge. the first problem is that it appears distutils does not find the default "site-packages" on its own? distutils.error.DistUtilsPlatformError: OK, where DO site-specific modules go on the Mac? does mac have some sort of PYTHONHOME equivalent it can use to find its home? certainly there must be something because python can find all its own modules and packages. can distutils somehow tap into this knowledge to save the installer from entering the site-packages path when installing? the second problem is a little more serious. my project is using an external setup file to get all the extension compile flags. the library directories are being sent with "-Llibrary/path/lib" this works for windows and unix, but on macos it is creating absolute paths, not relative. so it looks like to get it working the user needs something like -Lmacintosh hd/libs, but the space is throwing things for a loop. does a backslash work before the space? anyways, when given a relative path it is becoming an absolute path lastly, and most important of all, it looks like the libraries being passed in with "-l" are not even being sent through the XML to codewarrior. anyways, my guess is that this is all related to using the external Setup file and loading it with read_setup_file(). is that not really been used for macos stuff? if sending proper Extension objects to the setup function is all thats needed then i can workaround this pretty easy. has anyone had any luck building distutils extensions on macos when needing to link in extra libraries? From jack@oratrix.nl Sat May 26 18:05:01 2001 From: jack@oratrix.nl (Jack Jansen) Date: Sat May 26 17:05:01 2001 Subject: [Distutils] macos support (weak?) In-Reply-To: Message by "Pete Shinners" , Fri, 25 May 2001 23:45:49 -0700 , <003c01c0e5af$81f61700$0200a8c0@petebox> Message-ID: <20010526210431.DE87CDDDE3@oratrix.oratrix.nl> Recently, "Pete Shinners" said: > i've got some people doing MacOS testing for my extension > package. currently support is great under windows and unix. > > i was under the impression python2.1 shipped with some > distutils code that was ready to go for macos? but from the > reports i hear this is not the case? Yes, it's in there. You need the distutils support with MacPython, though, some of the fixes needed for MacPython 2.1 did not make it to the CVS repository in time. But the one incorporated in the MacPython distirbution should work. But then: I'm the only one to ever have tested it, so..... :-) > the first problem is that it appears distutils does not > find the default "site-packages" on its own? It finds it for me. Hmm, idea: maybe the directory should exist, and it doesn't exist in the distribution? Ask your mac tester to create Python:Lib:site-packages and try again. If that's the problem I'll force site-packages to be created next time. > does mac have some sort of PYTHONHOME equivalent it can use to > find its home? certainly there must be something because python > can find all its own modules and packages. can distutils somehow > tap into this knowledge to save the installer from entering the > site-packages path when installing? sys.prefix. And this is indeed what distutils uses, if I'm not mistaken. > the second problem is a little more serious. my project is > using an external setup file to get all the extension compile flags. > the library directories are being sent with "-Llibrary/path/lib" > this works for windows and unix, but on macos it is creating > absolute paths, not relative. so it looks like to get it working > the user needs something like -Lmacintosh hd/libs, but the space > is throwing things for a loop. does a backslash work before the > space? anyways, when given a relative path it is becoming > an absolute path I never tried this, so I wouldn't know. For the things I tried I've never needed a library path (and it would actually be dangerous too) and I've always spelled out the filename paths to the specific libraries I use. But the thing to remember is that all the paths you use within distutils are URLs (even though they look very much like unix pathnames to the unsuspecting eye:-) so ../../bla should work even on a Mac, and you should be able to using %20 for space should work fine. > lastly, and most important of all, it looks like the libraries > being passed in with "-l" are not even being sent through the XML > to codewarrior. Where are you passing the libraries with "-l"? In the linker switches? That is the wrong place, unless I'm mistaken. They should be in the "libraries" attribute. -L and -l are unixisms. Hmm, that is: unless the distutils infrastructure on Windows parses them to emulate their behaviour, in which case I guess it should do the same on the Mac. Can anyone shed a light on this? > anyways, my guess is that this is all related to using the > external Setup file and loading it with read_setup_file(). is > that not really been used for macos stuff? if sending proper > Extension objects to the setup function is all thats needed then > i can workaround this pretty easy. You've lost me here. What's an external setup file? I asusme you don't mean the setup.py which is part of your package (and, thus, external to distutils) but something else? (the sofist answer would probably be "where is it documented?" :-) > has anyone had any luck building distutils extensions on macos > when needing to link in extra libraries? Yes, but I wouldn't be surprised if it was only your's truly:-) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From shredwheat@mediaone.net Sat May 26 18:31:44 2001 From: shredwheat@mediaone.net (Pete Shinners) Date: Sat May 26 17:31:44 2001 Subject: [Distutils] macos support (more info) References: <20010526210431.DE87CDDDE3@oratrix.oratrix.nl> Message-ID: <003201c0e62a$f0e30a80$0200a8c0@petebox> > Yes, it's in there. You need the distutils support with MacPython, > though, some of the fixes needed for MacPython 2.1 did not make it to > the CVS repository in time. But the one incorporated in the MacPython > distirbution should work. But then: I'm the only one to ever have > tested it, so..... :-) hey jack, with more testing and some coding we seem to have gotten somewhere. i'll explain the results as i know them... (btw, this was all with macpython 2.1 > > the first problem is that it appears distutils does not > > find the default "site-packages" on its own? > > It finds it for me. Hmm, idea: maybe the directory should exist, and > it doesn't exist in the distribution? Ask your mac tester to create > Python:Lib:site-packages and try again. If that's the problem I'll > force site-packages to be created next time. hmm, no followup info on this one, sorry > > the second problem is a little more serious. my project is > > using an external setup file to get all the extension compile flags. > > the library directories are being sent with "-Llibrary/path/lib" > > this works for windows and unix, but on macos it is creating > > absolute paths, not relative. > > I never tried this, so I wouldn't know. For the things I tried I've > never needed a library path (and it would actually be dangerous too) > and I've always spelled out the filename paths to the specific > libraries I use. ok, apparantly this was a one-liner fix in distutils. the include directories path was working fine, the library path was not working with relative paths. it sounds like metroworks treats include dirs and library dirs as the same thing. so one workaround was to just use all include dirs, but the one-line-fix has made things a little more consistent between platforms. > > lastly, and most important of all, it looks like the libraries > > being passed in with "-l" are not even being sent through the XML > > to codewarrior. > > Where are you passing the libraries with "-l"? In the linker switches? > That is the wrong place, unless I'm mistaken. They should be in the > "libraries" attribute. -L and -l are unixisms. right, it turns out these were being properly bassed in as the libraries attribute. the guy doing the testing had to go into the XML outputter and apparantly add the library stuff in there. it didn't look like the library info was being passed through. > > anyways, my guess is that this is all related to using the > > external Setup file and loading it with read_setup_file(). > > You've lost me here. What's an external setup file? I asusme you don't > mean the setup.py which is part of your package (and, thus, external > to distutils) but something else? (the sofist answer would probably be > "where is it documented?" :-) this was all wrong. later on we did some testing and the external setup file was definitely being parsed fine. the "read_setup_file" call in distutils reads in a Makefile-like textfile and turns it into a list of Extension objects. anyways, with a little digging we have got mac distutils working pretty well. we'll make sure the final code changes get back to you guys somehow. From Jason.Tishler@dothill.com Tue May 29 14:30:01 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Tue May 29 13:30:01 2001 Subject: [Distutils] Cygwin Python sys.platform and distutils.util.get_platform() patch Message-ID: <20010529132936.C498@dothill.com> --ohmds5dUDiOjm0u1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I would like to submit the attached patch (i.e., the first attachment) to Python CVS for inclusion in 2.2. IMO, this is the right solution for many reasons. I have included one example as the the second attachment. If the patch is accepted, then I will back patch and re-release my Cygwin Python 2.1 distribution. Since these two simple changes have possibly far reaching ramifications, I thought that it would be prudent to solicit feedback before submitting the patch. My goal is to eliminate (or at least minimize) the heartache caused by these changes, if any. Note that the two parts of the patch are actually independent but related so I have decided to submit them as one patch instead of two. The first part changes sys.platform, sys.path, and the platform specific module directory name as follows: $ # before patch $ python -c 'import sys; print sys.platform' cygwin_nt-4.01 $ python -c 'import sys; print sys.path' [..., '/usr/lib/python2.1/plat-cygwin_nt-4.01', ...] $ find /usr/lib/python2.1 -name '*cygwin*' -type d /usr/lib/python2.1/plat-cygwin_nt-4.01 $ # after patch $ python -c 'import sys; print sys.platform' cygwin $ python -c 'import sys; print sys.path' [..., '/usr/lib/python2.1/plat-cygwin', ...] $ find /usr/lib/python2.1 -name '*cygwin*' -type d /usr/lib/python2.1/plat-cygwin The second part changes sys.path (only when Python is run out of the build tree) and the directory names used by distutils when building extension modules: $ # before patch $ python -c 'import sys; print sys.path' [..., '/home/jt/src/Python-2.1/build/lib.cygwin_nt-4.0-1.3.2-i686-2.1'] $ find . -name '*cygwin*' ./build/lib.cygwin_nt-4.0-1.3.0-i686-2.1 ./build/temp.cygwin_nt-4.0-1.3.0-i686-2.1 $ # after patch $ python -c 'import sys; print sys.path' [..., '/home/jt/src/Python-2.1/build/lib.cygwin-1.3.2-i686-2.1'] $ find . -name '*cygwin*' ./build/lib.cygwin-1.3.2-i686-2.2 ./build/temp.cygwin-1.3.2-i686-2.2 If I don't receive any negative responses to this patch proposal by 2001/6/1 9:00 EDT, then I will submit this patch to the SourceForge Python Patch Manager. Thanks, Jason -- Jason Tishler Director, Software Engineering Phone: 732.264.8770 x235 Dot Hill Systems Corp. Fax: 732.264.8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com --ohmds5dUDiOjm0u1 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="python.patch" Index: configure.in =================================================================== RCS file: /cvsroot/python/python/dist/src/configure.in,v retrieving revision 1.218 diff -c -r1.218 configure.in *** configure.in 2001/05/11 16:10:56 1.218 --- configure.in 2001/05/28 01:10:03 *************** *** 66,71 **** --- 66,72 ---- MACHDEP="$ac_md_system$ac_md_release" case $MACHDEP in + cygwin*) MACHDEP="cygwin";; '') MACHDEP="unknown";; esac fi Index: Lib/distutils/util.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/util.py,v retrieving revision 1.63 diff -c -r1.63 util.py *** Lib/distutils/util.py 2001/03/23 17:30:26 1.63 --- Lib/distutils/util.py 2001/05/28 01:10:03 *************** *** 62,67 **** --- 62,68 ---- elif osname[:3] == "aix": return "%s-%s.%s" % (osname, version, release) elif osname[:6] == "cygwin": + osname = "cygwin" rel_re = re.compile (r'[\d.]+') m = rel_re.match(release) if m: --ohmds5dUDiOjm0u1 Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Sieve: cmu-sieve 2.0 Received: from dothill.com (artecon.artecon.com [206.2.34.1]) by ny.dothill.com (8.11.2/8.11.2) with ESMTP id f4Q06Cd16218 for ; Fri, 25 May 2001 20:06:13 -0400 Received: from niwa.cri.nz (clam.niwa.cri.nz [202.36.29.1]) by dothill.com (8.9.1b+Sun/8.9.1) with ESMTP id QAA22195 for ; Fri, 25 May 2001 16:48:01 -0700 (PDT) Received: from storm.niwa.cri.nz (storm.niwa.cri.nz [192.168.60.10]) by niwa.cri.nz (8.11.2/8.11.2) with ESMTP id f4Q068d08279 for ; Sat, 26 May 2001 12:06:08 +1200 (NZST) Received: from Hadfield (hadfield.greta.niwa.cri.nz [192.168.56.217]) by storm.niwa.cri.nz (8.11.1/8.11.2) with SMTP id f4Q068L74586 for ; Sat, 26 May 2001 12:06:08 +1200 (NZST) Message-ID: <000b01c0e577$b15b1820$d938a8c0@Hadfield> From: "Mark Hadfield" To: "Jason Tishler" Subject: Cygwin Python, sys.platform & distutils Date: Sat, 26 May 2001 12:06:20 +1200 Organization: NIWA MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Hello Jason I would like your opinion on another minor Cygwin Python distutils buglet. I recently installed Konrad Hinsen's Scientific Python package with the Python package you built for Cygwin. This package creates one C-extension DLL, which is supposed to be installed into /usr/lib/python2.1/site-packages/Scientific/cygwin_nt-4.01 ..but it is actually put into... /usr/lib/python2.1/site-packages/Scientific/cygwin_nt-4/01 ..where Python doesn't find it. Scientific Python's setup.py includes the following line ext_package = 'Scientific.'+sys.platform, I presume the '.' in sys.platform on the Cygwin system is being interpreted as a delimiter in the package's directory hierarchy. Konrad confirmed this interpretation, but also commented that he was not aware of any other platform which has a '.' in sys.platform. On the only other Unix platform I have easy access to, bash's MACHTYPE is alphaev6-dec-osf5.1 and Python's sys.platform is 'osf1V5'. So should Cygwin Python's sys.platform be calculated differently? --- Mark Hadfield m.hadfield@niwa.cri.nz http://katipo.niwa.cri.nz/~hadfield National Institute for Water and Atmospheric Research --ohmds5dUDiOjm0u1-- From BRADLEY.S.GASPARD@saic.com Wed May 30 14:38:01 2001 From: BRADLEY.S.GASPARD@saic.com (Gaspard, Bradley S) Date: Wed May 30 13:38:01 2001 Subject: [Distutils] SWIG Support Message-ID: I am a new distutils user. Can anyone tell me if the SWIG support is actually working and is there a way that it can support the -shadow option?? I am getting compile errors under distutils that I don't normally get (for an extention where I don't use the shadow option). Examples of setup.py scripts would be appreciated. SWIG1.1-883 Python 2.0 Thanks in advance. Brad From mats@laplaza.org Thu May 31 11:51:02 2001 From: mats@laplaza.org (Mats Wichmann) Date: Thu May 31 10:51:02 2001 Subject: [Distutils] some installer grumbles In-Reply-To: <20010529132936.C498@dothill.com> Message-ID: <5.1.0.14.1.20010531083657.00ac4e98@laplaza.org> I did something silly the other day in with PyXML, and ended up motivated to post a bugreport on the result. The developers kicked that back with the observation that it's a distutils issue. Sad to say, I've been on this list for a while but too busy to pay any real attention, these are probably old and thoroughly hashed-out issues, but I thought I'd toss this out here for comment before doing anything else with it: ==================== This is the result of operator error, but nonetheless... I accidentally launched an install of PyXML on a w2k system where it was already installed. I know the instructions say to remove old installations first (actually it wasn't an "old" installation, it was the same version, to be really nitpicky) as I said, Operator Error. However, at this point: (a) the existing installation is not detected with a bailout option ("blah blah already installed blah blah do you want to continue?") (b) there's no way to abort the installation once it starts (no cancel button) (c) you are prompted for EACH file as to whether to replace or not; there is no "yes to all" (or "no to all") so one would potentially have to click "yes" or "no" hundreds of times to complete for something like PyXML. ================== A little further comment: I ended up doing the NT/2000 equivalent of ps and kill: I brought up the task manager and blew the whole thing away. Needless to say, I don't think this is ideal or I wouldn't have gone to the effort of grumbling about it. ================== Here was the response to the report: The windows installer is created using the distutils bdist_wininst command. That means that we cannot easily change the user interface of the installation procedure. Please report this as a distutils bug in the Python project. ================== Any comments? Mats