From gward@cnri.reston.va.us Fri Oct 1 01:46:46 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Thu, 30 Sep 1999 20:46:46 -0400 Subject: [Distutils] distutils 0.1 problem and fix In-Reply-To: <3.0.5.32.19990929115703.009df100@mail.aracnet.com>; from Amos Latteier on Wed, Sep 29, 1999 at 11:57:03AM -0700 References: <3.0.5.32.19990929115703.009df100@mail.aracnet.com> Message-ID: <19990930204645.A14271@cnri.reston.va.us> On 29 September 1999, Amos Latteier said: > Three cheers for distutils 0.1! > > Unfortunately it fails to install itself on my NT box. Details, details. You Windows people are so picky. ;-) > running install_py > creating D:\Python1.5.2 > Traceback (innermost last): > File "setup.py", line 22, in ? > packages = ['distutils', 'distutils.command'], [...yikes -- at least my tracebacks aren't as bad as JPython's...] > File "distutils\util.py", line 318, in copy_tree > mkpath (dst, verbose=verbose) > File "distutils\util.py", line 74, in mkpath > raise DistutilsFileError, "%s: %s" % (head, errstr) > distutils.errors.DistutilsFileError: D:\Python1.5.2: File exists > > > To fix this I changed line 317 of distutils/util.py to: > > if not dry_run and not os.path.exists(os.path.normpath(dst)): Hmm, no, this is really mkpath's fault. But thanks for reminding me of the existence of normpath() -- somehow I forgot all about it. Anyone on a Windows box with Python and the Distutils handy, please try this out to see if I've fixed it. First, verify the bug: >>> from distutils.util import mkpath >>> mkpath ("\blah\") where "\blah" doesn't already exist on the current drive. Should blow up with that "File exists" error. Try it without the trailing backslash, too. Now apply this patch (in "Distutils-0.1\distutils"): *** util.py 1999/09/29 12:14:16 1.7 --- util.py 1999/10/01 00:25:18 *************** *** 39,40 **** --- 39,42 ---- + name = os.path.normpath (name) + if os.path.isdir (name): *************** *** 45,48 **** (head, tail) = os.path.split (name) - if not tail: # in case 'name' has trailing slash - (head, tail) = os.path.split (head) tails = [tail] # stack of lone dirs to create --- 47,48 ---- and try it again. Try it with and without the trailing backslash, on different and the same directories. Of course, I'll do all this tomorrow at work, where I have access to a Windows machine. Let me know if you find other bugs in mkpath though -- this is a surprisingly tricky little thing to get right. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From gward@cnri.reston.va.us Fri Oct 1 02:13:36 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Thu, 30 Sep 1999 21:13:36 -0400 Subject: [Distutils] Re: order on link In-Reply-To: <99093009361900.31817@almanac>; from Paul F. Dubois on Thu, Sep 30, 1999 at 09:33:19AM -0700 References: <19990923120928.C9298@cnri.reston.va.us> <99093009361900.31817@almanac> Message-ID: <19990930211336.B14297@cnri.reston.va.us> [Paul, I have cc'd this to the distutils sig, because I want the group's input on this. Hope this isn't too gross a violation of email etiquette...] On 30 September 1999, Paul F. Dubois said: > We are not getting anything loaded from our libraries on Linux. This > is because on Linux the .o's have to be before any of the libraries on > the command line. I suspect this is true on other platforms where the > order in which everything is listed is critical. > > You put the -L's -l's before the .o's. Hmmm, so I am. That's a bug. The fix is trivial: see lines 225-226 of distutils/unixccompiler.py -- just change the order of 'objects' and 'lib_opts'. Do you know if it ever matters where "-o" comes in? Currently, I have it last, and it could easily move too. Will a similar change have to be made to MSVCCompiler -- i.e is it as sensitive to order of link options as Unix compiler are? > Should there be an emergency escape hatch where I can list the exact > link line I want? At least while we are debugging this? Our feeling is > that this thing is close enough that we are dying to use it. I hope that won't be necessary. I've added one concession-to-reality that should partially address your needs, and have plans for another. The first is 'extra_preargs' and 'extra_postargs' parameters to all the link and compile methods in the compiler classes. For command-line driven compilers, the meaning is obvious -- just a list of strings that's inserted at the beginning or tacked on to the end of the command line. That's how I deal with "def" files now -- the 'build_ext' command looks for + ".def" in the directory of the first source file, and tacks that onto the MSVC linker command line with 'extra_postargs'. Works for me in theory (ie. I can generate sensible-looking MSVC command lines on my Linux box if I lie to Distutils and tell it it's using MSVC), but I dunno about the real world. For more science-fictiony cases (eg. driving a Mac compiler via whatever Apple's protocol for launching/talking to external programs is -- got an explanation from Jack Jansen a few weeks ago, but I forget the details), it's not clear if these will have any meaning, but using them kinda tosses portability out the window anyways. I still have to expose these options at the Distutils level -- ie. add 'extra_{cc,ld}_{pre,post}args' to the "build info" dictionary passed from the setup script to the 'build_ext' command. Again, using these will shoot portability to hell, but I can certainly see where it's needed. And you could always have extra code in setup.py to generate them for various platforms and thereby regain portability. The other trick I plan is to let you qualify a library name with a directory, so you could eg. have ext_modules = [('foo', {'source': [...], 'libraries': ['/devel/foo']})] and be guaranteed to link with /devel/libfoo.{a,so} (whichever the linker finds and prefers). I have to play around a bit to see how to make this work; I *hope* cc ... -Ldir1 -Ldir2 -llib1 -L/devel -lfoo -llib2 .. will do it but I'm not sure. If not I'll have to resolve /devel/foo to /devel/libfoo.{a,so} in the Distutils, which reeks of touchy portability problems. Got any better ideas? Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From dubois1@llnl.gov Fri Oct 1 03:10:02 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Thu, 30 Sep 1999 19:10:02 -0700 Subject: [Distutils] Re: order on link References: <19990923120928.C9298@cnri.reston.va.us> <99093009361900.31817@almanac> <19990930211336.B14297@cnri.reston.va.us> Message-ID: <000501bf0bb2$1257ee60$3c810b18@plstn1.sfba.home.com> > > Hmmm, so I am. That's a bug. The fix is trivial: see lines 225-226 of > distutils/unixccompiler.py -- just change the order of 'objects' and > 'lib_opts'. That did the trick! > Do you know if it ever matters where "-o" comes in? > Currently, I have it last, and it could easily move too. > Don't know. I doubt it. I did once run into a C compiler that had no -o option at all, however. You had to make the object file in the same directory as the source. > Will a similar change have to be made to MSVCCompiler -- i.e is it as > sensitive to order of link options as Unix compiler are? No idea. This sensitivity varies among Unix compilers, too. That leads to the nonsense of having to list a library twice. I'm too jet lagged to think about the other things you wrote. The package is useful to us even in its new state. Thanks. The .o's get left in the directory with setup.py; should be squirreled away someplace else. To allow for building for mutiple platforms it would be nice if the build products went into a platform-specific directory, eg. build/linux2. That way when you make a bug fix on one platform you don't have to blow everything away to do the next one. From gward@cnri.reston.va.us Fri Oct 1 13:33:58 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Fri, 1 Oct 1999 08:33:58 -0400 Subject: [Distutils] Re: order on link In-Reply-To: <000501bf0bb2$1257ee60$3c810b18@plstn1.sfba.home.com>; from Paul F. Dubois on Thu, Sep 30, 1999 at 07:10:02PM -0700 References: <19990923120928.C9298@cnri.reston.va.us> <99093009361900.31817@almanac> <19990930211336.B14297@cnri.reston.va.us> <000501bf0bb2$1257ee60$3c810b18@plstn1.sfba.home.com> Message-ID: <19991001083357.A14356@cnri.reston.va.us> On 30 September 1999, Paul F. Dubois said: > > Hmmm, so I am. That's a bug. The fix is trivial: see lines 225-226 of > > distutils/unixccompiler.py -- just change the order of 'objects' and > > 'lib_opts'. > > That did the trick! OK, I'll keep the fix for 0.11. > No idea. This sensitivity varies among Unix compilers, too. That leads to > the nonsense of having to list a library twice. Luckily, that's no problem with Distutils -- you just put it in the 'libraries' list twice, and it winds up on the command line twice. This will only handle mutual dependencies amongst libraries -- .o files are always elsewhere on the command line (because of line 225-226 in distutils/unixcompiler.py), so the system can't handle every possibility. > The .o's get left in the directory with setup.py; should be squirreled away > someplace else. To allow for building for mutiple platforms it would be nice > if the build products went into a platform-specific directory, eg. > build/linux2. That way when you make a bug fix on one platform you don't > have to blow everything away to do the next one. Yeah, I know about that. The status quo is no worse than a Makefile.pre.in system -- Numeric's current build system just drops .o files in the current directory -- so I figured it was OK if I did that in this version. There actually need to be *two* "build" directories: one for transient files (.o), and one for permanent files to install. And you're right, both of them need to be partitioned by platform, eg. "build.platlib-i86-linux" or "build.platlib-sparc-sun-solaris". My inclination would be to leave the source directories untouched, and put all intermediate and final files in other directories that are completely under Distutils control. Anyone see problems with that approach? Also, does anybody have Python code that divines that GNU style "cpu-vendor-os" string, or something similar? sys.platform is a start, but not enough because it doesn't say anything about the CPU architecture. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From robin@jessikat.demon.co.uk Sat Oct 2 15:10:59 1999 From: robin@jessikat.demon.co.uk (Robin Becker) Date: Sat, 2 Oct 1999 15:10:59 +0100 Subject: [Distutils] distutils & win9x Message-ID: Hi, I'm trying out distutils-0.1 and find a few problems. I tried to do python Setup.py -v install and got errors about C:\PYTHON already existing. This is because it appears that os.pathg.isdir('C:\\PYTHON\\') returns 0 for Win95 OSR2. I got round this by inserting the line name=os.path.dirname(name) at the beginning of mkpath in utils.py; however I am now getting the error running install running build running build_py creating build creating build\lib copying distutils\version.py -> build\lib\distutils Traceback (innermost last): File "Setup.py", line 22, in ? packages = ['distutils', 'distutils.command'], File "distutils\core.py", line 87, in setup dist.run_commands () File "distutils\core.py", line 377, in run_commands self.run_command (cmd) File "distutils\core.py", line 426, in run_command cmd_obj.run () File "distutils\command\install.py", line 279, in run self.run_peer ('build') File "distutils\core.py", line 710, in run_peer self.distribution.run_command (command) File "distutils\core.py", line 426, in run_command cmd_obj.run () File "distutils\command\build.py", line 47, in run self.run_peer ('build_py') File "distutils\core.py", line 710, in run_peer self.distribution.run_command (command) File "distutils\core.py", line 426, in run_command cmd_obj.run () File "distutils\command\build_py.py", line 82, in run self.build_packages () File "distutils\command\build_py.py", line 281, in build_packages self.build_module (module, module_file, package) File "distutils\command\build_py.py", line 247, in build_module self.copy_file (module_file, outfile) File "distutils\core.py", line 761, in copy_file self.distribution.dry_run) File "distutils\util.py", line 264, in copy_file _copy_file_contents (src, dst) File "distutils\util.py", line 187, in _copy_file_contents fdst = open(dst, 'wb') IOError: [Errno 2] No such file or directory: 'build\\lib\\distutils\\version.py' -- Robin Becker From gward@cnri.reston.va.us Sun Oct 3 22:27:25 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Sun, 3 Oct 1999 17:27:25 -0400 Subject: [Distutils] New snapshot available Message-ID: <19991003172724.A15760@cnri.reston.va.us> Hi all -- not ready for release 0.11 yet, but I just added some nice touches to the command-line syntax and semantics. In addition to the "classic" (already!) setup.py -nv build install you can now do things like this: setup.py -nv build -q install (build quietly, install verbosely) Also, I added a new global option '--force', which means to skip the lame dependency analysis that various bits of the Distutils do. This is necessary precisely because the dependency analysis is lame and will miss many things that might require a rebuild (at least when building C extensions). Thus you have the option to skip the lame dependency checking and rebuild everything. Yeah, sure, I'd *love* to reimplement "make" and "makedepend" in Python and thrown 'em into the Distutils, but this is an acceptable workaround. And of course you can use the lame dependency checking for some commands, and force rebuilds for others. Eg. simple timestamp checking is just fine for the 'build_py' command, but you might not trust it for 'build_ext' (eg. if you've been editing header files, which are not included in the dependency checking -- hey, I said it's lame!): setup.py -v build_py build_ext -f Hope this seems useful. If you want to try it, just get the latest from CVS, or download the snapshot from: http://www.python.org/sigs/distutils-sig/implementation.html And no, I haven't gotten to the bottom of what's going wrong with installing on Windows. Thanks for the bug reports -- I'll play around with it tomorrow at work. Enjoy! Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From gward@cnri.reston.va.us Sun Oct 3 22:31:15 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Sun, 3 Oct 1999 17:31:15 -0400 Subject: [Distutils] Windows wizards: I need porting help! Message-ID: <19991003173115.A15818@cnri.reston.va.us> Anyone who is a reasonably competent Windows/Python programmer can help me out here. (At least) two functions in the distutils.util module need help: 'mkpath()' and 'move_file()'. Actually, 'mkpath()' might be working fine now, but I'd like some of you Windows folks to take a look at it, beat on it, make sure it'll really work. Here's the code: ------------------------------------------------------------------------ # cache for by mkpath() -- in addition to cheapening redundant calls, # eliminates redundant "creating /foo/bar/baz" messages in dry-run mode PATH_CREATED = {} # I don't use os.makedirs because a) it's new to Python 1.5.2, and # b) it blows up if the directory already exists (I want to silently # succeed in that case). def mkpath (name, mode=0777, verbose=0, dry_run=0): """Create a directory and any missing ancestor directories. If the directory already exists, return silently. Raise DistutilsFileError if unable to create some directory along the way (eg. some sub-path exists, but is a file rather than a directory). If 'verbose' is true, print a one-line summary of each mkdir to stdout.""" global PATH_CREATED # XXX what's the better way to handle verbosity? print as we create # each directory in the path (the current behaviour), or only announce # the creation of the whole path? (quite easy to do the latter since # we're not using a recursive algorithm) name = os.path.normpath (name) if os.path.isdir (name): return if PATH_CREATED.get (name): return (head, tail) = os.path.split (name) tails = [tail] # stack of lone dirs to create while head and tail and not os.path.isdir (head): #print "splitting '%s': " % head, (head, tail) = os.path.split (head) #print "to ('%s','%s')" % (head, tail) tails.insert (0, tail) # push next higher dir onto stack #print "stack of tails:", tails # now 'head' contains the deepest directory that already exists # (that is, the child of 'head' in 'name' is the highest directory # that does *not* exist) for d in tails: #print "head = %s, d = %s: " % (head, d), head = os.path.join (head, d) if PATH_CREATED.get (head): continue if verbose: print "creating", head if not dry_run: try: os.mkdir (head) except os.error, (errno, errstr): raise DistutilsFileError, "%s: %s" % (head, errstr) PATH_CREATED[head] = 1 # mkpath () ------------------------------------------------------------------------ The other one, 'move_file()', almost certainly has portability problems. Please read it, bash on it, and help me fix it! Here's the code for it: ------------------------------------------------------------------------ def move_file (src, dst, verbose=0, dry_run=0): """Move a file 'src' to 'dst'. If 'dst' is a directory, the file will be moved into it with the same name; otherwise, 'src' is just renamed to 'dst'. Return the new full name of the file. Handles cross-device moves on Unix using 'copy_file()'. What about other systems???""" from os.path import exists, isfile, isdir, basename, dirname if verbose: print "moving %s -> %s" % (src, dst) if dry_run: return dst if not isfile (src): raise DistutilsFileError, \ "can't move '%s': not a regular file" % src if isdir (dst): dst = os.path.join (dst, basename (src)) elif exists (dst): raise DistutilsFileError, \ "can't move '%s': destination '%s' already exists" % \ (src, dst) if not isdir (dirname (dst)): raise DistutilsFileError, \ "can't move '%s': destination '%s' not a valid path" % \ (src, dst) copy_it = 0 try: os.rename (src, dst) except os.error, (num, msg): if num == errno.EXDEV: copy_it = 1 else: raise DistutilsFileError, \ "couldn't move '%s' to '%s': %s" % (src, dst, msg) if copy_it: copy_file (src, dst) try: os.unlink (src) except os.error, (num, msg): try: os.unlink (dst) except os.error: pass raise DistutilsFileError, \ ("couldn't move '%s' to '%s' by copy/delete: " + "delete '%s' failed: %s") % \ (src, dst, src, msg) return dst # move_file () ------------------------------------------------------------------------ Thanks! Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From gward@cnri.reston.va.us Mon Oct 4 13:30:01 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Mon, 4 Oct 1999 08:30:01 -0400 Subject: [Distutils] Oh yeah! one more feature Message-ID: <19991004083000.A15979@cnri.reston.va.us> Whoops, I totally forgot to mention the other potentially useful feature I added this weekend. Now, entries in the 'libraries' list can include a directory component, which will tell the compiler interface to instruct the C compiler to look only in that directory for that particular library. Umm, was that too opaque? Here's an example. Before this change, the example pil_setup.py included with the Distutils had this: ext_modules = \ [('_imaging', { 'sources': # ... # ... 'library_dirs': ['libImaging', '/usr/local/lib'], 'libraries': ['Imaging', 'jpeg', 'z', 'tcl8.0', 'tk8.0'] } This resulted (on Unix) in a link command like this: gcc -shared ... -LlibImaging -L/usr/local/lib \ -lImaging -ljpeg -lz -ltcl8.0 -ltk8.0 Now, we can specify that the "Imaging" library must come from the "libImaging" subdirectory, rather than being searched for all over the place. The current pil_setup.py has this: ext_modules = \ [('_imaging', { 'sources': # ... # ... 'library_dirs': ['/usr/local/lib'], 'libraries': ['libImaging/Imaging', 'jpeg', 'z', 'tcl8.0', 'tk8.0'] } Now the link command looks like this: gcc -shared ... -L/usr/local/lib \ libImaging/Imaging.a -ljpeg -lz -ltcl8.0 -ltk8.0 Note that this has the unpleasant consequence that the Distutils compiler class (UnixCCompiler, in this case) has to second-guess what the linker does to find a particular library. This isn't too hard for me using GCC, since GCC is fairly well documented in this regard. However I'm sure there are Unices where my rash assumptions (if "dir/foo" found in 'libraries' list, look for "dir/libfoo.so", then "dir/libfoo.a") will not hold. Of course, the directory can be absolute -- this shoots portability to hell, but should be welcome if you're using the Distutils as a private build tool and just want to be sure you're linking in *exactly* such-and-such version of a library. I know at least one person (hi Paul!) who will like this. And the implementation of this feature for MSVC is a wild guess on my part. Again, Windows people are kindly requested to take a look and see how close to reality my guess is. (See the 'gen_lib_options()' function in distutils.ccompiler, and then the 'find_library_file()' method in both distutils.unixccompiler and distutils.msvccompiler.) This code is all in yesterday's snapshot. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From arcege@shore.net Mon Oct 4 17:21:55 1999 From: arcege@shore.net (Michael P. Reilly) Date: Mon, 4 Oct 1999 12:21:55 -0400 (EDT) Subject: [Distutils] Oh yeah! one more feature In-Reply-To: <19991004083000.A15979@cnri.reston.va.us> from Greg Ward at "Oct 4, 99 08:30:01 am" Message-ID: <199910041621.MAA27453@northshore.shore.net> > Whoops, I totally forgot to mention the other potentially useful feature > I added this weekend. Now, entries in the 'libraries' list can include > a directory component, which will tell the compiler interface to > instruct the C compiler to look only in that directory for that > particular library. This can lead to some real problems. When given a shared object, most link editors (ld) will use that exact pathname to find the object at runtime, ignoring the -R (-rpath) and LD_LIBRARY_PATH (LIB_PATH) variables. The system had better determine when and how shared objects (.sl/.so) should be handled differently from archive files (.a). Also, considering that most testing is handled in the build environment (with those same pathnames), this type of problem might not be found until after installation. (Just as a rule, think of how the executable search path rules work. The library search path rules are similar: if there is a '/' in the filename, then do not search with LD_LIBRARY_PATH.) Does anyone know if this might happen with DLLs too? -Arcege -- ------------------------------------------------------------------------ | Michael P. Reilly, Release Engineer | Email: arcege@shore.net | | Salem, Mass. USA 01970 | | ------------------------------------------------------------------------ From gward@cnri.reston.va.us Mon Oct 4 17:28:23 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Mon, 4 Oct 1999 12:28:23 -0400 Subject: [Distutils] Oh yeah! one more feature In-Reply-To: <199910041621.MAA27453@northshore.shore.net>; from Michael P. Reilly on Mon, Oct 04, 1999 at 12:21:55PM -0400 References: <19991004083000.A15979@cnri.reston.va.us> <199910041621.MAA27453@northshore.shore.net> Message-ID: <19991004122822.A16138@cnri.reston.va.us> On 04 October 1999, Michael P. Reilly said: > > Whoops, I totally forgot to mention the other potentially useful feature > > I added this weekend. Now, entries in the 'libraries' list can include > > a directory component, which will tell the compiler interface to > > instruct the C compiler to look only in that directory for that > > particular library. > > This can lead to some real problems. When given a shared object, most > link editors (ld) will use that exact pathname to find the object at > runtime, ignoring the -R (-rpath) and LD_LIBRARY_PATH (LIB_PATH) > variables. The system had better determine when and how shared objects > (.sl/.so) should be handled differently from archive files (.a). Also, > considering that most testing is handled in the build environment (with > those same pathnames), this type of problem might not be found until > after installation. Hmmm, good point -- I hadn't considered this possibility. One possible "solution" (ahem) is to make the "libraries with a directory component" feature only look for static libraries. If you want to use a shared library, it had better work with -l/-L/-R. This is, I think, compatible with the two uses I see for this feature. The first use is how I demonstrated in pil_setup.py -- just link in a C library built for and with this extension module. In that case it'd be silly to make the C library shared, as we're building a shared object to load at runtime anyway. No point in having a small .so with the Python glue, which then loads a large .so with all the real C code. Just load one .so with both of them. The other use is to appease Paul Dubois and anyone else using the Distutils as a local build tool: Paul needs to be sure that he links with a specific development version of some library. Again, I don't see it as a big loss to say "if you want to link with library foo in /some/devel/dir, then it had better be a static library". Comments? Windows/DLL perspectives? Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From robin@alldunn.com Mon Oct 4 17:45:27 1999 From: robin@alldunn.com (Robin Dunn) Date: Mon, 4 Oct 1999 09:45:27 -0700 Subject: [Distutils] Oh yeah! one more feature References: <19991004083000.A15979@cnri.reston.va.us> <199910041621.MAA27453@northshore.shore.net> <19991004122822.A16138@cnri.reston.va.us> Message-ID: <00fa01bf0e87$ddac3e90$1a25d2d1@jenkondev.com> > > Comments? Windows/DLL perspectives? DLLs are normally searched for in this order (from memory so I may have left one out...): 1. the directory where the .exe file is located 2. the windows system directory 3. the windows directory 4. the directories on the PATH For DLLs that are dynamically loaded, like Python extension modules, you specify the whole path to the DLL. -- Robin Dunn Software Craftsman robin@AllDunn.com http://AllDunn.com/robin/ http://AllDunn.com/wxPython/ Check it out! From fcy@acm.org Wed Oct 6 16:16:24 1999 From: fcy@acm.org (Fred Yankowski) Date: Wed, 06 Oct 1999 10:16:24 -0500 Subject: [Distutils] distutils installation fails in WinNT; sys.exec_prefix empty Message-ID: <37FB67C8.6904B489@acm.org> Greetings, I just downloaded Distutils-0.1.tar.gz (which might be better named Distutils-0.1.tgz to accommodate WinZip), expanded it, and tried installing it by running 'python -v setup.py install'. It ran OK for a while, then took an exception in _init_nt() in sysconfig.py, apparently because sys.exec_prefix (and sys.prefix, for that matter) are both set to '' on my system. I just installed Python 1.5.2 yesterday, on a WinNT 4.0 (SP 5) machine. Other than choosing a non-default installation path (different volume), I think I took all defaults when running the Python installer. -- Fred Yankowski From dubois1@llnl.gov Fri Oct 8 16:04:54 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Fri, 8 Oct 1999 08:04:54 -0700 Subject: [Distutils] Installing applications Message-ID: <000a01bf119e$7ce439e0$3c810b18@plstn1.sfba.home.com> This is a multi-part message in MIME format. ------=_NextPart_000_0007_01BF1163.CE218140 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I know that I don't read a lot of what goes on in our manuals, = newsgroups and sigs, so if this problem has a known answer I would = appreciate a pointer. Distutils solves the "install a package" problem. Now, how do we solve = the "make it an application" problem? Here is what I mean. I have a Python package with a driver routine = inside. I need some way of starting up Python with that script as the = input. So I end up writing something like this example from the Pyfort = package, file pyfort: #!/usr/bin/env python=20 import Pyfort.driver import sys Pyfort.driver.run(sys.argv[1:]) The first problem for me is that this chooses the python on the path, = not necessarily the python into which I have installed this package. In = our environment we have multiple Pythons since we version our software = package. So if someone runs "pyfort" all is well, but if they do=20 /usr/local/cdat/experimental/bin/pyfort but their path finds /usr/local/cdat/public/bin/python, the import = Pyfort.driver gets the "public" rather than the "experimental" version. = But if I hardwire the path into the first line, I can't tar it up and = send it to somebody somewhere else. (Like to my customers, for example). Python has a -c option, but I can't expect the users to cram the above = into a -c line.=20 The second problem is that I had to write (and more seriously, install) = this little file in the first place. Seems dumb when all I want is the = effect of=20 python = /usr/local/cdat/experimental/lib/python1.5/site-packages/Pyfort/driver.py= We have another app where essentially we want to run python, import some = stuff, and go to the prompt. ------=_NextPart_000_0007_01BF1163.CE218140 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I know that I don't read a lot of what goes on in = our manuals,=20 newsgroups and sigs, so if this problem has a known answer I would = appreciate a=20 pointer.
 
Distutils solves the "install a package" problem. = Now, how do=20 we solve the "make it an application" problem?
 
Here is what I mean. I have a Python package with a = driver=20 routine inside. I need some way of starting up Python with that script = as the=20 input. So I end up writing something like this example from the Pyfort = package,=20 file pyfort:
 
#!/usr/bin/env python
import = Pyfort.driver
import=20 sys
Pyfort.driver.run(sys.argv[1:])
The first problem for me is that this chooses the = python on=20 the path, not necessarily the python into which I have installed this = package.=20 In our environment we have multiple Pythons since we version our = software=20 package. So if someone runs "pyfort" all is well, but if they do =
 
/usr/local/cdat/experimental/bin/pyfort=20 <args>
 
but their path finds = /usr/local/cdat/public/bin/python, the=20 import Pyfort.driver gets the "public" rather than the "experimental" = version.=20 But if I hardwire the path into the first line, I can't tar it up and = send it to=20 somebody somewhere else. (Like to my customers, for = example).
 
Python has a -c option, but I can't expect the users = to cram=20 the above into a -c line.
 
The second problem is that I had to write (and more = seriously,=20 install) this little file in the first place. Seems dumb when all I want = is the=20 effect of
python=20 /usr/local/cdat/experimental/lib/python1.5/site-packages/Pyfort/driver.py= =20 <args>
 
We have another app where essentially we want to run = python,=20 import some stuff, and go to the prompt.
 
------=_NextPart_000_0007_01BF1163.CE218140-- From jim@interet.com Fri Oct 8 20:35:48 1999 From: jim@interet.com (James C. Ahlstrom) Date: Fri, 08 Oct 1999 15:35:48 -0400 Subject: [Distutils] Installing applications References: <000a01bf119e$7ce439e0$3c810b18@plstn1.sfba.home.com> Message-ID: <37FE4794.96847DE@interet.com> > "Paul F. Dubois" wrote: > > I know that I don't read a lot of what goes on in our manuals, > newsgroups and sigs, so if this problem has a known answer I would > appreciate a pointer. > > Distutils solves the "install a package" problem. Now, how do we solve > the "make it an application" problem? In general, this is an unsolved problem. We have it too since we are shipping a major Python app to customers, as well as using this app and a few others internally. See: ftp.interet.com/pub/bootmodule.html ftp.interet.com/pub/pylib.html > #!/usr/bin/env python > import Pyfort.driver > import sys > Pyfort.driver.run(sys.argv[1:]) > The first problem for me is that this chooses the python on the path, > not necessarily the python into which I have installed this package. > ... > But if I hardwire the path into the first line, I can't tar > it up and send it to somebody somewhere else. (Like to my customers, > for example). As a Windows dweeb, it took me a while to understand this. It looks like you have compiled the various Python versions with a different $prefix, so each carries its own custom PYTHONPATH. Otherwise the Python binary used can not affect which version of Pyfort/driver.py gets imported. Although the directory of the Python binary is appended to PYTHONPATH, these are all bin directories, and the packages are in lib directories so the modification to PYTHONPATH doesn't matter. The directory of the Python script is prepended to PYTHONPATH so specifying a script name almost works. The problem is that you put pyfort.py into a bin directory too, so prepending this to PYTHONPATH does not help get the package from the lib directory either. I think if you move pyfort to the lib directory it will work. But you must still specify which pyfort you want. If the lib directory is on PATH, "pyfort" works. For a different pyfort, you will need to specify the whole path. I am not sure I have understood this correctly, so please complain if I did not. Jim Ahlstrom From fcy@acm.org Fri Oct 22 20:56:57 1999 From: fcy@acm.org (Fred Yankowski) Date: Fri, 22 Oct 1999 14:56:57 -0500 Subject: [Distutils] patch to utils.py for Windows os.path.isdir glitch Message-ID: <19991022145657.A43602@enteract.com> I was finally able to install the distutils 0.1 package on my Windows NT machine running Python 1.5.2, after getting around two problems: 1) I had a very old python.exe sitting in my PATH, which when run would report itself as Python version 1.5.2 but would cause sys.prefix and sys.exec_prefix to be empty because Python couldn't find the expected libraries relative to the python.exe location. I deleted that old python.exe and now sys.prefix and sys.exec_prefix are set correctly. Thanks to Mark Hammond for suggesting the fix. 2) Running 'python -v start.py install' would result in an exception in utils.py when os.mkdir() would be called on an existing directory. It seems that on Windows, os.path.isdir() does not accept trailing backslashes on a directory name. For example, os.path.isdir('c:\\progra~1') returns 1 on my machine, but os.path.isdir('c:\\progra~1\\') returns 0. I tweaked utils.py to account for that and now the installation seems to finish OK. I have attached the patch with my suggested change. -- Fred Yankowski fred@OntoSys.com tel: +1.630.879.1312 Principal Consultant www.OntoSys.com fax: +1.630.879.1370 OntoSys, Inc 38W242 Deerpath Rd, Batavia, IL 60510, USA -- =================================================================== RCS file: RCS/util.py retrieving revision 1.1 retrieving revision 1.2 diff -c -r1.1 -r1.2 *** util.py 1999/10/22 18:32:41 1.1 --- util.py 1999/10/22 19:35:03 1.2 *************** *** 37,50 **** # the creation of the whole path? (quite easy to do the latter since # we're not using a recursive algorithm) if os.path.isdir (name): return if PATH_CREATED.get (name): return (head, tail) = os.path.split (name) - if not tail: # in case 'name' has trailing slash - (head, tail) = os.path.split (head) tails = [tail] # stack of lone dirs to create while head and tail and not os.path.isdir (head): --- 37,52 ---- # the creation of the whole path? (quite easy to do the latter since # we're not using a recursive algorithm) + (head, tail) = os.path.split (name) + if not tail: # in case 'name' has trailing slash + name = head + if os.path.isdir (name): return if PATH_CREATED.get (name): return (head, tail) = os.path.split (name) tails = [tail] # stack of lone dirs to create while head and tail and not os.path.isdir (head): From gward@cnri.reston.va.us Sat Oct 23 21:33:33 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Sat, 23 Oct 1999 16:33:33 -0400 Subject: [Distutils] patch to utils.py for Windows os.path.isdir glitch In-Reply-To: <19991022145657.A43602@enteract.com>; from Fred Yankowski on Fri, Oct 22, 1999 at 02:56:57PM -0500 References: <19991022145657.A43602@enteract.com> Message-ID: <19991023163333.A47@cnri.reston.va.us> On 22 October 1999, Fred Yankowski said: > I was finally able to install the distutils 0.1 package on my Windows > NT machine running Python 1.5.2, after getting around two problems: Great! > 1) I had a very old python.exe sitting in my PATH, which when run > would report itself as Python version 1.5.2 but would cause > sys.prefix and sys.exec_prefix to be empty because Python couldn't > find the expected libraries relative to the python.exe location. I > deleted that old python.exe and now sys.prefix and sys.exec_prefix > are set correctly. Thanks to Mark Hammond for suggesting the fix. Ahh, good -- when I read your first post, I was inclined to shoot back with a "Sounds like a broken Python installation", but I know very very little about Python on Windows so I kept my mouth shut. Looks like I would have been right. Should I add code somewhere in Distutils to detect empty sys.prefix or exec_prefix? This really sounds like a clear indication that the Python installation is badly broken, so there's no point in trying to install anything because it simply won't work. > 2) Running 'python -v start.py install' would result in an exception > in utils.py when os.mkdir() would be called on an existing > directory. It seems that on Windows, os.path.isdir() does not > accept trailing backslashes on a directory name. For example, > os.path.isdir('c:\\progra~1') returns 1 on my machine, but > os.path.isdir('c:\\progra~1\\') returns 0. I tweaked utils.py to > account for that and now the installation seems to finish OK. I > have attached the patch with my suggested change. Yep, that's already been fixed. It just took me a couple of weeks to get around to trying it out on a Windows machine myself. Now done. See my next post... Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913 From gward@cnri.reston.va.us Sat Oct 23 21:37:54 1999 From: gward@cnri.reston.va.us (Greg Ward) Date: Sat, 23 Oct 1999 16:37:54 -0400 Subject: [Distutils] ANNOUNCE: Distutils 0.1.1 Message-ID: <19991023163753.B47@cnri.reston.va.us> Finally got around to fixing those Windows portability bugs. Also hacked on the "dist" command a bit, and did some interesting stuff with global options. Here's the change log: release 0.1.1: * fixed 'mkpath()' function so it should work under DOS/Windows * changes to how we link C code: - under Unix, object files precede -l options on link command line - libraries now can have a directory component, which forces the library to be searched for in only that directory * added --force and --quiet global options * made global options (--verbose, --dry-run, and now --force and --quiet too) valid at each command as well as for the whole distribution * 'dist' command now works on Unices other than Linux; generates tar, tar.Z, tar.gz, and ZIP files. Still Unix-dependent though. RTFS, or the CVS logs, for details. Oh yeah: download it from http://www.python.org/sigs/distutils-sig/ Enjoy! Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913