From trentm at ActiveState.com Fri Sep 2 02:26:56 2005 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 1 Sep 2005 17:26:56 -0700 Subject: [Distutils] Fixing the stdlib In-Reply-To: <4310BE7F.90109@colorstudy.com> References: <4310BE7F.90109@colorstudy.com> Message-ID: <20050902002656.GA31548@ActiveState.com> [Ian Bicking wrote] > Vaguely off-topic, but I figured someone here might know... > > Anyway, I want to provide backported stdlib modules to older versions of > Python. Most of these are trivial to backport, so it's just a matter of > accumulating them in one place. Is it important in your use case to magically stick them in the same place as the standard locations? I remember a few people mentioning that they have their own "compat" (or "compat23", ...) modules kicking around. It might be nice to have a collective "compat" or "backported" package out there, then users could explicitly: try: from compat import tempfile except ImportError: import tempfile Dunno if that matches up with what you wanted to do. > Thoughts? An import hook that does the magic? Trent -- Trent Mick TrentM at ActiveState.com From bob at redivi.com Fri Sep 2 02:36:55 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 1 Sep 2005 17:36:55 -0700 Subject: [Distutils] Fixing the stdlib In-Reply-To: <20050902002656.GA31548@ActiveState.com> References: <4310BE7F.90109@colorstudy.com> <20050902002656.GA31548@ActiveState.com> Message-ID: <19C1BEA2-EBAA-4309-AED4-4F79122A5CCF@redivi.com> On Sep 1, 2005, at 5:26 PM, Trent Mick wrote: > [Ian Bicking wrote] > >> Vaguely off-topic, but I figured someone here might know... >> >> Anyway, I want to provide backported stdlib modules to older >> versions of >> Python. Most of these are trivial to backport, so it's just a >> matter of >> accumulating them in one place. >> > > Is it important in your use case to magically stick them in the same > place as the standard locations? I remember a few people mentioning > that > they have their own "compat" (or "compat23", ...) modules kicking > around. It might be nice to have a collective "compat" or "backported" > package out there, then users could explicitly: > > try: > from compat import tempfile > except ImportError: > import tempfile > > Dunno if that matches up with what you wanted to do. > > >> Thoughts? >> > > An import hook that does the magic? You can also install a .pth file that looks like this, which is a little bit less magical than an import hook: import sys; sys.path.insert(0, 'modules-that-override-the-stdlib-are- here') Of course, you can't override anything that is imported before the site module is, but most of those are built-in or have baked-in references in the interpreter anyway. -bob From pje at telecommunity.com Sat Sep 3 07:06:45 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 03 Sep 2005 01:06:45 -0400 Subject: [Distutils] setuptools 0.6a1 candidate 2 snapshot In-Reply-To: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050903010319.01b89080@mail.telecommunity.com> There were bug reports about bdist_rpm, and some issues installing on SuSE Linux, which has a patched version of distutils for no particular reason (that Google knows about, anyway). In the process of fixing these, I found some other problems and went ahead and fixed those too. So, there's now an 0.6a1c2 snapshot you can use, via the instructions below. Please let me know about any problems; if everything looks good, we'll go to 0.6a1 sometime next week. At 12:03 AM 8/22/2005 -0400, Phillip J. Eby wrote: >This thing's been in limbo too long, so I've made a prerelease >snapshot. You can get it with: > > easy_install -f http://peak.telecommunity.com/snapshots/ setuptools > >I have fixed all reported bugs, and implemented quite a few new features >like MD5 checking (which works even with PyPI's md5 data) and an enhanced >"build_ext --inplace" that lets you switch between different Python >interpreters without having to recompile your extensions. (The extensions >are built in build/, then copied to the source tree whenever you >"build_ext" using a different interpreter than the last.) > >If you have any projects using setuptools, I encourage you to give this >snapshot a try, so I can find out if there are still any lurking bugs. If >there are no bug reports by next weekend, this will get pushed out as 0.6a1. >If you want to look at the documentation, use CVS or browse the >EasyInstall.txt, setuptools.txt, and pkg_resources.txt files found at: > > >http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/ > >(I just made some changes to these, however, that may take a few hours to >update in SourceForge's anonymous CVS.) From ianb at colorstudy.com Wed Sep 7 00:21:59 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 06 Sep 2005 17:21:59 -0500 Subject: [Distutils] setuptools: Utility of setup() arguments Message-ID: <431E1687.1060102@colorstudy.com> I'm wondering, is there any reason to use the keyword arguments to setup() that write files to the .egg-info directory, instead of writing those files myself (assuming I was going to hardcode them in the Python source anyway). Specifically I'm coming back to some deployment stuff, and I'd like to make an installation process that installs all the dependencies using --multi-version, and then makes the deployed application require those exact versions -- this way nothing else on the machine should be effected by the installation. Doing this in an automated way seems reasonably straight-forward; read the requires.txt file, figure out what the versions are we installed just now (which might have build tags), and rewrite that file. I can rewrite that file easily, not so easy to rewrite setup.py. Relatedly, I'd prefer having entry_points.txt in my repository, since it's annoying when I forget to update it, or when I run "sudo python setup.py develop" it'll be written out as root, and I'll have to fix the permissions before rerunning "setup.py egg_info". Frankly I'd rather have PKG-INFO in the repository as a text file too, though it concerns me more that it is formed from several keyword arguments instead of one. So, to the degree this works (and I haven't tried it with PKG-INFO), is there any reason to use setup() arguments over editing the files it generates directly? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Wed Sep 7 00:45:50 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 06 Sep 2005 18:45:50 -0400 Subject: [Distutils] setuptools: Utility of setup() arguments In-Reply-To: <431E1687.1060102@colorstudy.com> Message-ID: <5.1.1.6.0.20050906182923.01b5abe8@mail.telecommunity.com> At 05:21 PM 9/6/2005 -0500, Ian Bicking wrote: >Specifically I'm coming back to some deployment stuff, and I'd like to >make an installation process that installs all the dependencies using >--multi-version, and then makes the deployed application require those >exact versions -- this way nothing else on the machine should be >effected by the installation. Doing this in an automated way seems >reasonably straight-forward; read the requires.txt file, figure out what >the versions are we installed just now (which might have build tags), >and rewrite that file. I can rewrite that file easily, not so easy to >rewrite setup.py. Relatedly, I'd prefer having entry_points.txt in my >repository, since it's annoying when I forget to update it, or when I >run "sudo python setup.py develop" it'll be written out as root, and >I'll have to fix the permissions before rerunning "setup.py egg_info". > >Frankly I'd rather have PKG-INFO in the repository as a text file too, >though it concerns me more that it is formed from several keyword >arguments instead of one. > >So, to the degree this works (and I haven't tried it with PKG-INFO), is >there any reason to use setup() arguments over editing the files it >generates directly? Only that I personally found editing the files directly to be an enormous pain. :) There are numerous ways to solve your problem, however. For example, you can create a command that runs egg_info and then rewrites the requires.txt file, and run it before commands that need egg_info. For example, you could create an alias like: develop = egg_info strict_requires develop that runs your "exact_requires" to rewrite the file before running develop. Or, you can use a function when passing install_requires: setup( ... install_requires = maybe_exact("Req1", "Req2", ...) ) And have the "maybe_exact" function figure out if it needs to add exact version data to the requirements, perhaps reading from another configuration file. I'm pretty sure that these are just two of the more obvious ways to do it; there are bound to be others. Which way works best depends somewhat on your scenario for how you'd like to activate or deactivate the feature. From ianb at colorstudy.com Wed Sep 7 00:59:59 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 06 Sep 2005 17:59:59 -0500 Subject: [Distutils] setuptools: Utility of setup() arguments In-Reply-To: <5.1.1.6.0.20050906182923.01b5abe8@mail.telecommunity.com> References: <5.1.1.6.0.20050906182923.01b5abe8@mail.telecommunity.com> Message-ID: <431E1F6F.90001@colorstudy.com> Phillip J. Eby wrote: > Only that I personally found editing the files directly to be an > enormous pain. :) Why? The only issue I see is that setup.py puts all that data in one place, where otherwise it's several separate files to work with. But anyway, I guess I'm thinking about tools to make that manipulation easier or more consistent, so I'm apt to consider them separately. > There are numerous ways to solve your problem, however. For example, > you can create a command that runs egg_info and then rewrites the > requires.txt file, and run it before commands that need egg_info. For > example, you could create an alias like: > > develop = egg_info strict_requires develop > > that runs your "exact_requires" to rewrite the file before running develop. I think I'd probably want something like: strict_install = egg_info --tag-date strict_requires install I feel like using a specific set of requirements means that the package itself is a new version, hence --tag-date. And the whole thing is best done as an entirely new command (strict_install) since this is something I'd only do for some installations (not during development, for instance), and I don't really want to override an existing command. I'm a little wary of rewriting something that egg_info writes, because egg_info can rewrite it later. But I suppose the command alias makes it into a single operation, so it doesn't seem as fragile. Though one advantage of editing the files is that I could apply strict_requires to a tag and commit those changes, encapsulating the exact environment that was used when creating that tag. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Wed Sep 7 01:21:28 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 06 Sep 2005 19:21:28 -0400 Subject: [Distutils] setuptools: Utility of setup() arguments In-Reply-To: <431E1F6F.90001@colorstudy.com> References: <5.1.1.6.0.20050906182923.01b5abe8@mail.telecommunity.com> <5.1.1.6.0.20050906182923.01b5abe8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050906191914.01b6cc18@mail.telecommunity.com> At 05:59 PM 9/6/2005 -0500, Ian Bicking wrote: >Phillip J. Eby wrote: >>Only that I personally found editing the files directly to be an enormous >>pain. :) > >Why? Because I have to create a new file. And if it's a new project, a directory to go with it. Adding arguments to setup() is much easier. Also, setup() arguments are somewhat easier to document. >I'm a little wary of rewriting something that egg_info writes, because >egg_info can rewrite it later. But I suppose the command alias makes it >into a single operation, so it doesn't seem as fragile. Though one >advantage of editing the files is that I could apply strict_requires to a >tag and commit those changes, encapsulating the exact environment that was >used when creating that tag. If your command or commands use a configuration file, or store the info in setup.cfg, you can still do that. From joachim at ee.ucla.edu Thu Sep 8 20:07:04 2005 From: joachim at ee.ucla.edu (Joachim Dahl) Date: Thu, 08 Sep 2005 11:07:04 -0700 Subject: [Distutils] distutils questions Message-ID: <43207DC8.1040809@ee.ucla.edu> Hi, I have a regular setup script for an extension module using core.setup. Currently I have to manually run a make script to build the C object files, and then link them using the Python setup script. I would like to include the make file into the setup script, and I have been looking at how to tweak distutils, but I am quite in over my head here... My problem is that the C source files are compiled several times into different object depending on the preprocessor options, and thus I cannot simply include the source files into the "sources" list in the Extension module, e.g., as glpk = Extension('glpk', libraries = ['glpk'], include_dirs = [ GLPK_INC_DIR ], library_dirs = [ GLPK_LIB_DIR ], sources = ['C/glpk.c'] ) extmods += [glpk]; I need the setup script to build the module like this: gcc -Ddef1 -c foo.c -o foo_def1.o gcc -Ddef2 -c foo.c -o foo_def2.o gcc foo_def1.o foo_def2.o -o myext_module.o Is there currently a (relatively simple) way to include something like that into a distutils setup script, or is it easier to keep running a separate make file to build the different object files before linking? - Joachim From pje at telecommunity.com Thu Sep 8 20:18:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 08 Sep 2005 14:18:38 -0400 Subject: [Distutils] distutils questions In-Reply-To: <43207DC8.1040809@ee.ucla.edu> Message-ID: <5.1.1.6.0.20050908141722.01f3fea8@mail.telecommunity.com> At 11:07 AM 9/8/2005 -0700, Joachim Dahl wrote: >I need the setup script to build the module like this: >gcc -Ddef1 -c foo.c -o foo_def1.o >gcc -Ddef2 -c foo.c -o foo_def2.o >gcc foo_def1.o foo_def2.o -o myext_module.o > >Is there currently a (relatively simple) way to include something like >that into a distutils setup script, or is it easier to keep running a >separate >make file to build the different object files before linking? ZODB does this by making a short 'foo_def1.c' that looks like: #define def1 #include "foo.c" and repeating. So, if you don't mind doing that to get your various foo_def*.o files, then you're good to go, without a makefile. From scottakirkwood at gmail.com Fri Sep 9 04:06:09 2005 From: scottakirkwood at gmail.com (Scott Kirkwood) Date: Thu, 8 Sep 2005 23:06:09 -0300 Subject: [Distutils] Having problems with "scripts" section Message-ID: Ok, I give up. I'm using setuptools==0.5a13 and have the following problem. I'm trying (under Ubuntu Linux) to have my easy_install.py install a script to run my utility. However, it's unable to find the utility and I don't know why. The latest incarnation I changed the name of the script to runWx. Here's what happends when I run it: $ runWx Traceback (most recent call last): File "/usr/bin/runWx", line 4, in ? pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') File "/usr/lib/python2.4/site-packages/setuptools-0.5a13-py2.4.egg/pkg_resources.py", line 111, in run_script require(dist_spec)[0].metadata.run_script(script_name, ns) File "/usr/lib/python2.4/site-packages/setuptools-0.5a13-py2.4.egg/pkg_resources.py", line 627, in run_script raise ResolutionError("No script named %r" % script_name) pkg_resources.ResolutionError: No script named 'runWx' it can't find runWx, ok, why not? here's the egg list: $ unzip -l dist/wxOptParse-0.1.2-py2.4.egg Archive: dist/wxOptParse-0.1.2-py2.4.egg Length Date Time Name -------- ---- ---- ---- 21025 09-08-05 12:54 wxoptparse/__init__.py 20091 09-08-05 22:26 wxoptparse/__init__.pyc 1315 09-08-05 22:26 EGG-INFO/PKG-INFO 11 09-08-05 22:26 EGG-INFO/top_level.txt 0 09-08-05 22:26 EGG-INFO/zip-safe 63 09-08-05 22:15 EGG-INFO/scripts/runWx -------- ------- 42505 6 files Here's the contents of /usr/bin/wxRun which appears to have been created: $ cat /usr/bin/runWx #!/usr/bin/python # EASY-INSTALL-SCRIPT: 'wxOptParse==0.1.2','runWx' import pkg_resources pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') Contents of my original runWx ?#!python import wxoptparse wxoptparse.handleCommandLine() My setup.py file: from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages # Notes to self: # python setup.py sdist --formats=zip # To create the zip file # python setup.py --command-packages=setuptools.command bdist_egg # To create the egg file # python setup.py register # to register with PyPI # setup(name="wxOptParse", version="0.1.2", description="wxOptParse: run the command line options from a dialog box.", long_description="""\ wxOptParse is a Python program that brings up a graphical representation of the options that another python program has for the command line, via the optparse module. What this means is that if if you have a program that uses optparse you can click on checkboxes, edit boxes, combo boxes, etc. instead of using the command line. """, author="Scott Kirkwood", author_email="scottakirkwood at gmail.com", url="http://wxoptparse.berlios.de/", download_url=' http://developer.berlios.de/project/showfiles.php?group_id=2209&release_id=3368 ', packages=find_packages(exclude='tests'), scripts=["runWx"], keywords=['wxOptParse', 'optparse', 'python', 'wxPython'], license='GNU GPL', zip_safe=True, platforms=['POSIX', 'Windows', 'MacOS X'], classifiers=[ 'Development Status :: 3 - Alpha', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: GNU General Public License (GPL)', 'Operating System :: OS Independent', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Microsoft :: Windows', 'Operating System :: POSIX', 'Programming Language :: Python', ], # package_data={'': '*.xml'}, # install_requires=['wxPython>=2.5.3'], # Doesn't work on my machine ) And my MANIFEST.in file: include setup.py include ez_setup.py include runWx include wxoptparse/runWx include README.txt include docs/*.css include docs/*.rst include docs/*.gif include docs/*.html include docs/*.png include wxoptparse/*.py Plus you can find all these files and more at http://developer.berlios.de/projects/wxoptparse/ Any help would be appreciated, this program won't be very useful if it's can't be run from the command line. -Scott -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050908/247b7eb2/attachment.htm From pje at telecommunity.com Fri Sep 9 06:07:18 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Sep 2005 00:07:18 -0400 Subject: [Distutils] Having problems with "scripts" section In-Reply-To: Message-ID: <5.1.1.6.0.20050909000238.01cdbee8@mail.telecommunity.com> At 11:06 PM 9/8/2005 -0300, Scott Kirkwood wrote: >Ok, I give up. I'm using setuptools==0.5a13 and have the following problem. >I'm trying (under Ubuntu Linux) to have my easy_install.py install a >script to run my utility. However, it's unable to find the utility and I >don't know why. >The latest incarnation I changed the name of the script to runWx. Here's >what happends when I run it: >$ runWx >Traceback (most recent call last): > File "/usr/bin/runWx", line 4, in ? > pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') > File > "/usr/lib/python2.4/site-packages/setuptools-0.5a13-py2.4.egg/pkg_resources.py", > line 111, in run_script > require(dist_spec)[0].metadata.run_script(script_name, ns) > File > "/usr/lib/python2.4/site-packages/setuptools-0.5a13-py2.4.egg/pkg_resources.py", > line 627, in run_script > raise ResolutionError("No script named %r" % script_name) >pkg_resources.ResolutionError: No script named 'runWx' > >it can't find runWx, ok, why not? Can you try with setuptools 0.6a1c2? Use: easy_install.py -Uf http://peak.telecommunity.com/snapshots setuptools to upgrade. Rebuild and reinstall your project. If it still has a problem, please let me know. I remember fixing some script-related bugs between 0.5 and 0.6, so this might be something that's already fixed. If not, point me to the sources and I'll see if I can reproduce the problem. From scottakirkwood at gmail.com Sat Sep 10 04:29:34 2005 From: scottakirkwood at gmail.com (Scott Kirkwood) Date: Fri, 9 Sep 2005 23:29:34 -0300 Subject: [Distutils] Having problems with "scripts" section In-Reply-To: <5.1.1.6.0.20050909134627.01cda018@mail.telecommunity.com> References: <5.1.1.6.0.20050909000238.01cdbee8@mail.telecommunity.com> <5.1.1.6.0.20050909115949.01cde6f8@mail.telecommunity.com> <5.1.1.6.0.20050909134627.01cda018@mail.telecommunity.com> Message-ID: With Phillips's help I figured out the problem: * * scott at scott:~/scott/python/wxoptparse$ runWx Traceback (most recent call last): File "/usr/bin/runWx", line 4, in ? pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') File "/usr/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg/pkg_resources.py", line 407, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg/pkg_resources.py", line 914, in run_script raise ResolutionError("No script named %r" % script_name) pkg_resources.ResolutionError: No script named 'runWx' scott at scott:~/scott/python/wxoptparse$ cd .. scott at scott:~/scott/python$ runWx usage: wxoptparse.py ** * *Basically, I was testing within my source directory and it was getting confused when it saw a folder called wxoptparse. The error was not very helpful in this case. * **distutils-sig at python.org* On 9/9/05, Phillip J. Eby wrote: > > At 01:11 PM 9/9/2005 -0300, Scott Kirkwood wrote: > >Sorry I forgot to add the script to subversion, that's not the problem. > > > >I've updated subversion now, so the scripts directory should be OK. > > Here's what I did. I'm running on cygwin, not Linux, but it seems like it > should be similar. In any case, I'm not getting the problem. I wonder if > perhaps you have a PYTHONPATH set up that is somehow finding the wrong > version of your egg? (i.e., a version that doesn't have the script). Just > a thought. > > pje at pje-2 ~/projects/wxoptparse > $ svn up > A scripts/runWx > Updated to revision 7. > > pje at pje-2 ~/projects/wxoptparse > $ python setup.py install > running install > running bdist_egg > running egg_info > writing ./wxOptParse.egg-info/PKG-INFO > writing top-level names to ./wxOptParse.egg-info/top_level.txt > installing library code to build/bdist.cygwin-1.5.12-i686/egg > running install_lib > running build_py > installing scripts to build/bdist.cygwin-1.5.12-i686/egg/EGG-INFO/scripts > running install_scripts > running build_scripts > copying scripts/runWx -> build/scripts-2.3 > creating build/bdist.cygwin-1.5.12-i686/egg/EGG-INFO/scripts > copying build/scripts-2.3/runWx -> > build/bdist.cygwin-1.5.12-i686/egg/EGG-INFO/s > cripts > changing mode of build/bdist.cygwin-1.5.12-i686/egg/EGG-INFO/scripts/runWx > to 77 > 7 > copying ./wxOptParse.egg-info/PKG-INFO -> > build/bdist.cygwin-1.5.12-i686/egg/EGG > -INFO > copying ./wxOptParse.egg-info/top_level.txt -> > build/bdist.cygwin-1.5.12-i686/eg > g/EGG-INFO > creating dist > creating 'dist/wxOptParse-0.1.2-py2.3.egg' and adding > 'build/bdist.cygwin-1.5.12 > -i686/egg' to it > removing 'build/bdist.cygwin-1.5.12-i686/egg' (and everything under it) > Processing wxOptParse-0.1.2-py2.3.egg > Copying wxOptParse-0.1.2-py2.3.egg to /usr/lib/python2.3/site-packages > Adding wxOptParse 0.1.2 to easy-install.pth file > Installing runWx script to /usr/bin > > Installed /usr/lib/python2.3/site-packages/wxOptParse-0.1.2-py2.3.egg > Processing dependencies for wxOptParse==0.1.2 > > pje at pje-2 ~/projects/wxoptparse > $ runWx > Traceback (most recent call last): > File "/usr/bin/runWx", line 4, in ? > pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') > File "/home/pje/setuptools/pkg_resources.py", line 407, in run_script > self.require(requires)[0].run_script(script_name, ns) > File "/home/pje/setuptools/pkg_resources.py", line 927, in run_script > exec script_code in namespace, namespace > File "/usr/bin/runWx", line 3, in ? > import pkg_resources > File "build/bdist.cygwin-1.5.12-i686/egg/wxoptparse/__init__.py", line > 5, in ? > > ImportError: No module named elementtree > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050909/73797aa6/attachment.htm From pje at telecommunity.com Sat Sep 10 04:42:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Sep 2005 22:42:42 -0400 Subject: [Distutils] Having problems with "scripts" section In-Reply-To: References: <5.1.1.6.0.20050909134627.01cda018@mail.telecommunity.com> <5.1.1.6.0.20050909000238.01cdbee8@mail.telecommunity.com> <5.1.1.6.0.20050909115949.01cde6f8@mail.telecommunity.com> <5.1.1.6.0.20050909134627.01cda018@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050909224102.02a2e3f0@mail.telecommunity.com> At 11:29 PM 9/9/2005 -0300, Scott Kirkwood wrote: >With Phillips's help I figured out the problem: > >scott at scott:~/scott/python/wxoptparse$ runWx >Traceback (most recent call last): > File "/usr/bin/runWx", line 4, in ? > pkg_resources.run_script('wxOptParse==0.1.2', 'runWx') > File > "/usr/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg/pkg_resources.py", > line 407, in run_script > self.require(requires)[0].run_script(script_name, ns) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg/pkg_resources.py", > line 914, in run_script > raise ResolutionError("No script named %r" % script_name) >pkg_resources.ResolutionError: No script named 'runWx' >scott at scott:~/scott/python/wxoptparse$ cd .. >scott at scott:~/scott/python$ runWx >usage: wxoptparse.py > >Basically, I was testing within my source directory and it was getting >confused when it saw a folder called wxoptparse. The error was not very >helpful in this case. When testing, use 'setup.py develop' or 'setup.py test', and you won't have this issue; the "develop" command sets up the scripts so they run from your source tree, rather than trying to run them from an egg. From pau at newtral.org Tue Sep 13 13:06:38 2005 From: pau at newtral.org (Pau Aliagas) Date: Tue, 13 Sep 2005 13:06:38 +0200 (CEST) Subject: [Distutils] how to build an rpm with Egg-ified projects? Message-ID: Hi all. I've posted this question to the sqlobject list, as sqlobject has moved to an easy-setup intallation to include formencode (also egg-ified) and take advantage of setuptools. I'm trying to build Fedora rpms for sqlobect as I have been doing lately, using the "usual" python templates, but I see no way to achieve it since the setuptools change, pardon my ignorance, and I've tried really hard. I attach the 2 self-made spec files I use, that used to build as a non-root user just executing: $ rpmbuild -ba python-sqlobject.spec Now I get these errors: ---8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--- Traceback (most recent call last): File "setup.py", line 47, in ? extras_require={ File "/usr/lib/python2.3/site-packages/PIL/__init__.py", line 51, in setup File "/usr/lib/python2.3/distutils/core.py", line 149, in setup dist.run_commands() File "/home/pau/LnxZip/RPM/BUILD/SQLObject-0.6.1/setuptools-0.5a13-py2.3.egg/setuptools/dist.py", line 417, in run_commands File "/home/pau/LnxZip/RPM/BUILD/SQLObject-0.6.1/setuptools-0.5a13-py2.3.egg/setuptools/dist.py", line 426, in install_eggs File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/home/pau/LnxZip/RPM/BUILD/SQLObject-0.6.1/setuptools-0.5a13-py2.3.egg/setuptools/command/bdist_egg.py", line 173, in run File "/home/pau/LnxZip/RPM/BUILD/SQLObject-0.6.1/setuptools-0.5a13-py2.3.egg/setuptools/command/bdist_egg.py", line 161, in call_command File "/usr/lib/python2.3/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/usr/lib/python2.3/distutils/command/install_lib.py", line 92, in run self.byte_compile(outfiles) File "/usr/lib/python2.3/distutils/command/install_lib.py", line 128, in byte_compile dry_run=self.dry_run) File "/usr/lib/python2.3/distutils/util.py", line 434, in byte_compile raise ValueError, \ ValueError: invalid prefix: filename 'build/bdist.linux-i686/egg/sqlobject/boundattributes.py' doesn't start with '/home/pau/LnxZip/tmp/python-sqlobject-0.6.1-94.pau-root-pau' error: Bad exit status from /home/pau/LnxZip/tmp/rpm-tmp.54280 (%install) ---8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--- What I want is a way to distribute an rpm, no need to be able o hae multiple versions. I can make and require the needed RPMS, like formecode Can anybody enlighten me on how to do it? I'll really appreciate it as I want to upgrade and contribute, if possible. Pau -------------- next part -------------- # TemplateVer: 0.4 %{?!python:%define python python} %{?!pybasever:%{expand:%%define pybasever %(%{__python} -c "import sys ; print sys.version[:3]")}} %{!?python_sitearch: %define python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib(1)")} %{!?python_sitelib: %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} %define origname SQLObject Name: %{python}-sqlobject Version: 0.6.1 Release: 94.pau Summary: SQLObject is an object-relational mapper. Group: Development/Libraries License: LGPL URL: http://sqlobject.org Source0: %{origname}-%{version}.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) BuildArch: noarch BuildRequires: %{python}-devel Requires: python-abi = %{pybasever} %description SQLObject is an *object-relational mapper*. It allows you to translate RDBMS table rows into Python objects, and manipulate those objects to transparently manipulate the database. In using SQLObject, you will create a class definition that will describe how the object translates to the database table. SQLObject will produce the code to access the database, and update the database with your changes. The generated interface looks similar to any other interface, and callers need not be aware of the database backend. %prep %setup -n %{origname}-%{version} -q %build CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py build %install rm -rf $RPM_BUILD_ROOT %{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT --record=INSTALLED_FILES.tmp # Ghost optimized sed 's/\(.*\.pyo\)/%ghost \1/' < INSTALLED_FILES.tmp > INSTALLED_FILES find $RPM_BUILD_ROOT%{python_sitearch} -type d \ | sed "s|^$RPM_BUILD_ROOT|%dir |" >> INSTALLED_FILES %clean rm -rf $RPM_BUILD_ROOT %files -f INSTALLED_FILES %defattr(-,root,root,-) %doc README.txt docs/* %changelog * Mon Aug 29 2005 Pau Aliagas 0.6.1-94.pau - latest CVS - require FormEncode * Mon Jul 4 2005 Pau Aliagas 0.6.1-93.pau - latest CVS * Wed Jun 1 2005 Pau Aliagas 0.6.1-92.pau - latest CVS with SingleJoin - update spec file * Wed May 25 2005 Pau Aliagas 0.6.1-91.pau - modify installed files to fix x86_64 version * Tue May 24 2005 Pau Aliagas 0.6.1-91 - svn version - try to update to MySQL-python-1.2.0 to access mysql-4.1 * Wed Apr 24 2005 Pau Aliagas 0.6.1-90 - svn version * Wed Apr 20 2005 Pau Aliagas 0.6.1-1 - new version * Wed Sep 22 2004 Jeff Pitman 0.6-1 - new version * Tue Sep 07 2004 Jeff Pitman 0.5.2-1 - new version -------------- next part -------------- # TemplateVer: 0.4 %{?!python:%define python python} %{?!pybasever:%{expand:%%define pybasever %(%{__python} -c "import sys ; print sys.version[:3]")}} %{!?python_sitearch: %define python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib(1)")} %{!?python_sitelib: %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} %define origname FormEncode Name: %{python}-formencode Version: 0.2.2 Release: 2088.1 Summary: HTML form validation, generation, and convertion package Group: Development/Libraries License: Python Software Foundation License URL: http://formencode.org/ Source0: %{origname}-%{version}.tar.bz2 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) BuildArch: noarch BuildRequires: %{python}-devel Requires: python-abi = %{pybasever} %description FormEncode performs look-before-you-leap validation. The idea being that you check all the data related to an operation, then apply it. %prep %setup -n %{origname}-%{version} -q %build CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py build %install rm -rf $RPM_BUILD_ROOT %{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT --record=INSTALLED_FILES.tmp # Ghost optimized sed 's/\(.*\.pyo\)/%ghost \1/' < INSTALLED_FILES.tmp > INSTALLED_FILES find $RPM_BUILD_ROOT%{python_sitearch} -type d \ | sed "s|^$RPM_BUILD_ROOT|%dir |" >> INSTALLED_FILES %clean rm -rf $RPM_BUILD_ROOT %files -f INSTALLED_FILES %defattr(-,root,root,-) %doc docs/* %changelog * Tue Sep 13 2005 Pau Aliagas 0.2.2-2088.1 - updated version from svn trunk * Mon Aug 29 2005 Pau Aliagas 1.0-763 - first version from svn trunk From elvelind at gmail.com Tue Sep 13 21:07:03 2005 From: elvelind at gmail.com (Elvelind Grandin) Date: Tue, 13 Sep 2005 21:07:03 +0200 Subject: [Distutils] question about setuptools Message-ID: <5035593a05091312075e961d76@mail.gmail.com> I dont know if this is the right place to ask but I know i'v read a couple of mails on the list about setuptools so I will give it a try. I seem to have a problem with setuptools. I install all packages with --home=~ (using .pydistutils.cfg) . this means I need to use require to import a egg, which is fine. But I cant import pkg_resources.require since it is installed in ~ and needs to be required itself. cheers elvelind grandin From pje at telecommunity.com Tue Sep 13 21:26:37 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Sep 2005 15:26:37 -0400 Subject: [Distutils] question about setuptools In-Reply-To: <5035593a05091312075e961d76@mail.gmail.com> Message-ID: <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> At 09:07 PM 9/13/2005 +0200, Elvelind Grandin wrote: >I seem to have a problem with setuptools. I install all packages with >--home=~ (using >.pydistutils.cfg) . this means I need to use require to import a egg, >which is fine. But I cant import pkg_resources.require since it is >installed in ~ and needs to be required itself. See: http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/EasyInstall.txt?rev=HEAD&view=auto under "Non-Root Installation" for details of the best way to do this; it may require a slight adjustment to your layout, but it will then allow you to use .pth files in your home-based setup. You will have to clear out your --home setup from .pydistutils.cfg, though. (Also, you may need the latest snapshot of setuptools (0.6a1c2) for the described installation procedure to work, as older versions may try to install to the "real" Python installation instead of your alternate home, when following the procedure given on the page above.) From pje at telecommunity.com Tue Sep 13 21:56:23 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Sep 2005 15:56:23 -0400 Subject: [Distutils] how to build an rpm with Egg-ified projects? In-Reply-To: Message-ID: <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> At 01:06 PM 9/13/2005 +0200, Pau Aliagas wrote: >What I want is a way to distribute an rpm, no need to be able o hae >multiple versions. I can make and require the needed RPMS, like formecode > >Can anybody enlighten me on how to do it? I'll really appreciate it as I >want to upgrade and contribute, if possible. Are you using the very latest setuptools (0.6a1c2)? It is the only version with working bdist_rpm support, as far as I know. From ianb at colorstudy.com Tue Sep 13 21:58:20 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 13 Sep 2005 14:58:20 -0500 Subject: [Distutils] how to build an rpm with Egg-ified projects? In-Reply-To: <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> References: <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> Message-ID: <43272F5C.3000700@colorstudy.com> Phillip J. Eby wrote: > At 01:06 PM 9/13/2005 +0200, Pau Aliagas wrote: > > >>What I want is a way to distribute an rpm, no need to be able o hae >>multiple versions. I can make and require the needed RPMS, like formecode >> >>Can anybody enlighten me on how to do it? I'll really appreciate it as I >>want to upgrade and contribute, if possible. > > > Are you using the very latest setuptools (0.6a1c2)? It is the only version > with working bdist_rpm support, as far as I know. Incidentally, is it time for ez_setup.py to be updated to install 0.6? It seems like it's time. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Tue Sep 13 22:03:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Sep 2005 16:03:54 -0400 Subject: [Distutils] how to build an rpm with Egg-ified projects? In-Reply-To: <43272F5C.3000700@colorstudy.com> References: <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050913160303.02011388@mail.telecommunity.com> At 02:58 PM 9/13/2005 -0500, Ian Bicking wrote: >Phillip J. Eby wrote: >>At 01:06 PM 9/13/2005 +0200, Pau Aliagas wrote: >> >>>What I want is a way to distribute an rpm, no need to be able o hae >>>multiple versions. I can make and require the needed RPMS, like formecode >>> >>>Can anybody enlighten me on how to do it? I'll really appreciate it as I >>>want to upgrade and contribute, if possible. >> >>Are you using the very latest setuptools (0.6a1c2)? It is the only >>version with working bdist_rpm support, as far as I know. > >Incidentally, is it time for ez_setup.py to be updated to install 0.6? It >seems like it's time. Yeah, I'll try to go ahead and make the 0.6a1 release official tonight. From rkern at ucsd.edu Wed Sep 14 01:44:34 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 13 Sep 2005 16:44:34 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> References: <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > See: > > http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/EasyInstall.txt?rev=HEAD&view=auto > > under "Non-Root Installation" for details of the best way to do this; it > may require a slight adjustment to your layout, but it will then allow you > to use .pth files in your home-based setup. You will have to clear out > your --home setup from .pydistutils.cfg, though. Do you think it might be possible for ez_setup.py and easy_install.py to interpret the --home and --prefix options sensibly? Two people I've tried to convert to setuptools work exclusively on Linux with non-root installations. --prefix was the first thing they tried because it's the standard way for distutils to do this kind of thing. I suspect many other people will do the same. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pje at telecommunity.com Wed Sep 14 02:11:50 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Sep 2005 20:11:50 -0400 Subject: [Distutils] question about setuptools In-Reply-To: References: <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> At 04:44 PM 9/13/2005 -0700, Robert Kern wrote: >Phillip J. Eby wrote: > > > See: > > > > > http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/EasyInstall.txt?rev=HEAD&view=auto > > > > under "Non-Root Installation" for details of the best way to do this; it > > may require a slight adjustment to your layout, but it will then allow you > > to use .pth files in your home-based setup. You will have to clear out > > your --home setup from .pydistutils.cfg, though. > >Do you think it might be possible for ez_setup.py and easy_install.py to >interpret the --home and --prefix options sensibly? They already respect these options. The problem is that Python's 'site' module will *only* recognize .pth files in a very limited number of places, which means that installing setuptools itself won't work with those options. > Two people I've >tried to convert to setuptools work exclusively on Linux with non-root >installations. --prefix was the first thing they tried because it's the >standard way for distutils to do this kind of thing. I suspect many >other people will do the same. --prefix only makes sense if you've set things up the way shown on the page above, and if you set it up that way, you won't need --prefix. I realize the situation sucks, but unfortunately I'm not the one who made it suck, and am not in a position to fix it. Guido previously vetoed Bob Ippolito's proposal for more sane .pth processing on platforms other than Mac. From rkern at ucsd.edu Wed Sep 14 03:43:13 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 13 Sep 2005 18:43:13 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> References: <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 04:44 PM 9/13/2005 -0700, Robert Kern wrote: > >>Phillip J. Eby wrote: >> >>>See: >> >>http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/EasyInstall.txt?rev=HEAD&view=auto >> >>>under "Non-Root Installation" for details of the best way to do this; it >>>may require a slight adjustment to your layout, but it will then allow you >>>to use .pth files in your home-based setup. You will have to clear out >>>your --home setup from .pydistutils.cfg, though. >> >>Do you think it might be possible for ez_setup.py and easy_install.py to >>interpret the --home and --prefix options sensibly? > > They already respect these options. The problem is that Python's 'site' > module will *only* recognize .pth files in a very limited number of places, > which means that installing setuptools itself won't work with those options. I wasn't suggesting that it try to modify the .pth-enabled directories itself, but that easy_install.py and ez_setup.py (especially) should be able to calculate --site-dirs from --prefix. And that ez_setup.py should actually take the --prefix option on the command line gracefully. There are many systems which use other prefixes and have already set up their sitecustomize.py (or site.py) to enable .pth handling for them. Lots of packages use .pth's. [From a CVS checkout of setuptools on a Debian unstable system:] """ $ sudo python2.4 ez_setup.py --prefix=/usr/local . usage: ez_setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: ez_setup.py --help [cmd1 cmd2 ...] or: ez_setup.py --help-commands or: ez_setup.py cmd --help error: option --prefix not recognized """ When the prefix is set in ~/.pydistutils.cfg, the egg gets installed, but not the .pth's: """ $ sudo ./ez_setup.py . Processing . Running setup.py -q bdist_egg --dist-dir /usr/local/kern_src/python/nondist/sandbox/setuptools/egg-dist-tmp-85Xeyn Installing easy_install.py script to /usr/local/bin Installed /usr/local/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg Because this distribution was installed --multi-version or --install-dir, before you can import modules from this package in an application, you will need to 'import pkg_resources' and then use a 'require()' call similar to one of these examples, in order to select the desired version: pkg_resources.require("setuptools") # latest installed version pkg_resources.require("setuptools==0.6a1c2") # this exact version pkg_resources.require("setuptools>=0.6a1c2") # this version or higher Processing dependencies for setuptools==0.6a1c2 """ But if --site-dirs is also provided, everything works: """ $ sudo python2.4 ez_setup.py --site-dirs=/usr/local/lib/python2.4/site-packages . Processing . Running setup.py -q bdist_egg --dist-dir /usr/local/kern_src/python/nondist/sandbox/setuptools/egg-dist-tmp--GRqoe Adding setuptools 0.6a1c2 to easy-install.pth file Installing easy_install.py script to /usr/local/bin Installed /usr/local/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg Processing dependencies for setuptools==0.6a1c2 """ My request is that if --prefix is provided, then the value of --site-dirs be calculated from it. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pje at telecommunity.com Wed Sep 14 05:21:57 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Sep 2005 23:21:57 -0400 Subject: [Distutils] question about setuptools In-Reply-To: References: <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> At 06:43 PM 9/13/2005 -0700, Robert Kern wrote: >My request is that if --prefix is provided, then the value of >--site-dirs be calculated from it. I'd suggest using PYTHONHOME instead, since that will actually make Python run from that directory. However, my main concern is not making these path calculations any more screwy than they already have to be, as they are already quite complex. So, if you can suggest a patch to implement your desired behavior that doesn't break anything else and doesn't make my head explode upon reviewing it, I'll be happy to check it in. :) From rkern at ucsd.edu Wed Sep 14 06:42:49 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 13 Sep 2005 21:42:49 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> References: <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 06:43 PM 9/13/2005 -0700, Robert Kern wrote: > >>My request is that if --prefix is provided, then the value of >>--site-dirs be calculated from it. > > I'd suggest using PYTHONHOME instead, since that will actually make Python > run from that directory. That's not what these users need, however. Python is installed to /usr and the standard library and all of the third-party packages that are part of Debian's packaging system go under /usr/lib/python2.x/ . Third-party packages that are compiled by the user go under /usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes this is by setting --prefix=/usr/local for distutils. > However, my main concern is not making these path > calculations any more screwy than they already have to be, as they are > already quite complex. So, if you can suggest a patch to implement your > desired behavior that doesn't break anything else and doesn't make my head > explode upon reviewing it, I'll be happy to check it in. :) The path part isn't hard. If --prefix or --home is provided, the install command will calculate the path for you and place it in install.install_lib . The hard part seems to be figuring out whether the user explicitly provided --prefix or --install-lib or neither. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pau at newtral.org Wed Sep 14 07:41:30 2005 From: pau at newtral.org (Pau Aliagas) Date: Wed, 14 Sep 2005 07:41:30 +0200 (CEST) Subject: [Distutils] how to build an rpm with Egg-ified projects? In-Reply-To: <5.1.1.6.0.20050913160303.02011388@mail.telecommunity.com> References: <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> <5.1.1.6.0.20050913155516.01eb1038@mail.telecommunity.com> <5.1.1.6.0.20050913160303.02011388@mail.telecommunity.com> Message-ID: On Tue, 13 Sep 2005, Phillip J. Eby wrote: >>>> What I want is a way to distribute an rpm, no need to be able o hae >>>> multiple versions. I can make and require the needed RPMS, like formecode >>>> >>>> Can anybody enlighten me on how to do it? I'll really appreciate it as I >>>> want to upgrade and contribute, if possible. >>> >>> Are you using the very latest setuptools (0.6a1c2)? It is the only >>> version with working bdist_rpm support, as far as I know. >> >> Incidentally, is it time for ez_setup.py to be updated to install 0.6? It >> seems like it's time. > > Yeah, I'll try to go ahead and make the 0.6a1 release official tonight. I guess then that, when SQLObject and FormEncode include latest and greatest setuptools, it should work transparently, so I wait for it and keep you posted. Thanks Pau From pje at telecommunity.com Wed Sep 14 08:07:19 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 02:07:19 -0400 Subject: [Distutils] question about setuptools In-Reply-To: References: <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> At 09:42 PM 9/13/2005 -0700, Robert Kern wrote: >Phillip J. Eby wrote: > > At 06:43 PM 9/13/2005 -0700, Robert Kern wrote: > > > >>My request is that if --prefix is provided, then the value of > >>--site-dirs be calculated from it. > > > > I'd suggest using PYTHONHOME instead, since that will actually make Python > > run from that directory. > >That's not what these users need, however. Python is installed to /usr >and the standard library and all of the third-party packages that are >part of Debian's packaging system go under /usr/lib/python2.x/ . >Third-party packages that are compiled by the user go under >/usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes >this is by setting --prefix=/usr/local for distutils. Well, as long as it ends up on sys.path *and* processes .pth files, it should be fine. Question: are these /usr/local/* versions on sys.path *before* the /usr versions? Or after? > > However, my main concern is not making these path > > calculations any more screwy than they already have to be, as they are > > already quite complex. So, if you can suggest a patch to implement your > > desired behavior that doesn't break anything else and doesn't make my head > > explode upon reviewing it, I'll be happy to check it in. :) > >The path part isn't hard. If --prefix or --home is provided, the install >command will calculate the path for you and place it in >install.install_lib . The hard part seems to be figuring out whether the >user explicitly provided --prefix or --install-lib or neither. I'll take your word for it. :) Again, I'll be happy to look at a patch, but I don't have the bandwidth to figure out this particular setup. One possible hack: assume that any directory on sys.path whose basename is "site-packages" is a valid --site-dir. The worst that can do is fail. :) From pje at telecommunity.com Wed Sep 14 08:08:05 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 02:08:05 -0400 Subject: [Distutils] setuptools 0.6a1 is officially released Message-ID: <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> and is available from PyPI. The latest bootstrap script is at: http://peak.telecommunity.com/dist/ez_setup.py And updated HTML documentation is available at: http://peak.telecommunity.com/DevCenter/setuptools (developers/deployers) http://peak.telecommunity.com/DevCenter/EasyInstall (administrators/users) http://peak.telecommunity.com/DevCenter/PkgResources (runtime reference) There are way too many new features to list coherently, but some highlights include better support for non-root installs, RPM support, performance enhancements, a new "entry point" system for linking plugins to extensible apps and frameworks, a lot more documentation, and more than a few bug fixes. Please send questions, comments, and bug reports to distutils-sig at python.org. From rkern at ucsd.edu Wed Sep 14 08:27:29 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 13 Sep 2005 23:27:29 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> References: <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 09:42 PM 9/13/2005 -0700, Robert Kern wrote: > >>Phillip J. Eby wrote: >> >>>At 06:43 PM 9/13/2005 -0700, Robert Kern wrote: >>> >>> >>>>My request is that if --prefix is provided, then the value of >>>>--site-dirs be calculated from it. >>> >>>I'd suggest using PYTHONHOME instead, since that will actually make Python >>>run from that directory. >> >>That's not what these users need, however. Python is installed to /usr >>and the standard library and all of the third-party packages that are >>part of Debian's packaging system go under /usr/lib/python2.x/ . >>Third-party packages that are compiled by the user go under >>/usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes >>this is by setting --prefix=/usr/local for distutils. > > Well, as long as it ends up on sys.path *and* processes .pth files, it > should be fine. Not for installing setuptools itself with ez_setup.py . It doesn't recognize /usr/local/... as a site-dir so it doesn't install the .pth files. It assumes that it must be a multi-version install. > Question: are these /usr/local/* versions on sys.path > *before* the /usr versions? Or after? After. Does it matter? /usr/local/* doesn't contain the stdlib; it doesn't override anything. >>>However, my main concern is not making these path >>>calculations any more screwy than they already have to be, as they are >>>already quite complex. So, if you can suggest a patch to implement your >>>desired behavior that doesn't break anything else and doesn't make my head >>>explode upon reviewing it, I'll be happy to check it in. :) >> >>The path part isn't hard. If --prefix or --home is provided, the install >>command will calculate the path for you and place it in >>install.install_lib . The hard part seems to be figuring out whether the >>user explicitly provided --prefix or --install-lib or neither. > > I'll take your word for it. :) Again, I'll be happy to look at a patch, > but I don't have the bandwidth to figure out this particular setup. I've attached my original attempt. As I mentioned it doesn't actually differentiate between the cases where --prefix is provided and --install-dir is. > One possible hack: assume that any directory on sys.path whose basename is > "site-packages" is a valid --site-dir. The worst that can do is fail. :) That's probably a decent rule. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: st-prefix.diff Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050913/dd60be6e/st-prefix.diff From belica at emm.sk Wed Sep 14 12:36:00 2005 From: belica at emm.sk (belica) Date: Wed, 14 Sep 2005 12:36:00 +0200 Subject: [Distutils] Detected as SPAM: TEST Message-ID: <000001c5b918$19c7df10$1801810a@stano> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050914/9259b3d9/attachment.htm From dangoor at gmail.com Wed Sep 14 16:26:13 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Wed, 14 Sep 2005 10:26:13 -0400 Subject: [Distutils] setuptools 0.6a1 is officially released In-Reply-To: <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> References: <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> Message-ID: <3f085ecd050914072675f3fee3@mail.gmail.com> I get a 404 when trying to install via ez_setup: python ez_setup.py Downloading http://www.python.org/packages/source/s/setuptools/setuptools-0.6a1-py2.4.egg urllib2.HTTPError: HTTP Error 404: Not Found The actual URL appears to be: http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a1-py2.4.egg Kevin On 9/14/05, Phillip J. Eby wrote: > and is available from PyPI. The latest bootstrap script is at: > > http://peak.telecommunity.com/dist/ez_setup.py > > And updated HTML documentation is available at: > > http://peak.telecommunity.com/DevCenter/setuptools (developers/deployers) > http://peak.telecommunity.com/DevCenter/EasyInstall (administrators/users) > http://peak.telecommunity.com/DevCenter/PkgResources (runtime reference) > > There are way too many new features to list coherently, but some highlights > include better support for non-root installs, RPM support, performance > enhancements, a new "entry point" system for linking plugins to extensible > apps and frameworks, a lot more documentation, and more than a few bug fixes. > > Please send questions, comments, and bug reports to distutils-sig at python.org. > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From dangoor at gmail.com Wed Sep 14 17:53:15 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Wed, 14 Sep 2005 11:53:15 -0400 Subject: [Distutils] setuptools 0.6a1 is officially released In-Reply-To: <3f085ecd050914072675f3fee3@mail.gmail.com> References: <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> <3f085ecd050914072675f3fee3@mail.gmail.com> Message-ID: <3f085ecd05091408534e36c834@mail.gmail.com> On 9/14/05, Kevin Dangoor wrote: > I get a 404 when trying to install via ez_setup: > The actual URL appears to be: > http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a1-py2.4.egg To clarify: the package is available at this URL. But, it would appear that ez_setup is trying to get it from the correct URL, but the Cheeseshop doesn't have the file there. Kevin From dangoor at gmail.com Wed Sep 14 17:57:14 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Wed, 14 Sep 2005 11:57:14 -0400 Subject: [Distutils] setuptools 0.6a1 is officially released In-Reply-To: <3f085ecd05091408534e36c834@mail.gmail.com> References: <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> <3f085ecd050914072675f3fee3@mail.gmail.com> <3f085ecd05091408534e36c834@mail.gmail.com> Message-ID: <3f085ecd0509140857400af5df@mail.gmail.com> On 9/14/05, Kevin Dangoor wrote: > On 9/14/05, Kevin Dangoor wrote: > > I get a 404 when trying to install via ez_setup: > > The actual URL appears to be: > > http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a1-py2.4.egg > > To clarify: the package is available at this URL. But, it would appear > that ez_setup is trying to get it from the correct URL, but the > Cheeseshop doesn't have the file there. That wasn't clear at all. My daughter is teething and woke up a lot last night, so I've got a bit of brain freeze... Let's try that again: The URL that I put in my original reply and that appears above does contain 0.6a1, so you can get the package from the Cheeseshop by following that URL. ez_setup is going to the correct URL, but the file is missing (in the packages/source tree on cheeseshop). So, all that needs to happen is the source package needs to appear in the cheeseshop as ez_setup is expecting. Kevin From pje at telecommunity.com Wed Sep 14 18:44:09 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 12:44:09 -0400 Subject: [Distutils] setuptools 0.6a1 is officially released In-Reply-To: <3f085ecd0509140857400af5df@mail.gmail.com> References: <3f085ecd05091408534e36c834@mail.gmail.com> <5.1.1.6.0.20050914020751.03260c58@mail.telecommunity.com> <3f085ecd050914072675f3fee3@mail.gmail.com> <3f085ecd05091408534e36c834@mail.gmail.com> Message-ID: <5.1.1.6.0.20050914124333.01b65320@mail.telecommunity.com> At 11:57 AM 9/14/2005 -0400, Kevin Dangoor wrote: >So, all that needs to happen is the source package needs to appear in >the cheeseshop as ez_setup is expecting. That appears to be fixed, and I've also changed ez_setup now to download from the right place when bootstrapping. From pje at telecommunity.com Wed Sep 14 19:02:49 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 13:02:49 -0400 Subject: [Distutils] question about setuptools In-Reply-To: References: <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> At 11:27 PM 9/13/2005 -0700, Robert Kern wrote: >Index: setuptools/command/easy_install.py >=================================================================== RCS >file: >/cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v >retrieving revision 1.28 diff -u -r1.28 easy_install.py --- >setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 >+++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ >-141,6 +141,10 @@ self.set_undefined_options('install', >('record', 'record')) normpath = map(normalize_path, >sys.path) self.all_site_dirs = get_site_dirs() >+ install_cmd = self.distribution.get_command_obj('install') >+ if install_cmd.prefix is not None: >+ >self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) >+ if self.site_dirs is not None: site_dirs = >[ os.path.expanduser(s.strip()) for s in >self.site_dirs.split(','_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig Your patch came through horribly mangled, as you can see. You probably need to make it an attachment rather than pasting it into your email. From rkern at ucsd.edu Thu Sep 15 03:08:06 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 14 Sep 2005 18:08:06 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> References: <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> Message-ID: <4328C976.2000003@ucsd.edu> Phillip J. Eby wrote: > At 11:27 PM 9/13/2005 -0700, Robert Kern wrote: > > >>Index: setuptools/command/easy_install.py >>=================================================================== RCS >>file: >>/cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v >>retrieving revision 1.28 diff -u -r1.28 easy_install.py --- >>setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 >>+++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ >>-141,6 +141,10 @@ self.set_undefined_options('install', >>('record', 'record')) normpath = map(normalize_path, >>sys.path) self.all_site_dirs = get_site_dirs() >>+ install_cmd = self.distribution.get_command_obj('install') >>+ if install_cmd.prefix is not None: >>+ >>self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) >>+ if self.site_dirs is not None: site_dirs = >>[ os.path.expanduser(s.strip()) for s in >>self.site_dirs.split(','_______________________________________________ >>Distutils-SIG maillist - Distutils-SIG at python.org >>http://mail.python.org/mailman/listinfo/distutils-sig > > Your patch came through horribly mangled, as you can see. You probably > need to make it an attachment rather than pasting it into your email. Odd. It *was* sent as an attachment. Thunderbird on OS X sees it fine through GMane. Maybe GMane mangled it somewhere along the line. Or perhaps line-endings are getting confused. I'm cc'ing this to you with an attached patch that uses CRLF line endings. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: st-prefix.diff Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050914/d8192283/st-prefix.diff From pje at telecommunity.com Thu Sep 15 03:41:26 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 21:41:26 -0400 Subject: [Distutils] question about setuptools In-Reply-To: <4328C976.2000003@ucsd.edu> References: <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> At 06:08 PM 9/14/2005 -0700, Robert Kern wrote: >Index: setuptools/command/easy_install.py >=================================================================== >RCS file: >/cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v >retrieving revision 1.28 >diff -u -r1.28 easy_install.py >--- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 >+++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 >@@ -141,6 +141,10 @@ > self.set_undefined_options('install', ('record', 'record')) > normpath = map(normalize_path, sys.path) > self.all_site_dirs = get_site_dirs() >+ install_cmd = self.distribution.get_command_obj('install') >+ if install_cmd.prefix is not None: >+ >self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) >+ > if self.site_dirs is not None: > site_dirs = [ > os.path.expanduser(s.strip()) for s in > self.site_dirs.split(',') This code will break on other people's installations, in that it will not be able to tell you're installing to a non-"site" directory unless you explicitly provide an --install-dir on the command line. It will thus create a non-working easy-install.pth, and will fail to warn the user that 'require()' is needed. From ianb at colorstudy.com Thu Sep 15 05:07:27 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 14 Sep 2005 22:07:27 -0500 Subject: [Distutils] ez_setup status? Message-ID: <4328E56F.6000503@colorstudy.com> What's the status of ez_setup from 0.6? The URL is still broken from me; I'm using an svn:external version, loaded from here: svn://svn.eby-sarna.com/svnroot/ez_setup And it's trying to load: http://cheeseshop.python.org/packages/source/s/setuptools/setuptools-0.6a1-py2.4.egg Instead of: http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a1-py2.4.egg I thought I saw Phillip mention it was fixed, but it doesn't seem to be, or maybe the svn version isn't fixed yet? From pje at telecommunity.com Thu Sep 15 05:30:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 23:30:54 -0400 Subject: [Distutils] ez_setup status? In-Reply-To: <4328E56F.6000503@colorstudy.com> Message-ID: <5.1.1.6.0.20050914232654.01f5f978@mail.telecommunity.com> At 10:07 PM 9/14/2005 -0500, Ian Bicking wrote: >What's the status of ez_setup from 0.6? The URL is still broken from >me; I'm using an svn:external version, loaded from here: > > svn://svn.eby-sarna.com/svnroot/ez_setup > >And it's trying to load: >http://cheeseshop.python.org/packages/source/s/setuptools/setuptools-0.6a1-py2.4.egg > >Instead of: >http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a1-py2.4.egg > > >I thought I saw Phillip mention it was fixed, but it doesn't seem to be, >or maybe the svn version isn't fixed yet? Argh. I missed the 2.4 part; I only fixed the 'cheeseshop' part. My test run apparently was an upgrade rather than a new install. It works fine for upgrades (where the URL is pulled from PyPI), but not for new installations. I've updated SVN and the master download file. Sorry about that. From rkern at ucsd.edu Thu Sep 15 06:39:47 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 14 Sep 2005 21:39:47 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> References: <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <4328C976.2000003@ucsd.edu> <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > This code will break on other people's installations, in that it will not > be able to tell you're installing to a non-"site" directory unless you > explicitly provide an --install-dir on the command line. It will thus > create a non-working easy-install.pth, and will fail to warn the user that > 'require()' is needed. I know. I said it was not satisfactory and that the real tricky part wasn't calculating the path but determining whether --prefix was explicitly provided or not. Ooh! But if install_cmd.prefix != sys.prefix, then --prefix must have been explicitly set by the user! Eureka! Now I just have to figure out a way to find out if --install-dir wasn't set.... -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pje at telecommunity.com Thu Sep 15 08:59:34 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 02:59:34 -0400 Subject: [Distutils] question about setuptools In-Reply-To: References: <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <4328C976.2000003@ucsd.edu> <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050915025737.03460e20@mail.telecommunity.com> At 09:39 PM 9/14/2005 -0700, Robert Kern wrote: >Phillip J. Eby wrote: > > > This code will break on other people's installations, in that it will not > > be able to tell you're installing to a non-"site" directory unless you > > explicitly provide an --install-dir on the command line. It will thus > > create a non-working easy-install.pth, and will fail to warn the user that > > 'require()' is needed. > >I know. I said it was not satisfactory and that the real tricky part >wasn't calculating the path but determining whether --prefix was >explicitly provided or not. > >Ooh! But if install_cmd.prefix != sys.prefix, then --prefix must have >been explicitly set by the user! Eureka! Now I just have to figure out a >way to find out if --install-dir wasn't set.... Maybe this is a dumb question, but if this is a Debian-specific issue, can't you just create a Debian package that combines the setuptools .egg and .pth files with a '/usr/lib/python2.4/distutils/distutils.cfg' file that contains the right options? Then, --prefix and --site-dirs would both be unnecessary at the CLI level. From rkern at ucsd.edu Thu Sep 15 09:25:13 2005 From: rkern at ucsd.edu (Robert Kern) Date: Thu, 15 Sep 2005 00:25:13 -0700 Subject: [Distutils] question about setuptools In-Reply-To: <5.1.1.6.0.20050915025737.03460e20@mail.telecommunity.com> References: <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5035593a05091312075e961d76@mail.gmail.com> <5.1.1.6.0.20050913152338.01eab800@mail.telecommunity.com> <5.1.1.6.0.20050913195714.01b79b40@mail.telecommunity.com> <5.1.1.6.0.20050913231120.01b5e870@mail.telecommunity.com> <5.1.1.6.0.20050914020322.03260c58@mail.telecommunity.com> <5.1.1.6.0.20050914130212.04096518@mail.telecommunity.com> <4328C976.2000003@ucsd.edu> <5.1.1.6.0.20050914213752.01b70aa8@mail.telecommunity.com> <5.1.1.6.0.20050915025737.03460e20@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Maybe this is a dumb question, but if this is a Debian-specific issue, > can't you just create a Debian package that combines the setuptools .egg > and .pth files with a '/usr/lib/python2.4/distutils/distutils.cfg' file > that contains the right options? Then, --prefix and --site-dirs would both > be unnecessary at the CLI level. I'm not about to request for a change in Debian's Python policy. But now that I've had a chance to review the new documentation (much improved! Thanks!), I think that the easiest route is to tell people to set their prefix and site-dirs in their ~/.pydistutils.cfg . And no, it's not just Debian. One of my complainants uses Fedora Core, and the other uses Debian, but maintains his self-built Python packages using Stow. I just use Debian as an example because I'm familiar with it. This is mostly an issue for people on various Unix-type setups with package managers. For example, Ian Bicking has the same issue with FreeBSD and comes to the same distutils.cfg solution: http://blog.ianbicking.org/alternate-python-install-dir.html -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From elvelind at gmail.com Thu Sep 15 10:38:22 2005 From: elvelind at gmail.com (Elvelind Grandin) Date: Thu, 15 Sep 2005 10:38:22 +0200 Subject: [Distutils] .pydistutils.cfg and record Message-ID: <5035593a0509150138691316e1@mail.gmail.com> hi. I have just discovered that setup.py can take a --record option. would it be possible add a line to .pydistutils.cfg so that each package records to a own file, called foo-1.2.0? -- cheers elvelind grandin From pje at telecommunity.com Thu Sep 15 17:19:05 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 11:19:05 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling Message-ID: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Every so often, there's a perennial debate on the distutils-sig about script filenames. Should they have .py extensions or not? What about .pyw apps on Windows? What about Mac OS? and so on. It occurred to me this morning that we now have a new tool for resolving this issue in setuptools' "entry points" feature. For various reasons, it's common practice to write scripts as Python modules with a "main" function of some kind. These modules are then run directly with '-m', or use "if __name__=='__main__'", or have a short script that imports the main function and runs it. So, if these "main" functions were simply declared as entry points in the project's setup script, then EasyInstall could automatically generate stub scripts for them, in a platform-appropriate fashion, with no need for '-m', "__name__=='__main__'", or fiddling with file extensions. For example, if PyUnit were distributed as an egg, with the following entry points: [distutils.console_apps] unittest = unittest:main Then EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments. Similarly, there could be a "distutils.gui_apps" entry point group, which could be handled differently according to the target platform. (For example, by creating desktop or menu shortcuts on Windows.) And, tools that create standalone applications or installers could use this information to create other kinds of wrappers around the same entry points. In order for this to work well, there are obviously some details to be worked out. Unix-like OSes are pretty obvious: #! line and no extension should work fine for all app types' executables. But adding desktop icons or menu items for GUI apps is very platform-specific (e.g. Gnome vs. KDE). So, whatever mechanism is used needs to be extensible and configurable. Ideally, it should be possible for the platform's Python installation to provide the necessary hooks to do this, because I personally don't want to have to write and maintain all that code. :) As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :) On Windows, I'd say that applications are pretty much always better as .exe's, whether console or GUI. The .py/.pyw form is dependent on a single global consistent version of Python, but it's possible and reasonable to have multiple Python versions installed. An .exe also has a lot more control over how Python is initialized, and that can be particularly important for applications. On the other hand, in the short run I can also see using .bat or .cmd files for console apps, and .pyw for GUI apps, just to have something that works and wait for the path management use cases for various .exe options to work themselves out. Anyway, my idea here is that when using setuptools, you would define entry points instead of creating scripts and listing them in setup(). Then, using either EasyInstall or "setup.py install" would automatically create platform-appropriate scripts. Some of the open questions: * What groups (if any) should exist besides "console_apps" and "gui_apps"? * How can we allow easy control of installation options on the target system? (e.g., I only want Windows Programs Menu items, you only want desktop shortcuts for KDE, etc., but we can't have six billion command line options, and all of this stuff can't go into setuptools anyway) * Should launchers hardcode their sys.path? This would prevent breakage caused by installing new and incompatible versions of their dependencies, but require an explicit upgrade action to update them, and make the installation process more complex. Thoughts, anyone? From dangoor at gmail.com Thu Sep 15 17:23:28 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Thu, 15 Sep 2005 11:23:28 -0400 Subject: [Distutils] bdist_egg ignores tag_svn_revision Message-ID: <3f085ecd050915082332da481c@mail.gmail.com> I've noticed that if I run a setup.py install, the svn revision shows up there. If I run setup.py bdist_egg, the filename generated does not have the revision. Is that on purpose? Kevin From pje at telecommunity.com Thu Sep 15 17:33:53 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 11:33:53 -0400 Subject: [Distutils] bdist_egg ignores tag_svn_revision In-Reply-To: <3f085ecd050915082332da481c@mail.gmail.com> Message-ID: <5.1.1.6.0.20050915113250.01f633d8@mail.telecommunity.com> At 11:23 AM 9/15/2005 -0400, Kevin Dangoor wrote: >I've noticed that if I run a setup.py install, the svn revision shows >up there. If I run setup.py bdist_egg, the filename generated does not >have the revision. > >Is that on purpose? No; bdist_egg should do exactly the same thing as the others, as they all use the 'egg_info' command to determine the tagging. I presume you have something like: [egg_info] tag_svn_revision = 1 in an appropriate config file? From ianb at colorstudy.com Thu Sep 15 17:41:39 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 15 Sep 2005 10:41:39 -0500 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Message-ID: <43299633.9040100@colorstudy.com> Phillip J. Eby wrote: > Every so often, there's a perennial debate on the distutils-sig about > script filenames. Should they have .py extensions or not? What about .pyw > apps on Windows? What about Mac OS? and so on. > > It occurred to me this morning that we now have a new tool for resolving > this issue in setuptools' "entry points" feature. For various reasons, > it's common practice to write scripts as Python modules with a "main" > function of some kind. These modules are then run directly with '-m', or > use "if __name__=='__main__'", or have a short script that imports the main > function and runs it. I've already noted my distaste for __name__ == '__main__', so I'd be happy to see it go (or, at least replaced with something better). An entry point seems easy. One issue that has come up for me is the way setuptools currently generates scripts causes problems for me. For instance, if I do tag_svn_revision, then the version of my package can change in place (whenever I do setup.py egg_info). The script has a version hardcoded in it, and so it fails. I'm not sure what the best way to handle this is, but the current behavior can look very ugly from a user's perspective when it fails. At least if I'm using tag_svn_revision, if I could put a requirement spec in setup.cfg to also control what the script requires, that would help. Also, if we support multiple versions of packages, each with a script tied to that version, then there needs to be an easier way to control the script names, e.g., --script-suffix=0.4 or something. The other issue is which Python version is used to load the script; it's not uncommon to want to switch versions. It would be nice if there was something -- an option, perhaps, or an environmental variable -- which could override the Python version. E.g. script --python-version=python2.4, or PYTHON_VERSION=python2.4 script (I don't think the later isn't reasonable to do in Windows, though) > * What groups (if any) should exist besides "console_apps" and "gui_apps"? I can't think of any, except of course for ad hoc subcommands like distutils.command. Well, rc scripts and system services perhaps, but that's not something I'd want to tackle right now; it's not even clear that a package should map to an rc script. > * Should launchers hardcode their sys.path? This would prevent breakage > caused by installing new and incompatible versions of their dependencies, > but require an explicit upgrade action to update them, and make the > installation process more complex. I'm wary of hardcoding paths. I don't think we should completely embrace encapsulation -- it might be better for long-term stability (maybe), but it makes it harder to fix a system that is broken (even when "brkoen" just means one missing requirement). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From dangoor at gmail.com Thu Sep 15 17:54:58 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Thu, 15 Sep 2005 11:54:58 -0400 Subject: [Distutils] bdist_egg ignores tag_svn_revision In-Reply-To: <5.1.1.6.0.20050915113250.01f633d8@mail.telecommunity.com> References: <3f085ecd050915082332da481c@mail.gmail.com> <5.1.1.6.0.20050915113250.01f633d8@mail.telecommunity.com> Message-ID: <3f085ecd050915085433471cf6@mail.gmail.com> On 9/15/05, Phillip J. Eby wrote: > No; bdist_egg should do exactly the same thing as the others, as they all > use the 'egg_info' command to determine the tagging. I presume you have > something like: > > [egg_info] > tag_svn_revision = 1 > > in an appropriate config file? Yep, but I just figured out what happened. I ran sdist on the original svn checkout to get a snapshot that I could hang on to. I then ran a bdist_egg *on that snapshot*, which would logically not appear to be an svn checkout. I was thrown off by the fact that tag_build still worked... It might be nice if an sdist that is tagged with a revision number creates eggs with that number, but that'd be a pretty low priority I would think. The only reason I even took an sdist snapshot is that I'm always paranoid about things disappearing off the net for one reason or another and like to have a local copy of parts that I'm actively using. Kevin From pje at telecommunity.com Thu Sep 15 18:07:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 12:07:54 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <43299633.9040100@colorstudy.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050915115911.01eca2f8@mail.telecommunity.com> At 10:41 AM 9/15/2005 -0500, Ian Bicking wrote: >One issue that has come up for me is the way setuptools currently >generates scripts causes problems for me. For instance, if I do >tag_svn_revision, then the version of my package can change in place >(whenever I do setup.py egg_info). The script has a version hardcoded in >it, and so it fails. Run "develop" instead of "egg_info", and your active scripts will be upgraded. (The "develop" command also runs "egg_info" as a subcommand.) > Also, if we support multiple versions of packages, each with a script > tied to that version, then there needs to be an easier way to control the > script names, e.g., --script-suffix=0.4 or something. That would indeed be handy. >The other issue is which Python version is used to load the script; it's >not uncommon to want to switch versions. It would be nice if there was >something -- an option, perhaps, or an environmental variable -- which >could override the Python version. E.g. script >--python-version=python2.4, or PYTHON_VERSION=python2.4 script (I don't >think the later isn't reasonable to do in Windows, though) I don't think this can be made to work in any meaningful fashion, since eggs are specific to a particular Python version (because of bytecode format differences), and may contain different modules or entry points or dependencies for different Python versions. For example, if somebody packages a new module for use with older Pythons, users of that package will likely have their setup() include a dependency for the separately-packaged module if running on the older Python. Another example: setuptools doesn't publish an entry point for the 'build_py' command when running on Python 2.4, but it does on 2.3 because the 2.3 build_py command doesn't support installing package data. >>* Should launchers hardcode their sys.path? This would prevent breakage >>caused by installing new and incompatible versions of their dependencies, >>but require an explicit upgrade action to update them, and make the >>installation process more complex. > >I'm wary of hardcoding paths. I don't think we should completely embrace >encapsulation -- it might be better for long-term stability (maybe), but >it makes it harder to fix a system that is broken (even when "brkoen" just >means one missing requirement). Well, with a sufficiently complex launcher, you could have it try to fix the problem (e.g. by running EasyInstall to find the missing bits). But I guess we can just evolve that way over time. :) From trentm at ActiveState.com Thu Sep 15 18:58:40 2005 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 15 Sep 2005 09:58:40 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Message-ID: <20050915165838.GA3219@ActiveState.com> I haven't yet played with setuptools/eggs/etc. so my comments might be ignorant. [Phillip J. Eby wrote] > Then EasyInstall could create a 'unittest' script with a #! line on > Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on > Windows. In each case, the generated program would simply load and run the > entry point with no arguments. Effbot has some good things to say about/use for this: http://effbot.org/zone/exemaker.htm > On Windows, I'd say that applications are pretty much always better as > .exe's, whether console or GUI. The .py/.pyw form is dependent on a single > global consistent version of Python, but it's possible and reasonable to > have multiple Python versions installed. An .exe also has a lot more > control over how Python is initialized, and that can be particularly > important for applications. On the other hand, in the short run I can also > see using .bat or .cmd files for console apps, and .pyw for GUI apps, just > to have something that works and wait for the path management use cases for > various .exe options to work themselves out. There is a bug in the current cmd.exe (or maybe it is only up to Win2k? can't remember) where shell redirection doesn't work if your script is launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal handling for kill a script is quite annoying if launched via a .bat file: you get a secondary question from the shell Terminate batch file (Y/N)? or something like that. A .exe launcher/stub is definitely preferred. Trent -- Trent Mick TrentM at ActiveState.com From trentm at ActiveState.com Thu Sep 15 19:00:02 2005 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 15 Sep 2005 10:00:02 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <43299633.9040100@colorstudy.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <43299633.9040100@colorstudy.com> Message-ID: <20050915170002.GB3219@ActiveState.com> [Ian Bicking wrote] > I've already noted my distaste for __name__ == '__main__', so I'd be > happy to see it go (or, at least replaced with something better). That facility is about one of the most useful in Python. ...but I haven't read your previous comments on that. :) Trent -- Trent Mick TrentM at ActiveState.com From dangoor at gmail.com Thu Sep 15 19:52:51 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Thu, 15 Sep 2005 13:52:51 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Message-ID: <3f085ecd050915105269b701ea@mail.gmail.com> Overall, I like the idea of using an entry point for scripts and getting some of the benefits you're talking about... On 9/15/05, Phillip J. Eby wrote: > As for Mac OS, I have almost no experience with it, so I'm not sure what > GUI applications there need. Does everything need py2app? If you have a > wx-based app, would you just make a #! script? Bob Ippolito previously > mentioned that you don't "install" applications there, that people just > drag applications wherever they want them rather than using shortcuts, so > at least that part isn't a problem. :) GUI apps on the Mac need py2app. Kevin From theller at python.net Thu Sep 15 19:54:59 2005 From: theller at python.net (Thomas Heller) Date: Thu, 15 Sep 2005 19:54:59 +0200 Subject: [Distutils] Improving distutils' script and GUI app handling References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> Message-ID: Trent Mick writes: > I haven't yet played with setuptools/eggs/etc. so my comments might be > ignorant. > > [Phillip J. Eby wrote] >> Then EasyInstall could create a 'unittest' script with a #! line on >> Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on >> Windows. In each case, the generated program would simply load and run the >> entry point with no arguments. > > Effbot has some good things to say about/use for this: > http://effbot.org/zone/exemaker.htm > > >> On Windows, I'd say that applications are pretty much always better as >> .exe's, whether console or GUI. The .py/.pyw form is dependent on a single >> global consistent version of Python, but it's possible and reasonable to >> have multiple Python versions installed. An .exe also has a lot more >> control over how Python is initialized, and that can be particularly >> important for applications. On the other hand, in the short run I can also >> see using .bat or .cmd files for console apps, and .pyw for GUI apps, just >> to have something that works and wait for the path management use cases for >> various .exe options to work themselves out. Which path management? Do you mean the default distutils build directory (which we discussed some time ago)? > There is a bug in the current cmd.exe (or maybe it is only up to Win2k? > can't remember) where shell redirection doesn't work if your script is > launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal > handling for kill a script is quite annoying if launched via a .bat > file: you get a secondary question from the shell > > Terminate batch file (Y/N)? > > or something like that. A .exe launcher/stub is definitely preferred. Isn't the shell redirection problem also present if the .py script is run via the shell association? The Ctrl+C behaviour annoys me also because I usually run my scripts via batch files (py23.bat, py24.bat, py_d.bat and so on). Maybe this problem can be avoided by copying the python.exe's somewhere to my path, with names p23.exe, p24.exe and so on. They only contain a pointer to the actual PythonXY.dll name, afaik. So I don't have to update p24.exe when I install 2.4.2 over my 2.4.1. Somewhat similar to the exemaker approach, but using the name of the exe to specify the Python version instead of a '-i ...' command line option, or the #! line in the script. The only missing piece so far, for me at least, is that it's not possible to use dotted module names with the -m option: p24 -m ctypes.wrap.h2xml windows.h Thomas From rkern at ucsd.edu Thu Sep 15 20:10:18 2005 From: rkern at ucsd.edu (Robert Kern) Date: Thu, 15 Sep 2005 11:10:18 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <3f085ecd050915105269b701ea@mail.gmail.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <3f085ecd050915105269b701ea@mail.gmail.com> Message-ID: Kevin Dangoor wrote: > On 9/15/05, Phillip J. Eby wrote: > >>As for Mac OS, I have almost no experience with it, so I'm not sure what >>GUI applications there need. Does everything need py2app? If you have a >>wx-based app, would you just make a #! script? Bob Ippolito previously >>mentioned that you don't "install" applications there, that people just >>drag applications wherever they want them rather than using shortcuts, so >>at least that part isn't a problem. :) > > GUI apps on the Mac need py2app. It depends. wx-based programs don't have a technical need to be in a .app bundle. They will run fine from the command line. Now, the *user* might want it packaged in a .app bundle, and we as developers might need to respond to that desire. I think that PyObjC apps might actually need to be in a .app bundle to work reliably, though. I have a feeling that there isn't a pressing need to be able to install .app bundles from easy_install. If the package maintainer is going to go to the trouble of writing a setup.py that can be used with py2app, he'll almost certainly actually build the .app binary and distribute it. Since they're standalone, they don't really interact with the other installed packages. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pje at telecommunity.com Thu Sep 15 20:49:55 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 14:49:55 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: References: <3f085ecd050915105269b701ea@mail.gmail.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <3f085ecd050915105269b701ea@mail.gmail.com> Message-ID: <5.1.1.6.0.20050915144733.01b6f768@mail.telecommunity.com> At 11:10 AM 9/15/2005 -0700, Robert Kern wrote: >I have a feeling that there isn't a pressing need to be able to install >.app bundles from easy_install. If the package maintainer is going to go >to the trouble of writing a setup.py that can be used with py2app, he'll >almost certainly actually build the .app binary and distribute it. Since >they're standalone, they don't really interact with the other installed >packages. If you can run a wx or tkinter (or other cross-platform GUI) app on Mac OS without needing anything special, but users would prefer to have the .app, it would probably be a good idea to have it, even if the original developer hasn't targeted that platform specifically. For example, packages like idle or the pydoc tkinter GUI would seem to merit something like this. From rkern at ucsd.edu Thu Sep 15 21:15:50 2005 From: rkern at ucsd.edu (Robert Kern) Date: Thu, 15 Sep 2005 12:15:50 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050915144733.01b6f768@mail.telecommunity.com> References: <3f085ecd050915105269b701ea@mail.gmail.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <3f085ecd050915105269b701ea@mail.gmail.com> <5.1.1.6.0.20050915144733.01b6f768@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 11:10 AM 9/15/2005 -0700, Robert Kern wrote: > >>I have a feeling that there isn't a pressing need to be able to install >>.app bundles from easy_install. If the package maintainer is going to go >>to the trouble of writing a setup.py that can be used with py2app, he'll >>almost certainly actually build the .app binary and distribute it. Since >>they're standalone, they don't really interact with the other installed >>packages. > > If you can run a wx or tkinter (or other cross-platform GUI) app on Mac OS > without needing anything special, but users would prefer to have the .app, > it would probably be a good idea to have it, even if the original developer > hasn't targeted that platform specifically. For example, packages like > idle or the pydoc tkinter GUI would seem to merit something like this. Oh, certainly it would be nice. But AFAICT, you really do have to write a setup.py specifically for py2app. The information you need isn't something you can simply divine from a generic setup.py . -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From ianb at colorstudy.com Fri Sep 16 00:55:26 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 15 Sep 2005 17:55:26 -0500 Subject: [Distutils] recursive package_data (setuptools?) Message-ID: <4329FBDE.9020308@colorstudy.com> I have a nested directory with templates, like templates/foo.pt templates/admin/bar.pt I thought I could include them in a package with: setup(... package_data={ 'mypackage': ['templates/*']}) But it doesn't work -- while the package is fine, when installed setuptools (maybe just inheriting distutils here) tries to copy templates/admin over, and it can't copy a directory. And, I assume, it won't even try templates/admin/bar.pt. So... is there a way to do this, without enumerating (a) every subdirectory I'm interested in, and (b) every extension I'm intersted in? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From dangoor at gmail.com Fri Sep 16 13:13:35 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Fri, 16 Sep 2005 07:13:35 -0400 Subject: [Distutils] recursive package_data (setuptools?) In-Reply-To: <4329FBDE.9020308@colorstudy.com> References: <4329FBDE.9020308@colorstudy.com> Message-ID: <3f085ecd05091604132c01de8e@mail.gmail.com> I've had success with something like this: package_data = { '' : ["*.pt"]} Of course, if you have page templates in directories other than "templates", you'll end up snagging those, too. Kevin On 9/15/05, Ian Bicking wrote: > I have a nested directory with templates, like > > templates/foo.pt > templates/admin/bar.pt > > I thought I could include them in a package with: > > setup(... > package_data={ > 'mypackage': ['templates/*']}) > > But it doesn't work -- while the package is fine, when installed > setuptools (maybe just inheriting distutils here) tries to copy > templates/admin over, and it can't copy a directory. And, I assume, it > won't even try templates/admin/bar.pt. > > So... is there a way to do this, without enumerating (a) every > subdirectory I'm interested in, and (b) every extension I'm intersted in? > > -- > Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Fri Sep 16 15:10:40 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Sep 2005 09:10:40 -0400 Subject: [Distutils] recursive package_data (setuptools?) In-Reply-To: <4329FBDE.9020308@colorstudy.com> Message-ID: <5.1.1.6.0.20050916090342.01b807c8@mail.telecommunity.com> At 05:55 PM 9/15/2005 -0500, Ian Bicking wrote: >I have a nested directory with templates, like > >templates/foo.pt >templates/admin/bar.pt > >I thought I could include them in a package with: > >setup(... > package_data={ > 'mypackage': ['templates/*']}) > >But it doesn't work -- while the package is fine, when installed >setuptools (maybe just inheriting distutils here) tries to copy >templates/admin over, and it can't copy a directory. And, I assume, it >won't even try templates/admin/bar.pt. > >So... is there a way to do this, without enumerating (a) every >subdirectory I'm interested in, and (b) every extension I'm intersted in? Write a function to do it for you? :) Seriously, it might not be a bad idea to add a 'find_package_data()' function along the lines of the sdist command's "include everything that's under source control" algorithm. Then you could just do: package_data = find_package_data() and be done with it. We could perhaps have it use the actual sdist command's file finding code, so that MANIFEST.in is respected if you're not using source control. It would need to filter out .py files and files that are not inside packages, and create a package_data mapping based on the nearest containing package. From trentm at ActiveState.com Fri Sep 16 22:10:43 2005 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 16 Sep 2005 13:10:43 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> Message-ID: <20050916201043.GA7448@ActiveState.com> [Thomas Heller wrote] > > There is a bug in the current cmd.exe (or maybe it is only up to Win2k? > > can't remember) where shell redirection doesn't work if your script is > > launched indirectly via a .bat of .cmd file. As well, Ctrl+C signal > > handling for kill a script is quite annoying if launched via a .bat > > file: you get a secondary question from the shell > > > > Terminate batch file (Y/N)? > > > > or something like that. A .exe launcher/stub is definitely preferred. > > Isn't the shell redirection problem also present if the .py script is > run via the shell association? Oh shoot, I think I was wrong and you are right: the shell redirection problem is for the case of using the PATHEXT stuff. I.e. I think shell redirection works fine with a .bat or .cmd stub. Sorry about causing confusion. > The Ctrl+C behaviour annoys me also because I usually run my scripts > via batch files (py23.bat, py24.bat, py_d.bat and so on). > Maybe this problem can be avoided by copying the python.exe's somewhere > to my path, with names p23.exe, p24.exe and so on. They only contain a > pointer to the actual PythonXY.dll name, afaik. So I don't have to > update p24.exe when I install 2.4.2 over my 2.4.1. I tend to copy python.exe to pythonXY.exe (as appropriate) after I install a new Python. I'm considering doing this by default in the ActivePython installers I make. (Similar to the pythonX.Y executable copies that get made on Linux/Un*x.) What do you think? > > Somewhat similar to the exemaker approach, but using the name of the exe > to specify the Python version instead of a '-i ...' command line option, > or the #! line in the script. Agreed. > The only missing piece so far, for me at least, is that it's not > possible to use dotted module names with the -m option: > > p24 -m ctypes.wrap.h2xml windows.h That's too bad. I didn't know "-m" didn't support that... but then I haven't really used it that much yet. Trent -- Trent Mick TrentM at ActiveState.com From p.f.moore at gmail.com Fri Sep 16 23:00:19 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 16 Sep 2005 22:00:19 +0100 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <20050916201043.GA7448@ActiveState.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> Message-ID: <79990c6b050916140067c88bd2@mail.gmail.com> On 9/16/05, Trent Mick wrote: > > The only missing piece so far, for me at least, is that it's not > > possible to use dotted module names with the -m option: > > > > p24 -m ctypes.wrap.h2xml windows.h > > That's too bad. I didn't know "-m" didn't support that... but then I > haven't really used it that much yet. There's a patch on SF to implement that (1043356 ) and a corresponding PEP (338), but I don't know what's required to get it added. Another annoyance with -m is that it doesn't support modules in zipfiles (e.g., zipped eggs). I submitted a bug for that one (1250389) but I don't have a C build environment that I can use to try out any form of fix - so I'm relying on someone else's goodwill to look into it... Paul. From pje at telecommunity.com Sat Sep 17 03:24:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Sep 2005 21:24:54 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050916211632.01b94520@mail.telecommunity.com> At 11:19 AM 9/15/2005 -0400, Phillip J. Eby wrote: >Every so often, there's a perennial debate on the distutils-sig about >script filenames. Should they have .py extensions or not? What about .pyw >apps on Windows? What about Mac OS? and so on. > >... > >So, if these "main" functions were simply declared as entry points in the >project's setup script, then EasyInstall could automatically generate stub >scripts for them, in a platform-appropriate fashion, with no need for '-m', >"__name__=='__main__'", or fiddling with file extensions. For example, >if PyUnit were distributed as an egg, with the following entry points: > > [distutils.console_apps] > unittest = unittest:main > >Then EasyInstall could create a 'unittest' script with a #! line on >Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on >Windows. In each case, the generated program would simply load and run the >entry point with no arguments. FYI, I've now implemented console script generation in the CVS version of setuptools, except that I decided to call the entry point group "console_scripts". There is a custom .exe launcher for Windows, too. The whole thing was surprisingly easy to implement, and the docs are updated in CVS as well. I'll probably implement the find_package_data() thing this weekend as well, and put it out as an 0.6a2 release. Note that generated scripts can't be run with -m, but then again neither can a lot of things. ;) Using -m scripts means your eggs can't be installed compressed anyway, so these "new-style" scripts are superior in almost every way. You don't need an if __name__=='__main__', you don't need separate scripts, just write your "main" function in a module somewhere and list the entry point in your setup(). Anyway, for backward-compatibility I'm still allowing "python -m easy_install" in the next release or two, but a warning is printed to the console. Later, this feature will be removed so that it's possible to install setuptools in compressed form. From dangoor at gmail.com Sat Sep 17 18:33:20 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Sat, 17 Sep 2005 12:33:20 -0400 Subject: [Distutils] Cheese Shop login requires login Message-ID: <3f085ecd05091709332b0468ee@mail.gmail.com> I'm not sure where Cheese Shop related questions would go. Apparently, the Cheese Shop requires you to be logged in in order to see the login page: http://cheeseshop.python.org/pypi?%3Aaction=login I assume if you're already logged in, you're fine. Kevin -- Kevin Dangoor Author of the Zesty News RSS newsreader email: kid at blazingthings.com company: http://www.BlazingThings.com blog: http://www.BlueSkyOnMars.com From pje at telecommunity.com Sat Sep 17 19:17:59 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 17 Sep 2005 13:17:59 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <79990c6b050916140067c88bd2@mail.gmail.com> References: <20050916201043.GA7448@ActiveState.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> Message-ID: <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> At 10:00 PM 9/16/2005 +0100, Paul Moore wrote: >On 9/16/05, Trent Mick wrote: > > > The only missing piece so far, for me at least, is that it's not > > > possible to use dotted module names with the -m option: > > > > > > p24 -m ctypes.wrap.h2xml windows.h > > > > That's too bad. I didn't know "-m" didn't support that... but then I > > haven't really used it that much yet. > >There's a patch on SF to implement that (1043356 ) and a corresponding >PEP (338), but I don't know what's required to get it added. > >Another annoyance with -m is that it doesn't support modules in >zipfiles (e.g., zipped eggs). Really, the only reasonable way to solve both problems is something like 'python -m run foo.bar ....'. That is, have a bootstrap module to do the rest. (Or alternately, have -m fallback to using the bootstrap module if it's unsuccessful.) However, after reflection, I think now that -m probably only really makes sense for stdlib modules, since projects using setuptools can now get all the benefits of -m without any of the drawbacks, without even writing any __name__=='__main__' code. For example defining an entry point thus: [console_scripts] h2xml = ctypes.wrap.h2xml:main automatically creates 'h2xml.exe' on Windows and a #!-prefixed 'h2xml' script on any other platform. (Using the current CVS version of setuptools, that is.) From pje at telecommunity.com Sun Sep 18 06:47:20 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 18 Sep 2005 00:47:20 -0400 Subject: [Distutils] setuptools 0.6a2 released Message-ID: <5.1.1.6.0.20050918000751.0206fea8@mail.telecommunity.com> It generates console scripts and .exe wrappers. I decided not to implement find_package_data(), because it turned out to be a lot more complex than I thought it would be. In fact, it was complex enough that I was wondering if I might need to rewrite the entire distutils in order to make it work well. :( Anyway, 0.6a2 is out, so enjoy. I'd be interested in hearing feedback on how the GUI wrappers should be done. I've already figured out how to compile a launcher variant that doesn't create a console window on Windows, so that part's easy. I guess we can always add fancy features for Mac et al later. One thing I'm not too sure about for GUIs on non-Windows platforms is the whole detach or not-to-detach thing. On Windows, I was planning to make GUIs run detached from the current console, but I'm not sure if that's the "normal" thing to do on other platforms. From hawk78_it at yahoo.it Sun Sep 18 12:38:47 2005 From: hawk78_it at yahoo.it (Vincenzo Di Massa) Date: Sun, 18 Sep 2005 12:38:47 +0200 Subject: [Distutils] Files under /etc and /usr/share Message-ID: <200509181238.47216.hawk78_it@yahoo.it> Hi, It is some weeks I put easy-deb on http://easy-deb.sf.net. I'm now considering improvements to make a new release. I have two things in mind: 1) make easy-deb a setup command (now it is an external tool) 2) provide a way to put files into /etc About point 2): Is it possible to install things into /etc using setuptools/easy_install? I think that this feature will be very appreciated. I know setup.py philosopy is to put everything inside eggs, but config files are a really special case! It would be nice beeing able to tell in setup.py what the config files/dirs are. The generated egg should install the config files on user request (using a gen_config script) or just when the neded are not config files on the system. Another idea is for heavy data. Consider for example moin_moin. Using easy-deb i managed to aotomatically pacakge it into an egg (which is contained into a deb-package). The problem here is that it installs the template files inside the egg. Since template files must be copied "somewere" it is bat to put them into the egg where they are hidden/zipped. Moreover it would be nice to make the eggs inside /usr/lib/python?.?/site-packages more lightweight removing package data from them. There are 2 kind of package data: data that is used *by* the module code and data that is used *with* the module. moinmoin is just an example of module whose data is not used by the code but with the code. I have no idea how this could be done, but it would be good (IMHO). Another nice thing to have in setuptools is a --nodeps option. Packagers really need to install an egg (into a package of course) even if its dependencies are not on the system. Skipped dependencies could be reported inside a skipped_deps.txt file or on stdout (so that packagers can add the dependency info to the generated packages). Ciao Vincenzo ___________________________________ Yahoo! Messenger: chiamate gratuite in tutto il mondo http://it.messenger.yahoo.com From ronaldoussoren at mac.com Sun Sep 18 17:36:24 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 18 Sep 2005 17:36:24 +0200 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <3f085ecd050915105269b701ea@mail.gmail.com> Message-ID: <30BD4A28-E1F9-4847-8828-58EC422A4B78@mac.com> On 15-sep-2005, at 20:10, Robert Kern wrote: > Kevin Dangoor wrote: > >> On 9/15/05, Phillip J. Eby wrote: >> >> >>> As for Mac OS, I have almost no experience with it, so I'm not >>> sure what >>> GUI applications there need. Does everything need py2app? If >>> you have a >>> wx-based app, would you just make a #! script? Bob Ippolito >>> previously >>> mentioned that you don't "install" applications there, that >>> people just >>> drag applications wherever they want them rather than using >>> shortcuts, so >>> at least that part isn't a problem. :) >>> >> >> GUI apps on the Mac need py2app. >> > > It depends. wx-based programs don't have a technical need to be in a > .app bundle. They will run fine from the command line. Now, the *user* > might want it packaged in a .app bundle, and we as developers might > need > to respond to that desire. IIRC you need to use bundles if you want full control over the menu contents. But I haven't used wx on the mac for quite a while. > > I think that PyObjC apps might actually need to be in a .app bundle to > work reliably, though. PyObjC apps do need to be in a .app bundle, that's needed to find resources, such as NIB files, using the Cocoa API's. > > I have a feeling that there isn't a pressing need to be able to > install > .app bundles from easy_install. If the package maintainer is going > to go > to the trouble of writing a setup.py that can be used with py2app, > he'll > almost certainly actually build the .app binary and distribute it. > Since > they're standalone, they don't really interact with the other > installed > packages. Think big :-). It would be nice if we'd end up with a generic method for describing how to build standalone apps that py2app and py2exe could hook into. You currently have to write slightly different code on windows and osx to get a standalone app. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050918/3e49f671/smime.bin From pje at telecommunity.com Sun Sep 18 17:49:39 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 18 Sep 2005 11:49:39 -0400 Subject: [Distutils] Files under /etc and /usr/share In-Reply-To: <200509181238.47216.hawk78_it@yahoo.it> Message-ID: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote: >Is it possible to install things into /etc using setuptools/easy_install? I >think that this feature will be very appreciated. >I know setup.py philosopy is to put everything inside eggs, but config files >are a really special case! >It would be nice beeing able to tell in setup.py what the config files/dirs >are. The generated egg should install the config files on user request (using >a gen_config script) or just when the neded are not config files on the >system. I think that this is very application-specific as to how it should be handled. It's very easy to create a package that contains configuration files, with a function in __init__.py to install them, perhaps with some options or an interactive setup facility. Listing the function in setup.py then allows a script wrapper to be created for setting up the data. Using pkg_resources.resource_filename() with a directory resource will unpack an entire directory tree to a temporary location, or give you the location of files that are already unpacked, so you can then use shutil or other normal ways of copying files. >Another idea is for heavy data. Consider for example moin_moin. >Using easy-deb i managed to aotomatically pacakge it into an egg (which is >contained into a deb-package). The problem here is that it installs the >template files inside the egg. Since template files must be copied "somewere" >it is bat to put them into the egg where they are hidden/zipped. Moreover it >would be nice to make the eggs inside /usr/lib/python?.?/site-packages more >lightweight removing package data from them. I don't really agree. Projects with this kind of data are best off if they provide a custom installation tool. Well, I should say that their *users* are best off if they provide an installation tool, rather than dumping the files in one place. Further, it is easier to support global installation of a package for multiple users, or for a single user with multiple installations of the tool. (E.g. multiple wikis.) If the data installation is done by easy_install, it can only be done once, and without any real options as to how it's done. If it's done by a function included with the package, then it can be done as many times as desired, and with whatever options the project wants to provide the user with. >There are 2 kind of package data: data that is used *by* the module code and >data that is used *with* the module. moinmoin is just an example of module >whose data is not used by the code but with the code. Well, in my way of thinking, such templates *are* used by the code, it's just that they should be used by the project's "instance install/config" script(s). >Another nice thing to have in setuptools is a --nodeps option. Packagers >really need to install an egg (into a package of course) even if its >dependencies are not on the system. Skipped dependencies could be reported >inside a skipped_deps.txt file or on stdout (so that packagers can add the >dependency info to the generated packages). We could have a --record-deps option, but you can also just pull the dependencies from the Distribution.requires() method. I have had it on my list to have a --no-deps option, though, I just haven't gotten around to it yet. From pje at telecommunity.com Sun Sep 18 21:08:14 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 18 Sep 2005 15:08:14 -0400 Subject: [Distutils] Broken setuptools again? In-Reply-To: <71AA85B9-422C-420E-94B6-32BE88CEC8A6@groovie.org> Message-ID: <5.1.1.6.0.20050918143737.01eb0c78@mail.telecommunity.com> At 11:05 AM 9/18/2005 -0700, Ben Bangert wrote: >For example, in the 0.6a2 upgrade, it seems the easy_install script >wasn't updated or something. So now easy_install.pth was updated to >refer to the new setuptools version, but easy_install.py itself was >not. This results in this message: > >infopkg_resources.VersionConflict: (setuptools 0.6a2 (/Library/ >Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/ >setuptools-0.6a2-py2.4.egg), Requirement.parse('setuptools==0.6a1')) > >The first line of traceback is more revealing: >Traceback (most recent call last): File "/usr/local/bin/ >easy_install.py", line 4, in ? pkg_resources.run_script >('setuptools==0.6a1', 'easy_install.py') > >Sure enough, the script wasn't updated during the setuptools 0.6a1 -> >0.6a2 upgrade. As such, easy_install is currently broken on my system >and completely unusable. That's because 'easy_install.py' is no longer the script that runs EasyInstall. It's 'easy_install' now, with no extension, as described in the current documentation. This is probably not as clear as it should have been; this was based on a very new feature, and honestly I forgot about the upgrade scenario, and the fact that existing EasyInstall users would be accustomed to running easy_install.py instead. (Especially since on Windows, an easy_install.py is still generated alongside the new easy_install.exe file.) I'll release 0.6a3 with an easy_install.py script that tells you to run 'easy_install' instead. (I did think of the "python -m easy_install" case, and if you run that it'll tell you to run it the new way.) >As I'm working on several Python programs, I've been pondering using >setuptools for their distro as I want the functionality it has but >I'm not ready to be sending users to something that breaks so often >(at least in my moderate usage attempts). When do you expect a >"Stable" release? Any ideas how I can fix my problem so I can use >setuptools again? Current versions of setuptools are maintained on the cheeseshop servers more or less indefinitely, so you should be able to just freeze what version of ez_setup.py you use by including it directly in your package. Unfortunately, the cheeseshop changed its download locations recently, and broke backward compatibility with existing copies of ez_setup. Hopefully this won't happen again, although I have no control over it. As for a stable release, all I can say is that setuptools' API stability is now pretty good, and I don't expect any significant changes. Hopefully, everything from here on out should be new features, and hopefully they will be add-ons rather than things needing to be grafted into the basic structure. Setuptools tends to use its own new features first, however, and the problem you experienced with 0.6a2 is an example. I don't take this to mean that 0.6a2 is broken, however. If you were installing 0.6a2 fresh, you wouldn't have had the problem that you did. It's less of a problem with stability per se, than with having robust upgrades. In the long run, we should have a package management tool that can deal with things like removing outdated scripts, but we don't have one now. If by "stable" you mean that setuptools doesn't have any bugs, then at this point the most "stable" releases are 0.5a13, 0.6a1, and 0.6a2. They do what they do, although each version has features that the earlier ones don't. 0.5a13 in particular has a wealth of limitations compared to the 0.6 releases, and the main reason there were so many releases in the 0.5 line is that it *was* unstable. There was quite a bit of reorganization of the installation process happening there. Because of the many problems with 0.5, the 0.6a1 release was developed in CVS without making interim releases to PyPI or updating the svn:externals ez_setup, precisely to avoid having these kinds of problems. So, I guess what I'm saying is that you have a couple of options regarding how you want to embed setuptools. You can use svn:externals, in which case you'll be getting only releases judged to be of moderately stable quality. There were several points between 0.5a13 and 0.6a1 (such as the two 0.6a1 candidate releases) that I did *not* update the svn:externals ez_setup, so that folks like yourself wouldn't end up accidentally participating in the testing. :) However, if you want to personally vet the versions you use with your package, you should be able to just keep a personal copy of ez_setup.py, and even modify it for your own use. Heck, you don't even have to modify it or keep such a version, because the 'use_setuptools()' function takes a version and a download base URL. You can save a copy of the setuptools eggs you want to use, and simply set the version and download base in your setup script. So, you don't have to depend on anybody else to keep the version you want available. So, to put a different spin on your question... you can determine what *you* consider to be a stable release of setuptools, and make your system use that. :) (It's true that if someone already has a newer version of setuptools, that's what will be used instead. However, that's *their* responsibility, and they can and should bitch to me if something breaks.) From hawk78_it at yahoo.it Mon Sep 19 02:51:55 2005 From: hawk78_it at yahoo.it (Vincenzo Di Massa) Date: Mon, 19 Sep 2005 02:51:55 +0200 Subject: [Distutils] Files under /etc and /usr/share In-Reply-To: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> References: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> Message-ID: <200509190251.55737.hawk78_it@yahoo.it> Sorry for the previous post... The yhaoo antispam system added bul_k to the subject... Alle 17:49, domenica 18 settembre 2005, hai scritto: > At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote: > >Is it possible to install things into /etc using setuptools/easy_install? > > I think that this feature will be very appreciated. > >I know setup.py philosopy is to put everything inside eggs, but config > > files are a really special case! > >It would be nice beeing able to tell in setup.py what the config > > files/dirs are. The generated egg should install the config files on user > > request (using a gen_config script) or just when the neded are not config > > files on the system. > > I think that this is very application-specific as to how it should be > handled. It's very easy to create a package that contains configuration > files, with a function in __init__.py to install them, perhaps with some > options or an interactive setup facility. Listing the function in setup.py > then allows a script wrapper to be created for setting up the data. > > Using pkg_resources.resource_filename() with a directory resource will > unpack an entire directory tree to a temporary location, or give you the > location of files that are already unpacked, so you can then use shutil or > other normal ways of copying files. What you say seems to be the right thing (tm) :-). What I am proposing is nothing more that a convention to do config files installation. Maybe it is just something that can be explained into the documentation. It would be nice to ask setup.py writers to adopt the "best practice" that you descibed before. Maybe we can define what the name of the function is and what parameters it requires: this will make easy and obvious what is the right way of handling config options. Once the function name is standardized... why not provide a default implementation that automatically installs the files you specify somewhere? It is tedious to have to write code to just handle a little set of config files. It is a very common use case, so it would be good to provide some authomatism to: 1) make the distribution of many modules easier 2) provide a standard way to do one thing > >Another idea is for heavy data. Consider for example moin_moin. > >Using easy-deb i managed to aotomatically pacakge it into an egg (which is > >contained into a deb-package). The problem here is that it installs the > >template files inside the egg. Since template files must be copied > > "somewere" it is bat to put them into the egg where they are > > hidden/zipped. Moreover it would be nice to make the eggs inside > > /usr/lib/python?.?/site-packages more lightweight removing package data > > from them. > > I don't really agree. Projects with this kind of data are best off if they > provide a custom installation tool. Well, I should say that their *users* > are best off if they provide an installation tool, rather than dumping the > files in one place. Further, it is easier to support global installation > of a package for multiple users, or for a single user with multiple > installations of the tool. (E.g. multiple wikis.) If the data > installation is done by easy_install, it can only be done once, and without > any real options as to how it's done. If it's done by a function included > with the package, then it can be done as many times as desired, and with > whatever options the project wants to provide the user with. I did not explain myself very well, sorry. What you say is absolutely right! What I meant is: Suppose you have a module that has a an installation tool that is able to install data files/templates to multiple dirs at different times. All the copies are taken from somewhere. Now the source copy is inside the egg. My proposal was to move the source copy out of the egg. The reasons are: 1) you should be able to modify the default template 2) the template can be big an big templates belong to something different from /usr/lib/python?.?/... 3) provide a standard way to do one thing Maybe setuptools could optionally put a couple scripts inside generated eggs that can: 1) put default config files in place 2) move the templates somewhere else (like /usr/share) This way config files and data files could be handled automatically like setuptools does for dependencies: before executing the module could call pkg_resurces.config_files("modulename") and pkg_resources.data_files("modulename") to make sure that the config files exist and that data files are where they shuold be. > >There are 2 kind of package data: data that is used *by* the module code > > and data that is used *with* the module. moinmoin is just an example of > > module whose data is not used by the code but with the code. > > Well, in my way of thinking, such templates *are* used by the code, it's > just that they should be used by the project's "instance install/config" > script(s). I understand your point of view, any template can be seen as used by the module's code, but... There is a little difference: there is data that the user will hand edit, and data used an modified only by the module. There is data that must installed only once, and data (like moinmoin templates) that can be installed multiple times. I wanted to distinguish data that can be somehow "hidden" from data that must be "visible". > >Another nice thing to have in setuptools is a --nodeps option. Packagers > >really need to install an egg (into a package of course) even if its > >dependencies are not on the system. Skipped dependencies could be reported > >inside a skipped_deps.txt file or on stdout (so that packagers can add the > >dependency info to the generated packages). > > We could have a --record-deps option, but you can also just pull the > dependencies from the Distribution.requires() method. I have had it on my > list to have a --no-deps option, though, I just haven't gotten around to it > yet. You are doing a really good job with setuptools! I really like how they are now. I'm just trying to add my 0.02? :-) I hope I'm not adding -2? Ciao Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it From hawk78_it at yahoo.it Mon Sep 19 02:40:40 2005 From: hawk78_it at yahoo.it (Vincenzo Di Massa) Date: Mon, 19 Sep 2005 02:40:40 +0200 Subject: [Distutils] [Bulk] Re: Files under /etc and /usr/share In-Reply-To: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> References: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> Message-ID: <200509190240.40852.hawk78_it@yahoo.it> Alle 17:49, domenica 18 settembre 2005, hai scritto: > At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote: > >Is it possible to install things into /etc using setuptools/easy_install? > > I think that this feature will be very appreciated. > >I know setup.py philosopy is to put everything inside eggs, but config > > files are a really special case! > >It would be nice beeing able to tell in setup.py what the config > > files/dirs are. The generated egg should install the config files on user > > request (using a gen_config script) or just when the neded are not config > > files on the system. > > I think that this is very application-specific as to how it should be > handled. It's very easy to create a package that contains configuration > files, with a function in __init__.py to install them, perhaps with some > options or an interactive setup facility. Listing the function in setup.py > then allows a script wrapper to be created for setting up the data. > > Using pkg_resources.resource_filename() with a directory resource will > unpack an entire directory tree to a temporary location, or give you the > location of files that are already unpacked, so you can then use shutil or > other normal ways of copying files. What you say seems to be the right thing (tm) :-). What I am proposing is nothing more that a convention to do config files installation. Maybe it is just something that can be explained into the documentation. It would be nice to ask setup.py writers to adopt the "best practice" that you descibed before. Maybe we can define what the name of the function is and what parameters it requires: this will make easy and obvious what is the right way of handling config options. Once the function name is standardized... why not provide a default implementation that automatically installs the files you specify somewhere? It is tedious to have to write code to just handle a little set of config files. It is a very common use case, so it would be good to provide some authomatism to: 1) make the distribution of many modules easier 2) provide a standard way to do one thing > >Another idea is for heavy data. Consider for example moin_moin. > >Using easy-deb i managed to aotomatically pacakge it into an egg (which is > >contained into a deb-package). The problem here is that it installs the > >template files inside the egg. Since template files must be copied > > "somewere" it is bat to put them into the egg where they are > > hidden/zipped. Moreover it would be nice to make the eggs inside > > /usr/lib/python?.?/site-packages more lightweight removing package data > > from them. > > I don't really agree. Projects with this kind of data are best off if they > provide a custom installation tool. Well, I should say that their *users* > are best off if they provide an installation tool, rather than dumping the > files in one place. Further, it is easier to support global installation > of a package for multiple users, or for a single user with multiple > installations of the tool. (E.g. multiple wikis.) If the data > installation is done by easy_install, it can only be done once, and without > any real options as to how it's done. If it's done by a function included > with the package, then it can be done as many times as desired, and with > whatever options the project wants to provide the user with. I did not explain myself very well, sorry. What you say is absolutely right! What I meant is: Suppose you have a module that has a an installation tool that is able to install data files/templates to multiple dirs at different times. All the copies are taken from somewhere. Now the source copy is inside the egg. My proposal was to move the source copy out of the egg. The reasons are: 1) you should be able to modify the default template 2) the template can be big an big templates belong to something different from /usr/lib/python?.?/... 3) provide a standard way to do one thing Maybe setuptools could optionally put a couple scripts inside generated eggs that can: 1) put default config files in place 2) move the templates somewhere else (like /usr/share) This way config files and data files could be handled automatically like setuptools does for dependencies: before executing the module could call pkg_resurces.config_files("modulename") and pkg_resources.data_files("modulename") to make sure that the config files exist and that data files are where they shuold be. > >There are 2 kind of package data: data that is used *by* the module code > > and data that is used *with* the module. moinmoin is just an example of > > module whose data is not used by the code but with the code. > > Well, in my way of thinking, such templates *are* used by the code, it's > just that they should be used by the project's "instance install/config" > script(s). I understand your point of view, any template can be seen as used by the module's code, but... There is a little difference: there is data that the user will hand edit, and data used an modified only by the module. There is data that must installed only once, and data (like moinmoin templates) that can be installed multiple times. I wanted to distinguish data that can be somehow "hidden" from data that must be "visible". > >Another nice thing to have in setuptools is a --nodeps option. Packagers > >really need to install an egg (into a package of course) even if its > >dependencies are not on the system. Skipped dependencies could be reported > >inside a skipped_deps.txt file or on stdout (so that packagers can add the > >dependency info to the generated packages). > > We could have a --record-deps option, but you can also just pull the > dependencies from the Distribution.requires() method. I have had it on my > list to have a --no-deps option, though, I just haven't gotten around to it > yet. You are doing a really good job with setuptools! I really like how they are now. I'm just trying to add my 0.02? :-) I hope I'm not adding -2? Ciao Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it From ianb at colorstudy.com Mon Sep 19 03:09:35 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 18 Sep 2005 20:09:35 -0500 Subject: [Distutils] Files under /etc and /usr/share In-Reply-To: <200509190251.55737.hawk78_it@yahoo.it> References: <5.1.1.6.0.20050918113722.01b9cd88@mail.telecommunity.com> <200509190251.55737.hawk78_it@yahoo.it> Message-ID: <432E0FCF.5030908@colorstudy.com> I haven't followed this discussion in detail, but I think I've started doing something like what Vincenzo asked about. It's not very general, though. In my case, I just put a file paste_deploy_config.ini_tmpl into the .egg-info directory, and then use that when deploying a web application. The file itself is a template for configuration, and those variables are filled in as part of the deployment. A more general and maybe even simpler method would simply be an entry point, like [etc_files], which would be an object with certain methods, which could be used to install it in some location. I imagine such an object would indicate the variables needed to install it, default location for the configuration file(s), and possibly other post-install routines. Anyway, I think that's probably what you are getting at here...? From pje at telecommunity.com Mon Sep 19 09:34:02 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 19 Sep 2005 03:34:02 -0400 Subject: [Distutils] Working around .pth/site problems Message-ID: <5.1.1.6.0.20050919023514.01b7da88@mail.telecommunity.com> The recent TurboGears release has been really interesting, in that it's given me a much better perspective on the kinds of problems Unix users have been having with EasyInstall. Basically, the fundamental problem in Unix-land is that the system-packaged Python expects to control all packages, and the default setup of Python's sys.path supports that theory. In essence, the problem is that Python adds all .pth-specified paths to the *end* of sys.path, meaning that system-installed packages in the stdlib or site-packages take precedence over anything else you can install, except for directories specified in PYTHONPATH or that are in the startup script's directory. Effectively, this means that easy-install.pth cannot be used to resolve the issue without using 'import' hackery such as importing pkg_resources or some other bootstrap module, in order to fudge the contents of sys.path. However, even such hackery isn't exactly ideal. The normal precedence for sys.path is: 1. the script directory, or current directory if no script 2. PYTHONPATH directories 3. system-defined directories 4. "site" directories hardcoded by site.py 5. directories specified by .pth files What we need to do is to insert certain directories between #2 and #3. Luckily, for Python 2.3 and 2.4, the first directory in category 3 always ends with "python2X.zip", so at least we can determine the correct insertion point. (By the way, in case you're wondering why eggs should go before system-defined directories as well as site directories, it's because it allows us to distribute e.g. back-ported standard library modules. For example, I'd love to make an egg for Python 2.4's doctest module and use it with Python 2.3. 2.3 also has a doctest module, but it lacks many useful unittest integration features.) Okay, so I could rework the .pth file processing to do this, such that it always chucks in a bootstrap module and the .pth file itself contains hackery to call it. That would allow you to override system packages if you had write access to an existing site-packages directory. But that's not really very useful, in terms of solving the big picture problem, which is how to use eggs (including setuptools itself) without affecting your main installation at *all*. And ideally, you should still be able to have default versions of packages selected, or at minimum there needs to be a way to get pkg_resources to be present, since pkg_resources can't be used to find its own egg. Okay, so suppose we just ditch easy-install.pth and setuptools.pth. Whenever we install setuptools in a target directory, we go ahead and install pkg_resources the "old-fashioned" way, because it's the only package that can't ever be multi-versioned anyway. (Ironically, setuptools itself could be multi-versioned, but setuptools really depends quite heavily on the pkg_resources version, so multi-versioning it isn't really practical.) Anyway, easy-install.pth could now be reserved for site-packages installs and other "site" directories, with all the current caveats applying. If you have control of your site-packages directory, the current scheme works just fine. But, if you don't have control, you can just set up an arbitrary "bin" directory for a given project, and install whatever you like there, as long as it's multi-version. If pkg_resources inserts require()'d eggs at the *beginning* of sys.path, instead of the end, then this would be perfect. However, there's a possible race condition here. Suppose the "foobar" package is installed in site-packages, and in an egg, and you import it before require()-ing it. Your require() would succeed, and add the egg to the front of sys.path. You're actually using the wrong version of "foobar", but have no way to know. Or worse, if there are multiple top-level modules or packages in "foobar", you might import some of them from one version and some from another! This problem is why I originally made pkg_resources add eggs to the end of sys.path. But, I now have a lot more data at my disposal. For example, eggs now carry a "top_level.txt" file listing their top-level module names, so in principle I could use that to check whether any of those modules had already been imported, and raise a VersionConflict error if so. That would probably address that issue. So, to recap my conclusions thus far: * Always install and compile pkg_resources the old-fashioned way, in addition to its egg form. * pkg_resources should add new eggs to the start of sys.path, and check for already-imported, top-level, non-namespace modules or packages With just these two changes, it should be possible to create usable runtime environments for tools that understand eggs, and any code that's packaged as a distutils or setuptools project. (Because you can then use setup.py develop or easy_install to generate script wrappers that ensure the right versions of things are require()'d onto sys.path.) This does not address default versions using easy-install.pth or setuptools.pth - in effect those are limited to installations with site-packages access, or for people using the "non-root installation" instructions for EasyInstall. However, it's fairly reasonable, I think, to say that you can't make something the default version if you already have a system-defined default version of something. The fact that you still have an option to explicitly require() what you want is a nice bonus, and it helps move us forward to the day when --multi-version will be the normal way to install things. The downside to these changes, of course, is that they may introduce new problems. In particular, I'll need to do some rework on ez_setup to allow for the possibility that pkg_resources can be on sys.path, but not setuptools. Thoughts, anyone? From ianb at colorstudy.com Mon Sep 19 17:46:07 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 19 Sep 2005 10:46:07 -0500 Subject: [Distutils] Installed but can't run the quickstart In-Reply-To: <778BE476-DBA7-4A74-84CD-41AFCF72F6F0@redivi.com> References: <1127136254.816893.7500@f14g2000cwb.googlegroups.com> <1226A6CB-FE49-43BF-83F6-816E106E8DA1@redivi.com> <1127138591.279122.310160@g49g2000cwa.googlegroups.com> <778BE476-DBA7-4A74-84CD-41AFCF72F6F0@redivi.com> Message-ID: <432EDD3F.4010005@colorstudy.com> Bob Ippolito wrote: > Well, if it makes a difference, EasyInstall works great on Mac OS X and > Windows. If Debian users had been more active with the EasyInstall > development process then the experience would be better there also. I've used EasyInstall on Debian all along, without problems (well, no more than anyone else I think). I guess I just happened to avoid conflicts with debian packages (well, sometimes). The setuptools deb package should just be removed; as far as I can tell nothing depends on it anyway (well, at least on my now-Ubuntu system "apt-cache rdepends python2.3-setuptools" shows nothing). There is a significant concern about setuptools conflicting with Debian packages, which I have encountered, and you can't always uninstall the Debian package because that can have cascading effects. At some point there needs to be a way to tell setuptools that a package has been provided, egg or no egg. Surprisingly nothing depends on ElementTree (that I can find), but docutils is caught up in the network of Debian dependencies. (Well, there's also ways of indicating to Debian that you've provided a dependency somehow besides the package, but I can't remember how.) Since this really has just become a Distutils discussion I'll copy it over there... -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From theller at python.net Tue Sep 20 18:36:56 2005 From: theller at python.net (Thomas Heller) Date: Tue, 20 Sep 2005 18:36:56 +0200 Subject: [Distutils] Getting tired with py2exe Message-ID: I'm slowly getting tired maintaining py2exe. It is far from perfect, although it has interesting features (I would say). The problem, apart from the work, is that it is good enough for me - I can do everything that I need with it. But I assume I use far less libaries than other Python programmers, so a lot of bugs will never bite me. It is also interesting that the recently introduced bundle-files option, which allows to build single-file exes has gained a lot of interest - although the ONLY use case (so far) I have myself for it is to implement inproc COM servers which will compatible with Python clients (and other Python inproc COM servers) because of the total isolation of the Python VMs. Is anyone interested in taking over the maintainance, documentation, and further development? Should py2exe be integrated into another, larger, package? Pywin32 comes to mind, but also Philip Eby's setuptools (that's why I post to distutils-sig as well)... Thomas From greg.steffensen at gmail.com Tue Sep 20 21:15:59 2005 From: greg.steffensen at gmail.com (Greg Steffensen) Date: Tue, 20 Sep 2005 15:15:59 -0400 Subject: [Distutils] setuptools package_data behavior Message-ID: <438a2361050920121520890ad4@mail.gmail.com> Hey, I'm trying to use setuptools (which looks extremely cool) to distribute a reasonably simple python package (clipartbrowser on pypi, if anyone's curious), and am running into difficulty figuring out how to include some data files. My package is more or less laid out as follows (based on the example layout provided in Ian's setuptools presentation): clipartbrowser/ README COPYING ez_setup.py setup.py clipartbrowser/ __init__.py browser.py clipartbrowser.glade repositories/ __init__.py localocal.py localocal.glade I already have the code setup to use pkg_resources to access the data files (clipartbrowser.glade and localocal.glade), and have tried to list them under package_data in setup.py several ways (including "package_data = {'' : '['*.glade']} " ), but they never seem to be included in the tarball when I use "setup.py sdist". They are included when I do "setup.py bdist_egg", but for now I'd rather distribute using traditional tarballs. Am I doing something wrong? Also, the README is magically included in the sdist tarball no matter what I do, but I'd like to include COPYING, AUTHORS, etc. Is there a "correct" way to do this? Thanks, Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050920/3c59715b/attachment.htm From greg.steffensen at gmail.com Tue Sep 20 21:22:42 2005 From: greg.steffensen at gmail.com (Greg Steffensen) Date: Tue, 20 Sep 2005 15:22:42 -0400 Subject: [Distutils] setuptools package_data behavior In-Reply-To: <438a2361050920121520890ad4@mail.gmail.com> References: <438a2361050920121520890ad4@mail.gmail.com> Message-ID: <438a236105092012223ea3e4a0@mail.gmail.com> Oops, typo: I've tried using package_data = {'' : ['*.glade'] } in setup.py, not package_data = {'' : '['*.glade']} as I originally wrote. The problem isn't the typo. Greg On 9/20/05, Greg Steffensen wrote: > > Hey, I'm trying to use setuptools (which looks extremely cool) to > distribute a reasonably simple python package (clipartbrowser on pypi, if > anyone's curious), and am running into difficulty figuring out how to > include some data files. My package is more or less laid out as follows > (based on the example layout provided in Ian's setuptools presentation): > > clipartbrowser/ > README > COPYING > ez_setup.py > setup.py > clipartbrowser/ > __init__.py > browser.py > clipartbrowser.glade > repositories/ > __init__.py > localocal.py > localocal.glade > > I already have the code setup to use pkg_resources to access the data > files (clipartbrowser.glade and localocal.glade), and have tried to list > them under package_data in setup.py several ways (including "package_data > = {'' : '['*.glade']} " ), but they never seem to be included in the tarball > when I use "setup.py sdist". They are included when I do "setup.pybdist_egg", but for now I'd rather distribute using traditional tarballs. Am > I doing something wrong? > > Also, the README is magically included in the sdist tarball no matter what > I do, but I'd like to include COPYING, AUTHORS, etc. Is there a "correct" > way to do this? Thanks, > > Greg > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050920/f694a93a/attachment.html From pje at telecommunity.com Tue Sep 20 21:25:18 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 15:25:18 -0400 Subject: [Distutils] setuptools package_data behavior In-Reply-To: <438a2361050920121520890ad4@mail.gmail.com> Message-ID: <5.1.1.6.0.20050920152134.01b99810@mail.telecommunity.com> At 03:15 PM 9/20/2005 -0400, Greg Steffensen wrote: >They are included when I do "setup.py bdist_egg", but for now I'd rather >distribute using traditional tarballs. Am I doing something wrong? > >Also, the README is magically included in the sdist tarball no matter what >I do, but I'd like to include COPYING, AUTHORS, etc. Is there a "correct" >way to do this? Thanks, By default, setuptools will include any file under Subversion or CVS revision control in the source distribution. If you're not using one of those revision control tools, you have to use the standard distutils MANIFEST.in rules file to specify what files to include in the source distribution. See here for more details: http://www.python.org/doc/2.4.1/dist/manifest.html From ianb at colorstudy.com Tue Sep 20 21:59:15 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Sep 2005 14:59:15 -0500 Subject: [Distutils] setuptools: name quoting Message-ID: <43306A13.2050409@colorstudy.com> I'm a bit confused about name quoting. I have a package with a name "Todo_SQLObject_ZPT", and egg_info writes to "Todo-SQLObject-ZPT.egg-info". Why? I thought -'s were bad, but _'s were okay...? I'm also getting a traceback when I try to do "python setup develop", following. The application that is causing this problem is at http://svn.pythonpaste.org/Paste/apps/Todo_SQLObject_ZPT/trunk (it's probably broken in other ways, I'm still in the process of converting it). I'm using the latest CVS version of setuptools. Todo_SQLObject_ZPT$ python setup.py egg_info develop running egg_info writing ./Todo-SQLObject-ZPT.egg-info/PKG-INFO writing top-level names to ./Todo-SQLObject-ZPT.egg-info/top_level.txt running develop Traceback (most recent call last): File "setup.py", line 18, in ? package_data={ File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.4/distutils/dist.py", line 965, in run_command cmd_obj.ensure_finalized() File "/usr/local/lib/python2.4/distutils/cmd.py", line 117, in ensure_finalized self.finalize_options() File "/usr/home/ianb/co/setuptools/setuptools/command/develop.py", line 45, in finalize_options easy_install.finalize_options(self) File "/usr/home/ianb/co/setuptools/setuptools/command/easy_install.py", line 181, in finalize_options self.package_index = self.create_index( File "/usr/home/ianb/co/setuptools/setuptools/package_index.py", line 128, in __init__ Environment.__init__(self,*args,**kw) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 558, in __init__ self.scan(search_path) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 588, in scan self.add(dist) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 610, in add if dist not in dists: File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1760, in __cmp__ def __cmp__(self, other): return cmp(self.hashcmp, other) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1760, in __cmp__ def __cmp__(self, other): return cmp(self.hashcmp, other) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1755, in lambda self: ( File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1782, in parsed_version self._parsed_version = pv = parse_version(self.version) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1797, in version raise ValueError( ValueError: ("Missing 'Version:' header and/or PKG-INFO file", Todo-SQLObject-ZPT [unknown version] (/usr/home/ianb/co/Todo_SQLObject_ZPT)) From bob at redivi.com Wed Sep 21 01:12:49 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 20 Sep 2005 19:12:49 -0400 Subject: [Distutils] [Py2exe-users] Re: Getting tired with py2exe In-Reply-To: References: Message-ID: <999C4F9B-61DB-4229-8CD4-FC2B929CAC10@redivi.com> On Sep 20, 2005, at 5:44 PM, Steve Holden wrote: > Thomas Heller wrote: > >> I'm slowly getting tired maintaining py2exe. It is far from perfect, >> although it has interesting features (I would say). >> The problem, apart from the work, is that it is good enough for me >> - I >> can do everything that I need with it. But I assume I use far less >> libaries than other Python programmers, so a lot of bugs will >> never bite >> me. >> It is also interesting that the recently introduced bundle-files >> option, >> which allows to build single-file exes has gained a lot of interest - >> although the ONLY use case (so far) I have myself for it is to >> implement >> inproc COM servers which will compatible with Python clients (and >> other >> Python inproc COM servers) because of the total isolation of the >> Python >> VMs. >> Is anyone interested in taking over the maintainance, >> documentation, and >> further development? >> Should py2exe be integrated into another, larger, package? Pywin32 >> comes to mind, but also Philip Eby's setuptools (that's why I post to >> distutils-sig as well)... >> > Ignoring all the philosophical questions I'd like to thank you for > all your hard work on py2exe over the years, which has benefited > the Windows Python community immeasurably. I'd like to thank you as well. Although I'm primarily a Mac OS X (and various other *nix-ish things) user myself, I have used py2exe on several occasions to package a commercial product and to give various one-off applications to clients. py2exe was also a large inspiration for py2app (which I have been neglecting lately). py2exe (and py2app) currently do everything I need them do (albeit with a little prodding), so that's why I've done so little with it in the past few months. I hope that the packager-future will be largely setuptools based and that the various platform-specific packagers will share a lot more code in the future (setuptools, modulegraph, etc.), making maintenance easier and more fun for everyone. This was my primary use case when I was initially discussing the egg spec with PJE back around pycon-time (though I have been unfortunately useless implementing and evolving it). Right now, I think the packagers and the packages are at odds, because the packagers need metadata that the packages don't provide (in a pre-setuptools universe)... so right now users (or the packagers) need to know a lot of magic incantations to make the various complicated Python packages work, where with setuptools based packages the magic incantations are built-in :) -bob From steve at holdenweb.com Tue Sep 20 23:44:06 2005 From: steve at holdenweb.com (Steve Holden) Date: Tue, 20 Sep 2005 22:44:06 +0100 Subject: [Distutils] Getting tired with py2exe In-Reply-To: References: Message-ID: Thomas Heller wrote: > I'm slowly getting tired maintaining py2exe. It is far from perfect, > although it has interesting features (I would say). > > The problem, apart from the work, is that it is good enough for me - I > can do everything that I need with it. But I assume I use far less > libaries than other Python programmers, so a lot of bugs will never bite > me. > > It is also interesting that the recently introduced bundle-files option, > which allows to build single-file exes has gained a lot of interest - > although the ONLY use case (so far) I have myself for it is to implement > inproc COM servers which will compatible with Python clients (and other > Python inproc COM servers) because of the total isolation of the Python > VMs. > > Is anyone interested in taking over the maintainance, documentation, and > further development? > > Should py2exe be integrated into another, larger, package? Pywin32 > comes to mind, but also Philip Eby's setuptools (that's why I post to > distutils-sig as well)... > Ignoring all the philosophical questions I'd like to thank you for all your hard work on py2exe over the years, which has benefited the Windows Python community immeasurably. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.pycon.org From pje at telecommunity.com Wed Sep 21 02:59:39 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 20:59:39 -0400 Subject: [Distutils] [Py2exe-users] Re: Getting tired with py2exe In-Reply-To: <999C4F9B-61DB-4229-8CD4-FC2B929CAC10@redivi.com> References: Message-ID: <5.1.1.6.0.20050920202429.020d5a48@mail.telecommunity.com> At 07:12 PM 9/20/2005 -0400, Bob Ippolito wrote: >I hope that the packager-future will be largely setuptools based and >that the various platform-specific packagers will share a lot more >code in the future (setuptools, modulegraph, etc.), making >maintenance easier and more fun for everyone. This was my primary >use case when I was initially discussing the egg spec with PJE back >around pycon-time (though I have been unfortunately useless >implementing and evolving it). > >Right now, I think the packagers and the packages are at odds, >because the packagers need metadata that the packages don't provide >(in a pre-setuptools universe)... so right now users (or the >packagers) need to know a lot of magic incantations to make the >various complicated Python packages work, where with setuptools based >packages the magic incantations are built-in :) Also, my outsider-observation of both py2exe and py2app is that they are significantly complicated by the attempt to select and distribute only a subset of the Python standard library. If you can live with just putting the entire stdlib in a zipfile, or relying on a locally-installed Python (e.g. via the .msi on Windows, or the system-provided Python on OS X) then the build process is a lot simpler. The latest setuptools does .exe wrappers for console scripts, and will soon have them for GUI scripts. Basically, if you want to build your package to a "single-directory deploy" with setuptools, you can do: python setup.py easy_install -ad deployment_directory . where deployment directory is the directory that you'll distribute. The -a is short for --always-copy, which means to copy all required eggs to the target directory whether they're already on sys.path or not. The script directory defaults to the -d directory, so the .py files and .exe wrappers for console scripts are generated there as well, but the .py files will be set to execute via the same Python you ran the above command with. So, if you wrap the directory up with an installer that also installs the Python version you depend on, you're basically done, at least for Windows. It's not perfect, I'll admit. You can't get that nice "drop the .exe in a directory and it's set to go" feeling. In principle I suppose you could cram the python DLL and stdlib zip and the needed modules all into one .exe file, but it seems counterproductive for the end user to have a dozen copies of Python on their machine, even with stripped-down stdlibs. DLL hell doesn't get into it because everything's versioned. Frankly, it seems to me like having an all-purpose egg installer that installed Python and put the eggs in a shared location would be even better, since applications with eggs in common wouldn't need them downloaded more than once. The trick to that is of course that somebody's got to write that installer. :) On Linux and the BSD's, it suffices to package your eggs and the Python they depend on, so distributing the result of 'easy_install -ad' is adequate for binary-compatible installs. Or, you can rely on the local packaging system to have the packages that install the eggs you need when you install a package for your app. The only tweak possibly necessary is the #! paths in the scripts. That leaves OS X, where I gather that convention and policy dictates that applications just be runnable without an install step. There you can bundle all your eggs, and rely on the system Python, as long as it's recent enough. Ideally, then, you need a wrapper for the "easy_install -ad" directory that makes one or more real OS X apps, in the way that setuptools makes .exe wrappers now. (Only more complex, since all it really does right now is copy and rename a standard launcher .exe file.) So, it seems to me that if you can rely on Python and the stdlib, the scope of what a py2exe or py2app tool has to do in the setuptools context is at least somewhat reduced. However, there's definitely a need for other developers to be involved, because with my current amount of spare time, it'll be a year or two before I could even think about doing anything py2app-ish. (I don't even have a Mac yet, for example.) It's possible I might get some support for OSAF for some of this, in which case my time and resources would be increased, but I haven't broached that subject yet. It's very likely that Chandler will be egg-based, but it's not clear to me yet whether that will be going all the way down to the .exe/.app/.so and installer level or not. But having a way that we could build Chandler executables for Windows, Mac, and Linux with a single (and relatively simple) setup.py would be very nice, and even better if it meant anybody could do similar single-setup cross-platform deployment. From pje at telecommunity.com Wed Sep 21 03:12:48 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 21:12:48 -0400 Subject: [Distutils] setuptools: name quoting In-Reply-To: <43306A13.2050409@colorstudy.com> Message-ID: <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> At 02:59 PM 9/20/2005 -0500, Ian Bicking wrote: >I'm a bit confused about name quoting. I have a package with a name >"Todo_SQLObject_ZPT", and egg_info writes to >"Todo-SQLObject-ZPT.egg-info". Why? I thought -'s were bad, but _'s >were okay...? It's safe to use either, but setuptools considers '-' to be canonical; this was a heuristic decision on my part because '-' appears more frequently used than '_' in the names of projects currently registered with PyPI. The problem, however, is that PyPI wants your uploaded files to *exactly* match the registered name. If you used '-' in setup.py, you must have '-' in the filename, and that's something setuptools cannot do. It would probably be best if setuptools ensured the "register" command always used '_', but then this defeats the purpose of having it that way in the first place! So, the only real fix is to change PyPI to accept files whose pkg_resources.safe_name() match the safe_name() of the project. In the meantime, you can't upload eggs to PyPI for packages with '-' or '_' in their names. (You can upload source distros or other formats, since those can work with ambiguous filenames.) I probably should put in an RFE or proposed patch, but I've been *really* busy lately. From ianb at colorstudy.com Wed Sep 21 03:19:42 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Sep 2005 20:19:42 -0500 Subject: [Distutils] setuptools: name quoting In-Reply-To: <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> References: <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> Message-ID: <4330B52E.3090802@colorstudy.com> Phillip J. Eby wrote: > At 02:59 PM 9/20/2005 -0500, Ian Bicking wrote: > >> I'm a bit confused about name quoting. I have a package with a name >> "Todo_SQLObject_ZPT", and egg_info writes to >> "Todo-SQLObject-ZPT.egg-info". Why? I thought -'s were bad, but _'s >> were okay...? > > > It's safe to use either, but setuptools considers '-' to be canonical; > this was a heuristic decision on my part because '-' appears more > frequently used than '_' in the names of projects currently registered > with PyPI. > > The problem, however, is that PyPI wants your uploaded files to > *exactly* match the registered name. If you used '-' in setup.py, you > must have '-' in the filename, and that's something setuptools cannot > do. It would probably be best if setuptools ensured the "register" > command always used '_', but then this defeats the purpose of having it > that way in the first place! > > So, the only real fix is to change PyPI to accept files whose > pkg_resources.safe_name() match the safe_name() of the project. In the > meantime, you can't upload eggs to PyPI for packages with '-' or '_' in > their names. (You can upload source distros or other formats, since > those can work with ambiguous filenames.) I probably should put in an > RFE or proposed patch, but I've been *really* busy lately. Richard, would you be able to make this change? It seems pretty reasonable to accept files that closely match project names. From bob at redivi.com Wed Sep 21 03:27:08 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 20 Sep 2005 21:27:08 -0400 Subject: [Distutils] [Py2exe-users] Re: Getting tired with py2exe In-Reply-To: <5.1.1.6.0.20050920202429.020d5a48@mail.telecommunity.com> References: <5.1.1.6.0.20050920202429.020d5a48@mail.telecommunity.com> Message-ID: On Sep 20, 2005, at 8:59 PM, Phillip J. Eby wrote: > > > At 07:12 PM 9/20/2005 -0400, Bob Ippolito wrote: > > >> I hope that the packager-future will be largely setuptools based and >> that the various platform-specific packagers will share a lot more >> code in the future (setuptools, modulegraph, etc.), making >> maintenance easier and more fun for everyone. This was my primary >> use case when I was initially discussing the egg spec with PJE back >> around pycon-time (though I have been unfortunately useless >> implementing and evolving it). >> >> Right now, I think the packagers and the packages are at odds, >> because the packagers need metadata that the packages don't provide >> (in a pre-setuptools universe)... so right now users (or the >> packagers) need to know a lot of magic incantations to make the >> various complicated Python packages work, where with setuptools based >> packages the magic incantations are built-in :) >> > > Also, my outsider-observation of both py2exe and py2app is that > they are significantly complicated by the attempt to select and > distribute only a subset of the Python standard library. No, stdlib is easy because it's not that wonky and I've already done a pretty thorough analysis of what imports what (from C or Python) and coded around it. The "coded around it" is basically just a couple lines of code that enumerate the implicit dependencies. It's third party that's hard. If everyone used pkg_resources and setuptools then that'd be easy too, but they don't yet :) py2app currently works around that fact by including package-specific knowledge for a couple popular packages, but I don't want to maintain that forever. -bob From pje at telecommunity.com Wed Sep 21 03:31:24 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 21:31:24 -0400 Subject: [Distutils] setuptools: name quoting In-Reply-To: <4330B52E.3090802@colorstudy.com> References: <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050920212443.020ce820@mail.telecommunity.com> At 08:19 PM 9/20/2005 -0500, Ian Bicking wrote: >Phillip J. Eby wrote: > > At 02:59 PM 9/20/2005 -0500, Ian Bicking wrote: > > > >> I'm a bit confused about name quoting. I have a package with a name > >> "Todo_SQLObject_ZPT", and egg_info writes to > >> "Todo-SQLObject-ZPT.egg-info". Why? I thought -'s were bad, but _'s > >> were okay...? > > > > > > It's safe to use either, but setuptools considers '-' to be canonical; > > this was a heuristic decision on my part because '-' appears more > > frequently used than '_' in the names of projects currently registered > > with PyPI. > > > > The problem, however, is that PyPI wants your uploaded files to > > *exactly* match the registered name. If you used '-' in setup.py, you > > must have '-' in the filename, and that's something setuptools cannot > > do. It would probably be best if setuptools ensured the "register" > > command always used '_', but then this defeats the purpose of having it > > that way in the first place! > > > > So, the only real fix is to change PyPI to accept files whose > > pkg_resources.safe_name() match the safe_name() of the project. In the > > meantime, you can't upload eggs to PyPI for packages with '-' or '_' in > > their names. (You can upload source distros or other formats, since > > those can work with ambiguous filenames.) I probably should put in an > > RFE or proposed patch, but I've been *really* busy lately. > >Richard, would you be able to make this change? It seems pretty >reasonable to accept files that closely match project names. FYI, here are the safe_name/safe_version source: def safe_name(name): """Convert an arbitrary string to a standard distribution name Any runs of non-alphanumeric characters are replaced with a single '-'. """ return re.sub('[^A-Za-z0-9]+', '-', name) def safe_version(version): """Convert an arbitrary string to a standard version string Spaces become dots, and all other non-alphanumeric characters become dashes, with runs of multiple dashes condensed to a single dash. """ version = version.replace(' ','.') return re.sub('[^A-Za-z0-9.]+', '-', version) To make an egg filename, these strings are taken and the '-' converted to a '_' so that it isn't confused with the '-' that goes between parts of the name. So given the metadata: setup( name="To-Do List", version="1.23 alpha!", ... ) The egg filename will be "To_Do_List-1.23.alpha_-py2.4-win32.egg" if generated by Python 2.4 on Windows (and it contains C extensions). Currently, if I understand it correctly, PyPI will reject this name if the name and version used for registration were "To-Do List" and "1.23 alpha!". From ianb at colorstudy.com Wed Sep 21 04:03:26 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Sep 2005 21:03:26 -0500 Subject: [Distutils] setuptools: site_dirs Message-ID: <4330BF6E.5090106@colorstudy.com> With easy_install I can specify --site-dirs to make it setup package with .pth files. But with setuptools alone I can't, particularly "python setup.py develop". I'm not sure if there's a setting in distutils.cfg I could use...? There doesn't seem to be any appropriate command-line option. I'm also not sure how the site directories are identified...? In this case, on Debian, /usr/local/lib/pythonX.Y/site-packages isn't recognized as such a directory, even after I added it with site.addsitedir in sitecustomize. Ian From rkern at ucsd.edu Wed Sep 21 05:38:41 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 20 Sep 2005 20:38:41 -0700 Subject: [Distutils] setuptools: site_dirs In-Reply-To: <4330BF6E.5090106@colorstudy.com> References: <4330BF6E.5090106@colorstudy.com> Message-ID: Ian Bicking wrote: > With easy_install I can specify --site-dirs to make it setup package > with .pth files. But with setuptools alone I can't, particularly > "python setup.py develop". I'm not sure if there's a setting in > distutils.cfg I could use...? There doesn't seem to be any appropriate > command-line option. > > I'm also not sure how the site directories are identified...? In this > case, on Debian, /usr/local/lib/pythonX.Y/site-packages isn't recognized > as such a directory, even after I added it with site.addsitedir in > sitecustomize. On Debian that directory is added to the list of site directories in site.addsitepackages(), which is automatically executed through site.main(). If it's not working on your machine, then I think it's a problem with your particular configuration. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From richardjones at optushome.com.au Wed Sep 21 05:46:57 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Wed, 21 Sep 2005 13:46:57 +1000 Subject: [Distutils] [Catalog-sig] setuptools: name quoting In-Reply-To: <4330B52E.3090802@colorstudy.com> References: <5.1.1.6.0.20050920210448.0348dd00@mail.telecommunity.com> <4330B52E.3090802@colorstudy.com> Message-ID: <200509211347.02242.richardjones@optushome.com.au> On Wed, 21 Sep 2005 11:19 am, Ian Bicking wrote: > Richard, would you be able to make this change? It seems pretty > reasonable to accept files that closely match project names. Yes, as long as there's no chance of conflicts between project names due to the rewriting, which I figure is pretty remote given we're not touching alphanums. As for when I would actually be able to make this change. Well, I'll look into it as soon as I can, but at the moment my plate is pretty full :( Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050921/ccc5ea18/attachment-0001.pgp From ianb at colorstudy.com Wed Sep 21 05:52:43 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Sep 2005 22:52:43 -0500 Subject: [Distutils] setuptools: site_dirs In-Reply-To: References: <4330BF6E.5090106@colorstudy.com> Message-ID: <4330D90B.8070103@colorstudy.com> Robert Kern wrote: > Ian Bicking wrote: > >>With easy_install I can specify --site-dirs to make it setup package >>with .pth files. But with setuptools alone I can't, particularly >>"python setup.py develop". I'm not sure if there's a setting in >>distutils.cfg I could use...? There doesn't seem to be any appropriate >>command-line option. >> >>I'm also not sure how the site directories are identified...? In this >>case, on Debian, /usr/local/lib/pythonX.Y/site-packages isn't recognized >>as such a directory, even after I added it with site.addsitedir in >>sitecustomize. > > > On Debian that directory is added to the list of site directories in > site.addsitepackages(), which is automatically executed through > site.main(). If it's not working on your machine, then I think it's a > problem with your particular configuration. That's what I figured. But it's working -- setuptools just doesn't know that it's working, and so it doesn't create .pth files. From growlf at biocede.com Wed Sep 21 06:08:35 2005 From: growlf at biocede.com (Garth Johnson) Date: Tue, 20 Sep 2005 22:08:35 -0600 Subject: [Distutils] [python-win32] Getting tired with py2exe In-Reply-To: References: Message-ID: <4330DCC3.6090409@biocede.com> Steve Holden wrote: >Ignoring all the philosophical questions I'd like to thank you for all >your hard work on py2exe over the years, which has benefited the Windows >Python community immeasurably. > >regards > Steve > > here here! I have just begun my trek into Python and am already relying upon Py2Exe for my projects. Thank you for an effective, well designed tool that is easy to use. It was one of the final deciding factors in our company's choice of languages. (not the least of which was the ease of which our dev team took to Python after pretzling with perl). I do hope you change your mind. Garth From rkern at ucsd.edu Wed Sep 21 06:38:32 2005 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 20 Sep 2005 21:38:32 -0700 Subject: [Distutils] setuptools: site_dirs In-Reply-To: <4330D90B.8070103@colorstudy.com> References: <4330BF6E.5090106@colorstudy.com> <4330D90B.8070103@colorstudy.com> Message-ID: Ian Bicking wrote: > Robert Kern wrote: > >>Ian Bicking wrote: >> >>>With easy_install I can specify --site-dirs to make it setup package >>>with .pth files. But with setuptools alone I can't, particularly >>>"python setup.py develop". I'm not sure if there's a setting in >>>distutils.cfg I could use...? There doesn't seem to be any appropriate >>>command-line option. >>> >>>I'm also not sure how the site directories are identified...? In this >>>case, on Debian, /usr/local/lib/pythonX.Y/site-packages isn't recognized >>>as such a directory, even after I added it with site.addsitedir in >>>sitecustomize. >> >>On Debian that directory is added to the list of site directories in >>site.addsitepackages(), which is automatically executed through >>site.main(). If it's not working on your machine, then I think it's a >>problem with your particular configuration. > > That's what I figured. But it's working -- setuptools just doesn't know > that it's working, and so it doesn't create .pth files. Ah, okay. I thought you were saying that it wasn't working in general (since site.addsitedir() doesn't affect whether or not easy_install will install to it; it just affects whether or not .pth files will work in general). You're right; there doesn't seem to be a way to tell the develop command that the directory is a site-dir. Perhaps it could check for the existence of easy-install.pth. Or perhaps the --site-dirs option should be duplicated there, too. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From ronaldoussoren at mac.com Wed Sep 21 21:49:05 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 21 Sep 2005 21:49:05 +0200 Subject: [Distutils] [Py2exe-users] Re: Getting tired with py2exe In-Reply-To: <5.1.1.6.0.20050920202429.020d5a48@mail.telecommunity.com> References: <5.1.1.6.0.20050920202429.020d5a48@mail.telecommunity.com> Message-ID: <32E115FE-CD93-42DA-B945-B550B21FDB3E@mac.com> On 21-sep-2005, at 2:59, Phillip J. Eby wrote: > > > > That leaves OS X, where I gather that convention and policy > dictates that > applications just be runnable without an install step. There you can > bundle all your eggs, and rely on the system Python, as long as > it's recent > enough. Ideally, then, you need a wrapper for the "easy_install -ad" > directory that makes one or more real OS X apps, in the way that > setuptools > makes .exe wrappers now. (Only more complex, since all it really does > right now is copy and rename a standard launcher .exe file.) > Relying on the system python might not be a smart thing to do. Apple is taking backward compatibility seriously, but not always for the unix-y bits. That may or may not include Python. It would be rather annoying if an application did depent on the system python and Apple decided to ship another release of python. > > So, it seems to me that if you can rely on Python and the stdlib, > the scope > of what a py2exe or py2app tool has to do in the setuptools context > is at > least somewhat reduced. However, there's definitely a need for other > developers to be involved, because with my current amount of spare > time, > it'll be a year or two before I could even think about doing anything > py2app-ish. (I don't even have a Mac yet, for example.) > Building something py2app-ish is easy enough: take py2app and hack it until it fits into your scheme. I've done this at work to build standone unix daemons and that wasn't very hard. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050921/fc883f37/smime.bin From ianb at colorstudy.com Wed Sep 21 22:11:31 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Sep 2005 15:11:31 -0500 Subject: [Distutils] setuptools docs: Declaring Dependencies Message-ID: <4331BE73.7060908@colorstudy.com> I'm reading through the documentation some... Regarding this: > Note, by the way, that if you declare your dependencies in setup.py, > you do not need to use the require() function in your scripts or > modules, as long as you either install the project or use setup.py > develop to do development work on it. (See "Development Mode" below > for more details on using setup.py develop.) I don't think this is quite true. In particular, I've noticed if you install a package A --multi-version, it fulfills the requirement for A. And if you have a package B that requires A, when you require('B') then A will be loaded (added to sys.path, more technically). But if B is *not* installed --multi-version you can start using it without require()ing it, and A won't have been loaded. Given this, I feel like I need to do the basic require()s in my __init__.py. But, at least for some packages (like SQLObject) I want the package to be usable without setuptools. It gets complicated then, because you'd have to do: try: import pkg_resources try: pkg_resources.require(...) except pkg_resources.ResolutionError: # Who knows how everything may be installed, lets be # optimistic and wait for a later ImportError... pass except ImportError: pass At least, that's the best I've figured out. From pje at telecommunity.com Wed Sep 21 22:24:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 21 Sep 2005 16:24:42 -0400 Subject: [Distutils] setuptools docs: Declaring Dependencies In-Reply-To: <4331BE73.7060908@colorstudy.com> Message-ID: <5.1.1.6.0.20050921161816.034875a8@mail.telecommunity.com> At 03:11 PM 9/21/2005 -0500, Ian Bicking wrote: >I'm reading through the documentation some... > >Regarding this: > > > Note, by the way, that if you declare your dependencies in setup.py, > > you do not need to use the require() function in your scripts or > > modules, as long as you either install the project or use setup.py > > develop to do development work on it. (See "Development Mode" below > > for more details on using setup.py develop.) > >I don't think this is quite true. In particular, I've noticed if you >install a package A --multi-version, it fulfills the requirement for A. > And if you have a package B that requires A, when you require('B') >then A will be loaded (added to sys.path, more technically). But if B >is *not* installed --multi-version you can start using it without >require()ing it, and A won't have been loaded. What I was trying to say was that scripts installed by easy_install or "setup develop" do not require any require()s, as that is handled by the generated wrapper. >Given this, I feel like I need to do the basic require()s in my >__init__.py. I'm not positive that's true. If you install B without --multi-version, in principle it should de-multi-versionize A as well. (At least, that's what I'd expect it to do; I'm not 100% sure if it does. I know for sure it'll do it if "A" isn't installed when you install "B".) That doesn't mean you can't go back and re-MV-ize "A", but there are always ways for people to break things. A full package management tool layered over setuptools should take care of things like warning you that you're going to deactivate something needed by something else. From ianb at colorstudy.com Thu Sep 22 01:32:09 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Sep 2005 18:32:09 -0500 Subject: [Distutils] setuptools docs: Declaring Dependencies In-Reply-To: <5.1.1.6.0.20050921161816.034875a8@mail.telecommunity.com> References: <5.1.1.6.0.20050921161816.034875a8@mail.telecommunity.com> Message-ID: <4331ED79.9020300@colorstudy.com> Phillip J. Eby wrote: >> Regarding this: >> >> > Note, by the way, that if you declare your dependencies in setup.py, >> > you do not need to use the require() function in your scripts or >> > modules, as long as you either install the project or use setup.py >> > develop to do development work on it. (See "Development Mode" below >> > for more details on using setup.py develop.) >> >> I don't think this is quite true. In particular, I've noticed if you >> install a package A --multi-version, it fulfills the requirement for A. >> And if you have a package B that requires A, when you require('B') >> then A will be loaded (added to sys.path, more technically). But if B >> is *not* installed --multi-version you can start using it without >> require()ing it, and A won't have been loaded. > > > What I was trying to say was that scripts installed by easy_install or > "setup develop" do not require any require()s, as that is handled by the > generated wrapper. You also mention modules there. >> Given this, I feel like I need to do the basic require()s in my >> __init__.py. > > > I'm not positive that's true. If you install B without --multi-version, > in principle it should de-multi-versionize A as well. (At least, that's > what I'd expect it to do; I'm not 100% sure if it does. I know for sure > it'll do it if "A" isn't installed when you install "B".) That doesn't > mean you can't go back and re-MV-ize "A", but there are always ways for > people to break things. Yes. But, I'm pretty sure I've done these things (though in retrospect it's hard to remember exactly what and in what order). In part it's because of my old sys.path ways. But I don't think it will be uncommon to get in these jams... so, there should be a clear way out. Not that the paragraph above will give a clear way no matter what it says... I think better error messages would probably be the best way to make this stuff easier. E.g., version conflict messages could include file paths. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Thu Sep 22 01:41:36 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Sep 2005 18:41:36 -0500 Subject: [Distutils] buildutils Message-ID: <4331EFB0.5040306@colorstudy.com> I thought I'd note that I've been doing a little bit of work on Ryan Tomayko's buildutils lately (http://buildutils.lesscode.org/). It now has distutils.command entry points, so any setuptools-based package will get the new commands. And I've moved the svntag command from PasteDeploy to buildutils. I guess next up I'm going to re-look at how to enable some of these commands conditionally, along the lines discussed in this thread: http://mail.python.org/pipermail/distutils-sig/2005-August/005011.html Maybe it will actually mean removing some of the current entry points, and requiring developers to specifically configure them (e.g., pudge, or publish). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ben at groovie.org Thu Sep 22 04:50:34 2005 From: ben at groovie.org (Ben Bangert) Date: Wed, 21 Sep 2005 19:50:34 -0700 Subject: [Distutils] #egg links off a find-links page Message-ID: <6B4880C9-ECE6-4799-BE0A-D6323F919DC3@groovie.org> I'm trying to make a bunch of subversion directories available on a page for use with find-links to provide one place for it to fetch all the packages I require. Here's the page where I put them all: http://pylons.groovie.org/ Running the ez_setup script like this: python ez_setup.py -f http://pylons.groovie.org/ Pylons It properly gets the Pylons trunk, checks the version, etc. Then it searches for dependencies, and searches PyPI for PasteDeploy even though I listed it with #egg=PasteDeploy on the page. Of course it doesn't find the one I specifically wanted in PyPI (a dev version), so it then fails rather than noticing the egg link. Any ideas? Thanks, Ben From pje at telecommunity.com Thu Sep 22 06:48:01 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 00:48:01 -0400 Subject: [Distutils] #egg links off a find-links page In-Reply-To: <6B4880C9-ECE6-4799-BE0A-D6323F919DC3@groovie.org> Message-ID: <5.1.1.6.0.20050922004523.01b8a4b8@mail.telecommunity.com> At 07:50 PM 9/21/2005 -0700, Ben Bangert wrote: >I'm trying to make a bunch of subversion directories available on a >page for use with find-links to provide one place for it to fetch all >the packages I require. Here's the page where I put them all: >http://pylons.groovie.org/ > >Running the ez_setup script like this: >python ez_setup.py -f http://pylons.groovie.org/ Pylons > >It properly gets the Pylons trunk, checks the version, etc. >Then it searches for dependencies, and searches PyPI for PasteDeploy >even though I listed it with #egg=PasteDeploy on the page. Of course >it doesn't find the one I specifically wanted in PyPI (a dev >version), so it then fails rather than noticing the egg link. > >Any ideas? $ easy_install -vvnf http://pylons.groovie.org/project Routes Reading http://pylons.groovie.org/project Found link: http://pylons.groovie.org/pylons/trunk#egg=Pylons Found link: http://pylons.groovie.org/files/Myghty-0.98c-py2.4.egg Found link: http://svn.pythonpaste.org/Paste/branches/mainline-refactor#egg=Paste Found link: http://pylons.groovie.org/files/WSGIUtils-0.6-py2.4.egg It looks like these are the only links it's spotting, due to the #egg pattern it's looking for: EGG_FRAGMENT = re.compile('^egg=(\\w+(-\\w+)?)$') This is broken, because the part after the '-' should allow '.', and it doesn't. I'll have to fix that in 0.3. In the meantime, you'll have to avoid using '.' in version numbers, or patch setuptools/package_index.py. From dangoor at gmail.com Thu Sep 22 16:10:16 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Thu, 22 Sep 2005 10:10:16 -0400 Subject: [Distutils] buildutils In-Reply-To: <4331EFB0.5040306@colorstudy.com> References: <4331EFB0.5040306@colorstudy.com> Message-ID: <3f085ecd050922071059019d89@mail.gmail.com> On 9/21/05, Ian Bicking wrote: > I thought I'd note that I've been doing a little bit of work on Ryan > Tomayko's buildutils lately (http://buildutils.lesscode.org/). It now > has distutils.command entry points, so any setuptools-based package will > get the new commands. And I've moved the svntag command from > PasteDeploy to buildutils. Nice! I'll have to update. > I guess next up I'm going to re-look at how to enable some of these > commands conditionally, along the lines discussed in this thread: > http://mail.python.org/pipermail/distutils-sig/2005-August/005011.html > > Maybe it will actually mean removing some of the current entry points, > and requiring developers to specifically configure them (e.g., pudge, or > publish). I have thought that the pudge command belongs in pudge itself and not buildutils. (Though I understand why it's in buildutils, since it predates the ability to define commands via entrypoints.) Kevin -- Kevin Dangoor Author of the Zesty News RSS newsreader email: kid at blazingthings.com company: http://www.BlazingThings.com blog: http://www.BlueSkyOnMars.com From wl at flexis.de Thu Sep 22 16:27:06 2005 From: wl at flexis.de (Wolfgang Langner) Date: Thu, 22 Sep 2005 16:27:06 +0200 Subject: [Distutils] api documentation wrong Message-ID: Hello, with my experiments I found an API documentation bug on: http://peak.telecommunity.com/DevCenter/PythonEggs install_on(path=sys.path) is obsolete now this is activate Version 0.6a2 of setuptools bye by Wolfgang From ianb at colorstudy.com Thu Sep 22 17:19:39 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 22 Sep 2005 10:19:39 -0500 Subject: [Distutils] setuptools: Inconsistent quoting Message-ID: <4332CB8B.4070803@colorstudy.com> I have a setup file, that looks roughly like: setup(... extras_require={ 'WSGI-Utils': ['WSGI-Utils'], }, entry_points=""" [paste.server_runner] wsgiutils=paste.script.wsgiutils_server:run_server [WSGI-Utils] """) When I try to use the wsgiutils entry point I get: pkg_resources.UnknownExtra: PasteScript 0.1dev-r3227 has no such extra feature 'wsgi_utils' If I rename it to WSGIUtils is all works. So apparently quoting is being applied differently depending on context. You can get the full setup.py script with: svn cat -r 3231 http://svn.pythonpaste.org/Paste/Script/trunk/setup.py -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Thu Sep 22 17:21:24 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 22 Sep 2005 10:21:24 -0500 Subject: [Distutils] buildutils In-Reply-To: <3f085ecd050922071059019d89@mail.gmail.com> References: <4331EFB0.5040306@colorstudy.com> <3f085ecd050922071059019d89@mail.gmail.com> Message-ID: <4332CBF4.5000106@colorstudy.com> Kevin Dangoor wrote: >>I guess next up I'm going to re-look at how to enable some of these >>commands conditionally, along the lines discussed in this thread: >>http://mail.python.org/pipermail/distutils-sig/2005-August/005011.html >> >>Maybe it will actually mean removing some of the current entry points, >>and requiring developers to specifically configure them (e.g., pudge, or >>publish). > > > I have thought that the pudge command belongs in pudge itself and not > buildutils. (Though I understand why it's in buildutils, since it > predates the ability to define commands via entrypoints.) But does the presence of pudge on the system mean anything more or less than the presence of buildutils? If pudge *isn't* on the system, I'd like the command to still show up for appropriate projects, but with a missing dependency. And just because pudge is on the system, doesn't mean it should be applied to every project. Though, it seems to apply fairly easily to projects that aren't expecting it, so that might not be accurate in this case. And something like flakes -- which is also generally applicable -- need not be specifically enabled. But pytest probably should be, and so forth. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Thu Sep 22 17:52:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 11:52:46 -0400 Subject: [Distutils] api documentation wrong In-Reply-To: Message-ID: <5.1.1.6.0.20050922115142.01b78940@mail.telecommunity.com> At 04:27 PM 9/22/2005 +0200, Wolfgang Langner wrote: >Hello, > >with my experiments I found an API documentation bug on: > >http://peak.telecommunity.com/DevCenter/PythonEggs > >install_on(path=sys.path) is obsolete >now this is activate > >Version 0.6a2 of setuptools Please see these pages instead: http://peak.telecommunity.com/DevCenter/PkgResources http://peak.telecommunity.com/DevCenter/setuptoools http://peak.telecommunity.com/DevCenter/EasyInstall I haven't had a chance to delete the outdated stuff from the PythonEggs page yet. From richardjones at optushome.com.au Fri Sep 23 02:25:51 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Fri, 23 Sep 2005 10:25:51 +1000 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <430A18D3.5040700@colorstudy.com> References: <430A18D3.5040700@colorstudy.com> Message-ID: <200509231025.52017.richardjones@optushome.com.au> On Tue, 23 Aug 2005 04:26 am, Ian Bicking wrote: > I don't know which side this belongs to, but I had a problem when I > tried to create a package with a "-" in it ("Paste-Deploy"). setup.py > register worked fine, and created a "Paste-Deploy" project; however, > when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and > PyPI wouldn't accept it, I believe because it thought it belonged to the > (nonexistant) Paste_Deploy project. The only restrictions PyPI places on filenames for uploads are: # check for valid filenames filename = content.filename if not safe_filenames.match(filename): raise FormError, 'invalid distribution file' # check for dodgy filenames if '/' in filename or '\\' in filename: raise FormError, 'invalid distribution file' # check the file for valid contents based on the type if not verify_filetype.is_distutils_file(content, filename, filetype): raise FormError, 'invalid distribution file' Where: safe_filenames = re.compile(r'.+?\.(exe|tar\.gz|bz2|rpm|deb|zip|tgz|egg)$', re.I) and "is_distutils_file" just looks at the extension and pokes into the file based on the extension to make sure that an ".exe" uplood looks kinda like an installer, and ".zip" and ".egg" uploads look kinda like ZIP files of distutils origin. No checks are made that a filename matches a package name. So given the metadata: setup( name="To-Do List", version="1.23 alpha!", ... ) as long as that *name* (and version) is passed unchanged to PyPI, a file named "frozzleplop-1.2.3.zip" could be attached to the "To-Do List" package. I can only assume that setuptools is mutating the name/version in order to generate a safe filename, but then passing the mutated name/version to PyPI as the release identifier. I think it's an unacceptable change to make to PyPI to accept the mutated name/version, as the name/version represents the unique identifier in the database for a package. Unique identifier collisions are possible when you start mangling them, and I'd really prefer to avoid such things. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/7f0886cb/attachment.pgp From pje at telecommunity.com Fri Sep 23 05:32:17 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 23:32:17 -0400 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <200509231025.52017.richardjones@optushome.com.au> References: <430A18D3.5040700@colorstudy.com> <430A18D3.5040700@colorstudy.com> Message-ID: <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> At 10:25 AM 9/23/2005 +1000, Richard Jones wrote: >I can only assume that setuptools is mutating the name/version in order to >generate a safe filename, but then passing the mutated name/version to PyPI >as the release identifier. I haven't attempted to reproduce Ian's problem, but I don't believe I'm doing this, at least not in the upload command. I'll have to check. > I think it's an unacceptable change to make to >PyPI to accept the mutated name/version, as the name/version represents the >unique identifier in the database for a package. Unique identifier collisions >are possible when you start mangling them, and I'd really prefer to avoid >such things. I'd like to encourage moving towards mangling the *keys* themselves, in order to be robust in the face of typos. I think allowing random punctuation and upper/lower case to distinguish projects (e.g. allowing SQLObject, sqlobject, and SQLobject to be different packages) is asking for trouble on the human side of things, entirely aside from allowing them in filenames, dealing with case-insensitive filesystems, and so on. Having a nice human readable name for the web page, PKG-INFO, and description are one thing, but having it used for filenames, URLs, and database keys is quite another. From richardjones at optushome.com.au Fri Sep 23 05:37:43 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Fri, 23 Sep 2005 13:37:43 +1000 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> References: <430A18D3.5040700@colorstudy.com> <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> Message-ID: <200509231337.46971.richardjones@optushome.com.au> On Fri, 23 Sep 2005 01:32 pm, Phillip J. Eby wrote: > I'd like to encourage moving towards mangling the *keys* themselves, in > order to be robust in the face of typos. ?I think allowing random > punctuation and upper/lower case to distinguish projects (e.g. allowing > SQLObject, sqlobject, and SQLobject to be different packages) is asking for > trouble on the human side of things, entirely aside from allowing them in > filenames, dealing with case-insensitive filesystems, and so on. ?Having a > nice human readable name for the web page, PKG-INFO, and description are > one thing, but having it used for filenames, URLs, and database keys is > quite another. I believe what you're proposing would require changing Python itself so it enforces restrictions on package names (ie. all lower-case, very limited punctuation, no whitespace). I think that's a good idea, but I also think at this point that the cat's out of the bag :( Happy to hear contradictory views and be shouted down. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/88f99793/attachment.pgp From p.f.moore at gmail.com Fri Sep 23 12:17:34 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 23 Sep 2005 11:17:34 +0100 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> Message-ID: <79990c6b05092303171a27cdd3@mail.gmail.com> On 9/17/05, Phillip J. Eby wrote: > However, after reflection, I think now that -m probably only really makes > sense for stdlib modules, since projects using setuptools can now get all > the benefits of -m without any of the drawbacks, without even writing any > __name__=='__main__' code. One thing you get with -m is that the module doesn't have to be on PATH. Where does setuptools install its wrapper executables? (The usual \Python24\Scripts directory isn't added to PATH by the installer). I know, I'm nitpicking. Sorry :-) Paul. From p.f.moore at gmail.com Fri Sep 23 12:27:57 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 23 Sep 2005 11:27:57 +0100 Subject: [Distutils] Working around .pth/site problems In-Reply-To: <5.1.1.6.0.20050919023514.01b7da88@mail.telecommunity.com> References: <5.1.1.6.0.20050919023514.01b7da88@mail.telecommunity.com> Message-ID: <79990c6b0509230327d79a32@mail.gmail.com> On 9/19/05, Phillip J. Eby wrote: > In essence, the problem is that Python adds all .pth-specified paths to the > *end* of sys.path, meaning that system-installed packages in the stdlib or > site-packages take precedence over anything else you can install, except > for directories specified in PYTHONPATH or that are in the startup script's > directory. I believe that this is a deliberate policy. I seem to recall Guido saying a long time ago that he doesn't want people to be able to override the stdlib easily, as it makes "core" python mutable. It's a long time ago, and I may have misremembered, or indeed Guido may have changed his mind since. I'd suggest raining this issue on python-dev as I don't believe that Guido reads distutils-sig. There may be scope for an "official" solution in 2.5, which can be supplemented by backward-compatibility hacks for earlier versions. Paul. PS Personally, I don't have any issue with what you propose - I use Python on Windows only at the moment, so I don't have the type of problems you describe. From theller at python.net Fri Sep 23 15:07:54 2005 From: theller at python.net (Thomas Heller) Date: Fri, 23 Sep 2005 15:07:54 +0200 Subject: [Distutils] Improving distutils' script and GUI app handling References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> Message-ID: Paul Moore writes: > On 9/17/05, Phillip J. Eby wrote: >> However, after reflection, I think now that -m probably only really makes >> sense for stdlib modules, since projects using setuptools can now get all >> the benefits of -m without any of the drawbacks, without even writing any >> __name__=='__main__' code. > > One thing you get with -m is that the module doesn't have to be on > PATH. Where does setuptools install its wrapper executables? (The > usual \Python24\Scripts directory isn't added to PATH by the > installer). > > I know, I'm nitpicking. Sorry :-) No, you're not. IMO. setuptools installs them in PythonXY\Scripts. BTW: Shouldn't 'setup.py develop --uninstall' remove them again? It doesn't. And yet another question: How should my setup-script start? Is this the correct way: """ from ez_setup import use_setuptools use_setuptools() from distutils.core import setup """ I'm considering to finally use setuptools in the comtypes setup script. Thomas From pje at telecommunity.com Fri Sep 23 17:40:57 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 11:40:57 -0400 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <200509231337.46971.richardjones@optushome.com.au> References: <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> <430A18D3.5040700@colorstudy.com> <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> At 01:37 PM 9/23/2005 +1000, Richard Jones wrote: >On Fri, 23 Sep 2005 01:32 pm, Phillip J. Eby wrote: > > I'd like to encourage moving towards mangling the *keys* themselves, in > > order to be robust in the face of typos. I think allowing random > > punctuation and upper/lower case to distinguish projects (e.g. allowing > > SQLObject, sqlobject, and SQLobject to be different packages) is asking for > > trouble on the human side of things, entirely aside from allowing them in > > filenames, dealing with case-insensitive filesystems, and so on. Having a > > nice human readable name for the web page, PKG-INFO, and description are > > one thing, but having it used for filenames, URLs, and database keys is > > quite another. > >I believe what you're proposing would require changing Python itself so it >enforces restrictions on package names (ie. all lower-case, very limited >punctuation, no whitespace). I think that's a good idea, but I also think at >this point that the cat's out of the bag :( Python doesn't let you use spaces and arbitrary punctuation in package names, so perhaps I've misunderstood you. Also, just in case you've misunderstood me, I'm referring above to *project* names, not package names. For example, PEAK has its project name registered as PEAK on PyPI, but its Python package name (that you actually import) is 'peak'. I'm referring above only to project names, not package names. From pje at telecommunity.com Fri Sep 23 17:46:28 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 11:46:28 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> Message-ID: <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> At 03:07 PM 9/23/2005 +0200, Thomas Heller wrote: >Paul Moore writes: > > > On 9/17/05, Phillip J. Eby wrote: > >> However, after reflection, I think now that -m probably only really makes > >> sense for stdlib modules, since projects using setuptools can now get all > >> the benefits of -m without any of the drawbacks, without even writing any > >> __name__=='__main__' code. > > > > One thing you get with -m is that the module doesn't have to be on > > PATH. Where does setuptools install its wrapper executables? (The > > usual \Python24\Scripts directory isn't added to PATH by the > > installer). > > > > I know, I'm nitpicking. Sorry :-) > >No, you're not. IMO. setuptools installs them in PythonXY\Scripts. Unless you set --install-scripts somewhere else, which you can do in a configuration file. To the greatest extent possible, I'm trying to have setuptools minimize surprises with respect to installation locations, by conforming to the active distutils configuration. >BTW: Shouldn't 'setup.py develop --uninstall' remove them again? It >doesn't. setuptools has hardly any uninstallation capabilities as yet. >And yet another question: How should my setup-script start? Is this the >correct way: > >""" >from ez_setup import use_setuptools >use_setuptools() > >from distutils.core import setup It's a way that works. I personally use 'from setuptools import setup' and whatever else I need to import, but whatever you prefer is fine. setuptools patches itself into the distutils core. It used to not do that, but then I realized that the way py2exe and other distutils extensions patch themselves in, you would lose setuptools functionality unless I patched myself in also. Hopefully in future everybody will use setuptools' extension functionality so that patching will be unnecessary. From Michael.Delaney at CatalinaMarketing.com Fri Sep 23 19:19:42 2005 From: Michael.Delaney at CatalinaMarketing.com (Michael Delaney) Date: Fri, 23 Sep 2005 13:19:42 -0400 Subject: [Distutils] Distributing Python Modules - Error using foo.py in section 2.1 Message-ID: <32DBAD32B9E0044881BDBD47F5E099A4E0E353@cmcstpexsan.catmktg.com> Greg Ward, I am reading the documentation you wrote titled "Distributing Python Modules". In section 2.1 (A Simple Example), I made a simple script named foo.py but I am not able to get it running following the steps you provided. The error I receive is listed below and I have attached the files. I am executing this in a console prompt using Windows XP as the Operating System. Could you please tell me what I am doing incorrectly? 2.1 A Simple Example foo.py # A setup script. # from distutils.core import setup setup(name="foo", version="1.0", py_modules=["foo"]) If all you want to do is distribute a module called foo, contained in a file foo.py, then your setup script can be as simple as this: from distutils.core import setup setup(name="foo", version="1.0", py_modules=["foo"]) To create a source distribution for this module, you would create a setup script, Setup.py, containing the above code, and run: python setup.py sdist The following error is displayed: C:\Program Files\Python24>python setup.py Traceback (most recent call last): File "setup.py", line 7, in ? py_modules=["foo"]) File "C:\Program Files\Python24\distutils\core.py", line 101, in setup _setup_distribution=dist=Klass(attrs) File "C:\Program Files\Python24\distutils\dist.py", line 130, in __init__settattr(self, method_name, getattr(self.metadata, method_name)) Attribute Error: Distribution Metadata instance has no attribute 'get___doc___' <> <> Thank you very much for your time. Michael Delaney Application Programmer Checkout Direct(r) 200 Carillon Parkway St. Petersburg, FL 33716 Ph: 727-579-5144 Michael.Delaney at CatalinaMarketing.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050923/39be98ff/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: foo.py Type: application/octet-stream Size: 46 bytes Desc: foo.py Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/39be98ff/foo.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: setup.py Type: application/octet-stream Size: 124 bytes Desc: setup.py Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/39be98ff/setup.obj From Michael.Delaney at CatalinaMarketing.com Fri Sep 23 20:17:52 2005 From: Michael.Delaney at CatalinaMarketing.com (Michael Delaney) Date: Fri, 23 Sep 2005 14:17:52 -0400 Subject: [Distutils] FW: Distributing Python Modules - Error using foo.py in section 2.1 - Corrected Message-ID: <32DBAD32B9E0044881BDBD47F5E099A4E0E356@cmcstpexsan.catmktg.com> Greg We do not have admin rights for our workstations here. After I was able to temporarily obtain admin rights and add python to the Path, your example works. Thanks. > -----Original Message----- > From: Michael Delaney > Sent: Friday, September 23, 2005 1:20 PM > To: 'distutils-sig at python.org' > Subject: Distributing Python Modules - Error using foo.py in > section 2.1 > > Greg Ward, > > I am reading the documentation you wrote titled "Distributing > Python Modules". In section 2.1 (A Simple Example), I made a simple > script named foo.py but I am not able to get it running following the > steps you provided. The error I receive is listed below and I have > attached the files. I am executing this in a console prompt using > Windows XP as the Operating System. Could you please tell me what I > am doing incorrectly? > > > 2.1 A Simple Example > > foo.py > > # A setup script. > # > > from distutils.core import setup > setup(name="foo", > version="1.0", > py_modules=["foo"]) > > > If all you want to do is distribute a module called foo, contained in > a file foo.py, > then your setup script can be as simple as this: > > from distutils.core import setup > setup(name="foo", > version="1.0", > py_modules=["foo"]) > > > To create a source distribution for this module, you would create a > setup script, > Setup.py, containing the above code, and run: > > python setup.py sdist > > > > The following error is displayed: > > C:\Program Files\Python24>python setup.py > > Traceback (most recent call last): > File "setup.py", line 7, in ? > py_modules=["foo"]) > File "C:\Program Files\Python24\distutils\core.py", line 101, > in setup _setup_distribution=dist=Klass(attrs) > File "C:\Program Files\Python24\distutils\dist.py", line 130, > in __init__settattr(self, method_name, getattr(self.metadata, > method_name)) > Attribute Error: Distribution Metadata instance has no attribute > 'get___doc___' > > > > <> > > <> > > > Thank you very much for your time. > > Michael Delaney > Application Programmer > Checkout Direct(r) > 200 Carillon Parkway > St. Petersburg, FL 33716 > Ph: 727-579-5144 > Michael.Delaney at CatalinaMarketing.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050923/dc307887/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: foo.py Type: application/octet-stream Size: 46 bytes Desc: foo.py Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/dc307887/foo-0001.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: setup.py Type: application/octet-stream Size: 124 bytes Desc: setup.py Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050923/dc307887/setup-0001.obj From theller at python.net Fri Sep 23 21:38:34 2005 From: theller at python.net (Thomas Heller) Date: Fri, 23 Sep 2005 21:38:34 +0200 Subject: [Distutils] easy_install question Message-ID: - When I run 'easy_install ctypes', the behaviour depends on the current directory. If I run this when the current dir contains a 'ctypes' subdirectory, it doesn't search pypi^H^H^H^HCheeseShop for this name, instead it seems to attempt to install from source in that directory. Is that intended? Can I override this behaviour? - When I install a package which requires another package is there a way to specify a download url for the required package? Thomas From pje at telecommunity.com Fri Sep 23 22:54:45 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 16:54:45 -0400 Subject: [Distutils] easy_install question In-Reply-To: Message-ID: <5.1.1.6.0.20050923164945.02ae0af8@mail.telecommunity.com> At 09:38 PM 9/23/2005 +0200, Thomas Heller wrote: >- When I run 'easy_install ctypes', the behaviour depends on the current >directory. If I run this when the current dir contains a 'ctypes' >subdirectory, it doesn't search pypi^H^H^H^HCheeseShop for this name, >instead it seems to attempt to install from source in that directory. >Is that intended? Yes. EasyInstall's arguments may be a URL, an existing local filename, or a "requirement" string, and the precedence is in that order. (i.e., it's checked to see if it's a URL, then if it's a filename, and finally parsing as a requirement string is done.) > Can I override this behaviour? Add a version specifier, e.g. "ctypes==4.0", or use an absolute URL. >- When I install a package which requires another package is there a way >to specify a download url for the required package? The --find-links option can include download links as well as URLs of pages containing download links. If you pass a URL that ends with a recognizable distutils filename, EasyInstall will use it if it matches the version requirement(s). A question: are you embedding EasyInstall in something else, or are these questions about direct use? From faassen at infrae.com Fri Sep 23 23:57:11 2005 From: faassen at infrae.com (Martijn Faassen) Date: Fri, 23 Sep 2005 23:57:11 +0200 Subject: [Distutils] python eggs and lxml Message-ID: <43347A37.1060209@infrae.com> Hi there, I've been looking at Python eggs, easy install and setuptools with a lot of interest -- very impressive work. I've been thinking of packaging lxml with it, and, on the larger scale, look into packaging Zope 3 with it. Concerning lxml I run into a few questions however. lxml depends on large external C libraries (libxml2 and libxslt). a) Is there a way to require versions of C libraries to be available in the Python eggs dependencies? I can't seem to find a reference to this scenario, but perhaps I didn't look carefully enough. The goal here would be to give users trying to install lxml (or something that depends on lxml) useful feedback about what in their system they're missing (or have the wrong version of). b) Going further, it'd be nice for some scenarios to actually be able to include private versions of libxml2/libxslt in a Python eggs. This is especially interesting on Windows deployments (where you'd include a binary of these libraries). Has something like this considered? I saw references to Pyrex support, but in lxml's case, the Pyrex code depends on a large underlying library. c) It's also interesting though for deployment on linux. It'd be nice to be able to include the source versions of specific versions of libxml2 and libxslt with lxml and to be able to build/install them such that they are only used for lxml. This way the system libraries (which may be out of date or have otherwise a wrong version) would not be in play and wouldn't be affected. If something like this were arranged, it'd be much easier to make lxml a requirement of a large package like for instance Zope 3 (which is being considered). I realize that any or all of these might be out of scope for easy install -- in the Linux case, it might be deferred to a Linux package management system, for instance. Still, I imagine the case where a Python library has a dependency on a potentially large non-Python codebase could be fairly common, and it'd be nice if such libraries could be "first class" easy_install citizens so that other Python libraries can safely depend on them. What are people's thoughts were about supporting such scenarios? Regards, Martijn From pje at telecommunity.com Sat Sep 24 00:12:05 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 18:12:05 -0400 Subject: [Distutils] python eggs and lxml In-Reply-To: <43347A37.1060209@infrae.com> Message-ID: <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> At 11:57 PM 9/23/2005 +0200, Martijn Faassen wrote: >I've been looking at Python eggs, easy install and setuptools with a lot >of interest -- very impressive work. I've been thinking of packaging >lxml with it, and, on the larger scale, look into packaging Zope 3 with it. > >Concerning lxml I run into a few questions however. > >lxml depends on large external C libraries (libxml2 and libxslt). > >a) Is there a way to require versions of C libraries to be available in >the Python eggs dependencies? I can't seem to find a reference to this >scenario, but perhaps I didn't look carefully enough. The goal here >would be to give users trying to install lxml (or something that depends >on lxml) useful feedback about what in their system they're missing (or >have the wrong version of). Your options here are the same as with any distutils package, which is to say you have to figure it out yourself. ;) You can add code to look for the libraries, embed your own source, etc. >b) Going further, it'd be nice for some scenarios to actually be able to >include private versions of libxml2/libxslt in a Python eggs. This is >especially interesting on Windows deployments (where you'd include a >binary of these libraries). Has something like this considered? I saw >references to Pyrex support, but in lxml's case, the Pyrex code depends >on a large underlying library. You can certainly do that; just list the appropriate .c files in your Extension. For PEAK on Python 2.3, I include an expat wrapping that adds the Python 2.4 pyexpat features this way, using something like: Extension("peak.util.pyexpat", [ "src/peak/util/pyexpat.c", "src/expat/xmlparse.c", "src/expat/xmltok.c", "src/expat/xmlrole.c", ], include_dirs=["src/expat"], define_macros=[('XML_STATIC',1),('HAVE_MEMMOVE',1)] # XXX ), >c) It's also interesting though for deployment on linux. It'd be nice to >be able to include the source versions of specific versions of libxml2 >and libxslt with lxml and to be able to build/install them such that >they are only used for lxml. This way the system libraries (which may be >out of date or have otherwise a wrong version) would not be in play and >wouldn't be affected. Yeah, just bake it in as shown above. >If something like this were arranged, it'd be much easier to make lxml a >requirement of a large package like for instance Zope 3 (which is being >considered). > >I realize that any or all of these might be out of scope for easy >install -- in the Linux case, it might be deferred to a Linux package >management system, for instance. Still, I imagine the case where a >Python library has a dependency on a potentially large non-Python >codebase could be fairly common, and it'd be nice if such libraries >could be "first class" easy_install citizens so that other Python >libraries can safely depend on them. What are people's thoughts were >about supporting such scenarios? Not all libraries can be bundled by source, of course. Sometimes you really need to use whatever the "system version" is, for one reason or another. Database clients, for example, are something you really really want to use the local version for. I'm thinking that the distutils could really use some sort of library-finding capabilities for that stuff, assuming they don't already have some I just haven't found yet. From trentm at ActiveState.com Sat Sep 24 00:45:20 2005 From: trentm at ActiveState.com (Trent Mick) Date: Fri, 23 Sep 2005 15:45:20 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> References: <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> Message-ID: <20050923224520.GA30074@ActiveState.com> > > > Where does setuptools install its wrapper executables? (The > > > usual \Python24\Scripts directory isn't added to PATH by the > > > installer). > > > >setuptools installs them in PythonXY\Scripts. > > Unless you set --install-scripts somewhere else, which you can do in a > configuration file. To the greatest extent possible, I'm trying to have > setuptools minimize surprises with respect to installation locations, by > conforming to the active distutils configuration. Does anybody else think that installing to PythonXY\Scripts (instead of to PythonXY) is broken? With Perl and Ruby, for example, scripts from a 3rd party package will be installed next to the main interpreter binary (i.e. on the PATH) on all platforms. Would having setuptools (and changing distutils) to install scripts next to python.exe wreak unwarranted havoc? Cheers, Trent -- Trent Mick TrentM at ActiveState.com From richardjones at optushome.com.au Sat Sep 24 00:45:36 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Sat, 24 Sep 2005 08:45:36 +1000 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> References: <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> Message-ID: <200509240845.41561.richardjones@optushome.com.au> On Sat, 24 Sep 2005 01:40 am, Phillip J. Eby wrote: > At 01:37 PM 9/23/2005 +1000, Richard Jones wrote: > >I believe what you're proposing would require changing Python itself so it > >enforces restrictions on package names (ie. all lower-case, very limited > >punctuation, no whitespace). I think that's a good idea, but I also think > > at this point that the cat's out of the bag :( > > Python doesn't let you use spaces and arbitrary punctuation in package > names, so perhaps I've misunderstood you. > > Also, just in case you've misunderstood me, I'm referring above to > *project* names, not package names. For example, PEAK has its project name > registered as PEAK on PyPI, but its Python package name (that you actually > import) is 'peak'. I'm referring above only to project names, not package > names. By "package name" I mean the distutils package name. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050924/7a409c39/attachment.pgp From pje at telecommunity.com Sat Sep 24 02:10:30 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 20:10:30 -0400 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <200509240845.41561.richardjones@optushome.com.au> References: <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> <5.1.1.6.0.20050922232636.01b5fc40@mail.telecommunity.com> <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050923201002.01b649c8@mail.telecommunity.com> At 08:45 AM 9/24/2005 +1000, Richard Jones wrote: >On Sat, 24 Sep 2005 01:40 am, Phillip J. Eby wrote: > > At 01:37 PM 9/23/2005 +1000, Richard Jones wrote: > > >I believe what you're proposing would require changing Python itself so it > > >enforces restrictions on package names (ie. all lower-case, very limited > > >punctuation, no whitespace). I think that's a good idea, but I also think > > > at this point that the cat's out of the bag :( > > > > Python doesn't let you use spaces and arbitrary punctuation in package > > names, so perhaps I've misunderstood you. > > > > Also, just in case you've misunderstood me, I'm referring above to > > *project* names, not package names. For example, PEAK has its project name > > registered as PEAK on PyPI, but its Python package name (that you actually > > import) is 'peak'. I'm referring above only to project names, not package > > names. > >By "package name" I mean the distutils package name. Okay, so I still don't understand why this requires "changing Python itself". Could you explain? From pje at telecommunity.com Sat Sep 24 02:13:34 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Sep 2005 20:13:34 -0400 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <20050923224520.GA30074@ActiveState.com> References: <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050923201036.02e5f8a8@mail.telecommunity.com> At 03:45 PM 9/23/2005 -0700, Trent Mick wrote: > > > > Where does setuptools install its wrapper executables? (The > > > > usual \Python24\Scripts directory isn't added to PATH by the > > > > installer). > > > > > >setuptools installs them in PythonXY\Scripts. > > > > Unless you set --install-scripts somewhere else, which you can do in a > > configuration file. To the greatest extent possible, I'm trying to have > > setuptools minimize surprises with respect to installation locations, by > > conforming to the active distutils configuration. > >Does anybody else think that installing to PythonXY\Scripts (instead of >to PythonXY) is broken? Windows doesn't really have any obvious standard place to put this sort of thing, unless you consider the Windows\Command directory. > With Perl and Ruby, for example, scripts from a >3rd party package will be installed next to the main interpreter binary >(i.e. on the PATH) on all platforms. Would having setuptools (and >changing distutils) to install scripts next to python.exe wreak >unwarranted havoc? Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless you put it there yourself. Second, the python.exe directory is on sys.path, so it would turn your scripts into modules, conflicting with any same-named modules. So, yes, I'm afraid "havoc" is the correct word, though it would certainly be nice if someone would step up and prove me wrong here. From richardjones at optushome.com.au Sat Sep 24 04:45:21 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Sat, 24 Sep 2005 12:45:21 +1000 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <5.1.1.6.0.20050923201002.01b649c8@mail.telecommunity.com> References: <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> <5.1.1.6.0.20050923201002.01b649c8@mail.telecommunity.com> Message-ID: <200509241245.25802.richardjones@optushome.com.au> On Sat, 24 Sep 2005 10:10 am, Phillip J. Eby wrote: > Okay, so I still don't understand why this requires "changing Python > itself". Could you explain? Distutils metadata capture is implemented in the Python core. We would want to implement any name restrictions there, surely? Otherwise people only get an error when attempting to use setuptools or register with PyPI, which would be just annoying. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050924/0025fbd9/attachment-0001.pgp From fdrake at gmail.com Sat Sep 24 05:40:21 2005 From: fdrake at gmail.com (Fred Drake) Date: Fri, 23 Sep 2005 23:40:21 -0400 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <200509241245.25802.richardjones@optushome.com.au> References: <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> <5.1.1.6.0.20050923201002.01b649c8@mail.telecommunity.com> <200509241245.25802.richardjones@optushome.com.au> Message-ID: <9cee7ab8050923204020fb2eef@mail.gmail.com> On 9/23/05, Richard Jones wrote: > Distutils metadata capture is implemented in the Python core. We would want to > implement any name restrictions there, surely? Otherwise people only get an > error when attempting to use setuptools or register with PyPI, which would be > just annoying. The use of distutils should not imply the use of PyPI. Perhaps we'd want distutils to issue a warning when building a distribution if the naming conventions weren't acceptable, but that's the most we'd want. That should be something that could easily be turned off for a site or an individual. -Fred -- Fred L. Drake, Jr. Zope Corporation From pje at telecommunity.com Sat Sep 24 07:45:01 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Sep 2005 01:45:01 -0400 Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...? In-Reply-To: <9cee7ab8050923204020fb2eef@mail.gmail.com> References: <200509241245.25802.richardjones@optushome.com.au> <5.1.1.6.0.20050923113747.01b77950@mail.telecommunity.com> <5.1.1.6.0.20050923201002.01b649c8@mail.telecommunity.com> <200509241245.25802.richardjones@optushome.com.au> Message-ID: <5.1.1.6.0.20050924012942.01b65c50@mail.telecommunity.com> At 11:40 PM 9/23/2005 -0400, Fred Drake wrote: >On 9/23/05, Richard Jones wrote: > > Distutils metadata capture is implemented in the Python core. We would > want to > > implement any name restrictions there, surely? Otherwise people only get an > > error when attempting to use setuptools or register with PyPI, which > would be > > just annoying. > >The use of distutils should not imply the use of PyPI. Perhaps we'd >want distutils to issue a warning when building a distribution if the >naming conventions weren't acceptable, but that's the most we'd want. >That should be something that could easily be turned off for a site or >an individual. Not only that, but I'm not suggesting we ban those characters from names. I'm suggesting merely that we strip them in a uniform way. The error message would be "somebody already has a project with a name that's too similar to yours", not "you have unacceptable characters in your project name". :) I'm suggesting, in other words, that "Foo*Bar" and "Foo!Bar" simply not be considered unique project names, not that whichever project registers the name first can't use it with funky punctuation in PKG-INFO and display it on their PyPI page that way. (I would also suggest that we clarify the rules for determining project name uniqueness and recommend people follow them for simplicity's sake, of course.) I'm also suggesting that if somebody goes to the URL "/pypi/foo--bar", it would still pull up the "Foo*Bar" project if that's the one that's registered, because canonicalizing 'foo--bar' should yield the same unique key as canonicalizing 'Foo*Bar'. (This is particularly nice for EasyInstall users, since it wouldn't need to fall back to pulling down the entire index to do a case-insensitive search when they don't match someone's CreativeCAPS in a project name.) In other words, all user inputs (URL or otherwise) should be normalized for key storage and lookup, distinct from the human-readable name of the package. (Setuptools implements this for eggs by having distinct "project_name" and "key" attributes.) This approach has a few important features: 1. It can be implemented without renaming existing packages, unless there are actual conflicts in PyPI today 2. It can be implemented without any need for co-operation from package authors, because it's strictly a PyPI-side change. 3. It allows authors to fully express their creativity in naming 4. It allows end-users to ignore the authors' creativity :) The principal downside, of course, is that it's probably not a minor change to the PyPI code base, with respect to the "two names" issue, or with respect to how lookups are done. Which is why I haven't been hounding Richard to do it. Well, maybe just a little. ;) From faassen at infrae.com Sat Sep 24 11:13:39 2005 From: faassen at infrae.com (Martijn Faassen) Date: Sat, 24 Sep 2005 11:13:39 +0200 Subject: [Distutils] python eggs and lxml In-Reply-To: <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> References: <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> Message-ID: <433518C3.7040104@infrae.com> Phillip J. Eby wrote: > At 11:57 PM 9/23/2005 +0200, Martijn Faassen wrote: > >> I've been looking at Python eggs, easy install and setuptools with a lot >> of interest -- very impressive work. I've been thinking of packaging >> lxml with it, and, on the larger scale, look into packaging Zope 3 >> with it. >> >> Concerning lxml I run into a few questions however. >> >> lxml depends on large external C libraries (libxml2 and libxslt). >> >> a) Is there a way to require versions of C libraries to be available in >> the Python eggs dependencies? I can't seem to find a reference to this >> scenario, but perhaps I didn't look carefully enough. The goal here >> would be to give users trying to install lxml (or something that depends >> on lxml) useful feedback about what in their system they're missing (or >> have the wrong version of). > > Your options here are the same as with any distutils package, which is > to say you have to figure it out yourself. ;) You can add code to look > for the libraries, embed your own source, etc. Right. I was hoping I didn't need to dive into the internals of distutils of course, but it's no surprise that I'd have to. >> b) Going further, it'd be nice for some scenarios to actually be able to >> include private versions of libxml2/libxslt in a Python eggs. This is >> especially interesting on Windows deployments (where you'd include a >> binary of these libraries). Has something like this considered? I saw >> references to Pyrex support, but in lxml's case, the Pyrex code depends >> on a large underlying library. > > You can certainly do that; just list the appropriate .c files in your > Extension. > > For PEAK on Python 2.3, I include an expat wrapping that adds the Python > 2.4 pyexpat features this way, using something like: > > Extension("peak.util.pyexpat", [ > "src/peak/util/pyexpat.c", "src/expat/xmlparse.c", > "src/expat/xmltok.c", "src/expat/xmlrole.c", > ], > include_dirs=["src/expat"], > define_macros=[('XML_STATIC',1),('HAVE_MEMMOVE',1)] # XXX > ), libxml2 however is a huge C library with its own configure script (that it really uses, as it ports to a zillion platforms), so just listing C files to compile might very well not work, right? I guess for Windows, I'd have make distutils run the configure script, then extract the dlls it produces and stuff them in the egg somehow. Any direction you'd point me towards for this? >> c) It's also interesting though for deployment on linux. It'd be nice to >> be able to include the source versions of specific versions of libxml2 >> and libxslt with lxml and to be able to build/install them such that >> they are only used for lxml. This way the system libraries (which may be >> out of date or have otherwise a wrong version) would not be in play and >> wouldn't be affected. > > Yeah, just bake it in as shown above. In this case, on Linux, I'd want to run the configure script when the egg is installed instead of when it's created, and stuff the .so files in the same place the egg is being installed to. >> If something like this were arranged, it'd be much easier to make lxml a >> requirement of a large package like for instance Zope 3 (which is being >> considered). >> >> I realize that any or all of these might be out of scope for easy >> install -- in the Linux case, it might be deferred to a Linux package >> management system, for instance. Still, I imagine the case where a >> Python library has a dependency on a potentially large non-Python >> codebase could be fairly common, and it'd be nice if such libraries >> could be "first class" easy_install citizens so that other Python >> libraries can safely depend on them. What are people's thoughts were >> about supporting such scenarios? > > Not all libraries can be bundled by source, of course. Sometimes you > really need to use whatever the "system version" is, for one reason or > another. Database clients, for example, are something you really really > want to use the local version for. Right, there are competing use cases here. What I'd like is an easy install for lxml that just works for people, without them having to worry about the right lxml2 versions being installed, etc. On Windows this means binaries, and on Linux this likely means it'll just compile upon install. Some classes of people, like distributors and some sysadmins, care about using the platform version of libxml2, and I'd also want to create an egg that allows you to install against the platform libraries. Would this be possible to be the same egg or would a different egg be needed? If a different egg, how does this work with the dependency system? I.e. these two eggs would be alternatives of each other dependency-wise. > I'm thinking that the distutils > could really use some sort of library-finding capabilities for that > stuff, assuming they don't already have some I just haven't found yet. Yes, that would indeed be useful. Thanks for the feedback! Regards, Martijn From mal at egenix.com Sat Sep 24 11:37:40 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 24 Sep 2005 11:37:40 +0200 Subject: [Distutils] python eggs and lxml In-Reply-To: <433518C3.7040104@infrae.com> References: <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> <433518C3.7040104@infrae.com> Message-ID: <43351E64.2020600@egenix.com> Martijn Faassen wrote: > Phillip J. Eby wrote: > >>At 11:57 PM 9/23/2005 +0200, Martijn Faassen wrote: >> >> >>>I've been looking at Python eggs, easy install and setuptools with a lot >>>of interest -- very impressive work. I've been thinking of packaging >>>lxml with it, and, on the larger scale, look into packaging Zope 3 >>>with it. >>> >>>Concerning lxml I run into a few questions however. >>> >>>lxml depends on large external C libraries (libxml2 and libxslt). >>> >>>a) Is there a way to require versions of C libraries to be available in >>>the Python eggs dependencies? I can't seem to find a reference to this >>>scenario, but perhaps I didn't look carefully enough. The goal here >>>would be to give users trying to install lxml (or something that depends >>>on lxml) useful feedback about what in their system they're missing (or >>>have the wrong version of). >> >> >>Your options here are the same as with any distutils package, which is >>to say you have to figure it out yourself. ;) You can add code to look >>for the libraries, embed your own source, etc. > > > Right. I was hoping I didn't need to dive into the internals of > distutils of course, but it's no surprise that I'd have to. Hi Martijn, I'd suggest you take a look at mxSetup.py which is included in all recent egenix-mx-* packages. The egenix-mx-experimental package makes heavy use of its features to build and include external libs. For the latest version, see: http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 24 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From pje at telecommunity.com Sat Sep 24 16:56:51 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Sep 2005 10:56:51 -0400 Subject: [Distutils] python eggs and lxml In-Reply-To: <43351E64.2020600@egenix.com> References: <433518C3.7040104@infrae.com> <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> <433518C3.7040104@infrae.com> Message-ID: <5.1.1.6.0.20050924105029.01b4e820@mail.telecommunity.com> At 11:37 AM 9/24/2005 +0200, M.-A. Lemburg wrote: >I'd suggest you take a look at mxSetup.py which is included >in all recent egenix-mx-* packages. The egenix-mx-experimental >package makes heavy use of its features to build and include >external libs. > >For the latest version, see: > >http://www.egenix.com/files/python/egenix-mx-base-2.1.0-2005-05-01.zip Wow. That's pretty impressive. I'm definitely going to steal some ideas from that for setuptools, especially the flag to indicate that an extension is optional; I could really use that for certain of my packages. I had already been thinking of adding a facility to build_ext to look for needed includes and libraries, but you've certainly put more thought into it than I have so far. From pje at telecommunity.com Sat Sep 24 17:23:52 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Sep 2005 11:23:52 -0400 Subject: [Distutils] python eggs and lxml In-Reply-To: <433518C3.7040104@infrae.com> References: <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> <5.1.1.6.0.20050923180510.02caa040@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050924110319.02dff998@mail.telecommunity.com> At 11:13 AM 9/24/2005 +0200, Martijn Faassen wrote: >I guess for Windows, I'd have make distutils run the configure script, >then extract the dlls it produces and stuff them in the egg somehow. Any >direction you'd point me towards for this? You'll need to have your setup script determine programmatically that it's running on Windows, and then add data files and 'eager_resources' to the project accordingly. In other words, to include shared libraries that aren't distutils-built extensions, you have to treat them as data files, but you must also list them in the 'eager_resources' setup keyword, as described here: http://peak.telecommunity.com/DevCenter/setuptools#automatic-resource-extraction so that the shared libraries will be automatically extracted to disk before they're linked to by any C extensions. You can do this on Unix platforms, too, of course, but the filenames will naturally be different. >In this case, on Linux, I'd want to run the configure script when the egg >is installed instead of when it's created, and stuff the .so files in the >same place the egg is being installed to. Eggs are not source distributions - they're prebuilt binaries for a particular platform. EasyInstall will find and use source distributions if there's no binary for the system, so from a user point of view it certainly happens when the egg is "installed", but that's only because the actual egg is being *built* locally. So, in any situation where you're not using the platform libraries, you'll need to follow the same embedding steps as for Windows - i.e., build the libraries and include them as data files. If you're linking with the platform libraries, you don't need to include them as data files. It's best, however, if you not think of this as install vs. create. The only installation steps eggs have are: * adding them to sys.path * creating local wrappers to run programs contained in them Everything else occurs when the egg is built, it's just that on Unix-y platforms you're more likely to be building the egg locally from source, rather than downloading a pre-built egg, if it contains C extensions. For Python-only projects, of course, eggs are cross-platform and ready-to-use binaries. >>Not all libraries can be bundled by source, of course. Sometimes you >>really need to use whatever the "system version" is, for one reason or >>another. Database clients, for example, are something you really really >>want to use the local version for. > >Right, there are competing use cases here. What I'd like is an easy >install for lxml that just works for people, without them having to worry >about the right lxml2 versions being installed, etc. On Windows this means >binaries, and on Linux this likely means it'll just compile upon install. Or more precisely, it means compiling on egg build, regardless of platform. It's just that for some platforms, you'll build the egg and distribute it instead of the end user building their own. Probably the simplest way to do this is to subclass setuptools 'build_ext' command and extend run() to 'configure' and 'make' the libraries before proceeding normally. >Some classes of people, like distributors and some sysadmins, care about >using the platform version of libxml2, and I'd also want to create an egg >that allows you to install against the platform libraries. Would this be >possible to be the same egg or would a different egg be needed? If a >different egg, how does this work with the dependency system? I.e. these >two eggs would be alternatives of each other dependency-wise. An egg is not a project, or vice versa. What you mean is that you want your project to be able to be built against different targets. The eggs are what gets built. If you've only used EasyInstall as a user, not a developer, this line is more blurry because EasyInstall downloads projects and builds the eggs for you if the author did not provide any eggs usable on your platform. But there is a distinction, and on the authoring side you should focus on the project and its ability to be built for different targets. Providing actual eggs is a convenience for your users, and does not enter into your development process. It's strictly a deployment/publishing step. Anyway, since the audience that wants to do this are more advanced users, and probably building your project individually rather than as part of a multi-project EasyInstall, I think it's reasonable to just have some value in a configuration file or environment variable or the standard distutils setup.cfg that allows skipping the configure+make. For that matter, just documenting how to patch the setup script to use the platform libraries might be sufficient for that audience in the case of this kind of embedding. From pje at telecommunity.com Sat Sep 24 19:04:07 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Sep 2005 13:04:07 -0400 Subject: [Distutils] Putting eggs first on sys.path Message-ID: <5.1.1.6.0.20050924122000.01b6ddf8@mail.telecommunity.com> One of the things that occasionally creates problems for installing applications with setuptools, and for certain non-root package installs on Unix, is the fact that eggs are normally added to the *end* of sys.path, rather than the beginning. I did this because I needed to maintain various invariants in pkg_resources, such as namespace packages' __path__ items needing to match sys.path order. Also, .pth files add entries to the end of sys.path, so changing this isn't really an option for EasyInstall-supplied default eggs. The problems with this are: 1. If you install an application but then later set an incompatible version of one of its requirements as the default version of that project, then the application will stop working 2. If you are using a simplistic non-root installation, system-installed eggs can override or conflict with your personal eggs, and prevent the use of entry points from them. So, after some thought, I think I have a way to adjust the existing policy that will deal with these problems, while still allowing most invariants to remain intact. It's a little kludgy, so I'm hoping somebody has a better idea. For EasyInstall-generated wrapper scripts, it's no big deal and is invisible to the user. For manual use, it seems a little clumsy, though. The idea is this: when pkg_resources is imported, it will check the __main__ module for a __requires__ variable. If found, it will do the equivalent of require()-ing that value, but with sys.path set to an empty list. It will then restore the old value of sys.path, adding it *after* the entries added by the require() process. Thus, the very first require() will insert entries at the start of sys.path, but in a consistent order. Thus, a script can effectively require package versions that are not the default. (If you try this currently, you get VersionConflict errors.) You might now ask, "Why __requires__? Why not just do this for the first require() call?" Unfortunately, it's not that simple. pkg_resources needs to export a global "working_set" object that lists the active eggs and their entry points. Once you've imported pkg_resources, then, this list needs to be in a consistent state. So it's a bit of a chicken and egg problem, in that you need pkg_resources imported to do require(), but if pkg_resources is imported then you need to have already done any require()s that override existing sys.path entries. Thus, putting a variable in a common module (e.g., by putting it in the script before importing pkg_resources) allows us to pass a parameter to something that hasn't been imported yet. For use in the interactive interpreter, things are a bit more complex, because it's possible that you could import something further down on sys.path before importing pkg_resources, possibly leading to an implicit conflict of some kind. You can still set __requires__ and import pkg_resources, but it looks weird to do that, and it's certainly not the usual way to do a require(), so it seems potentially confusing as well. Of course, I suppose I could just make it an "undocumented internal feature" of pkg_resources and setuptools, reserved for EasyInstall-generated scripts. This probably makes sense in that the default versions of packages are the ones that you'll nominally be using in the interactive interpreter. On the other hand, a simple non-root install won't work if you want to override site-wide EasyInstalled defaults, unless you do some fancy footwork in a sitecustomize.py or ~/ Maybe I'm just expecting too much, though. Perhaps it's unrealistic to expect to add new features, be 100% backward compatible, support everybody's personal quirky directory layout on Unix, AND still not have any kludgy bits. :) Thoughts, anyone? From pje at telecommunity.com Sat Sep 24 23:46:29 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Sep 2005 17:46:29 -0400 Subject: [Distutils] setuptools 0.6a3 released Message-ID: <5.1.1.6.0.20050924174357.02c01898@mail.telecommunity.com> This is mostly a bugfix release for various problems that people reported in the last week or two, but I also added improved version conflict handling for scripts (see my post re: sys.path and __requires__), and added the ability to generate GUI scripts with standard wrappers on non-Windows platforms, and .pyw/.exe wrappers on Windows. From william at opensource4you.com Sun Sep 25 12:11:19 2005 From: william at opensource4you.com (william@opensource4you.com) Date: Sun, 25 Sep 2005 12:11:19 +0200 (CEST) Subject: [Distutils] How use Distutils for plugins ? Message-ID: <32986.81.242.157.40.1127643079.squirrel@mail.opensource4you.com> I've discovered the Distutils mailing list after my post to comp.lang.python http://groups.google.be/group/comp.lang.python/browse_thread/thread/b66d6ea597df4dbf/da984e4f31a8f59f?hl=en#da984e4f31a8f59f Does any one have an idea how to use distutils to install sub-elements (f.e. plugins) of a main packages (already installed). Thanks. -- William: http://www.opensource4you.com From ianb at colorstudy.com Sun Sep 25 20:00:22 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 25 Sep 2005 13:00:22 -0500 Subject: [Distutils] setuptools: bug in run_script Message-ID: <4336E5B6.9020000@colorstudy.com> Ben Bangert encountered a problem where pkg_resources.WorkingSet.run_script didn't work; from line 407: self.require(requires)[0].run_script(script_name, ns) If all the packages are already loaded for some reason, then self.require() returns []. I don't really understand how we got in this state -- that apparently packages are available before they are being required. But so it is; it might be from a script wrapper created with 0.6a2? From ianb at colorstudy.com Sun Sep 25 20:20:15 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 25 Sep 2005 13:20:15 -0500 Subject: [Distutils] setuptools: bug in run_script In-Reply-To: <4336E5B6.9020000@colorstudy.com> References: <4336E5B6.9020000@colorstudy.com> Message-ID: <4336EA5F.6060501@colorstudy.com> Ian Bicking wrote: > Ben Bangert encountered a problem where > pkg_resources.WorkingSet.run_script didn't work; from line 407: > > self.require(requires)[0].run_script(script_name, ns) > > If all the packages are already loaded for some reason, then > self.require() returns []. Looking at it, I think self.require(requires)[0] would only work if the package was installed multi-version. When all the packages are activated by default, of course require() should return nothing. From ianb at colorstudy.com Sun Sep 25 20:36:07 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 25 Sep 2005 13:36:07 -0500 Subject: [Distutils] setuptools: bug in run_script In-Reply-To: <4336EA5F.6060501@colorstudy.com> References: <4336E5B6.9020000@colorstudy.com> <4336EA5F.6060501@colorstudy.com> Message-ID: <4336EE17.5080200@colorstudy.com> Ian Bicking wrote: > Ian Bicking wrote: > >>Ben Bangert encountered a problem where >>pkg_resources.WorkingSet.run_script didn't work; from line 407: >> >> self.require(requires)[0].run_script(script_name, ns) >> >>If all the packages are already loaded for some reason, then >>self.require() returns []. > > > Looking at it, I think self.require(requires)[0] would only work if the > package was installed multi-version. When all the packages are > activated by default, of course require() should return nothing. Nevermind that. The problem is the generated scripts: #!/usr/local/bin/python # EASY-INSTALL-SCRIPT: 'PasteScript==0.1dev-r3260','paster' __requires__ = 'PasteScript==0.1dev-r3260' import pkg_resources pkg_resources.run_script('PasteScript==0.1dev-r3260', 'paster') The __requires__ variable causes pkg_resources to require that package. So later when .run_script() is called, the package has already been loaded, and .requires() returns [], and it fails. I actually don't know what situation it would work in. But I also realize setuptools has had a hard time updating my scripts, so I didn't notice it myself. While we're at it, the explicit version in these scripts causes me no end of problems. Since scripts are generally not installed in a multi-version manner (only one script by a particular name can be access on $PATH), it seems they should have versionless requires, thus getting the newest version. Or, at least, they should load the "active" version, whatever that is (typically the newest version). From ianb at colorstudy.com Sun Sep 25 22:09:17 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 25 Sep 2005 15:09:17 -0500 Subject: [Distutils] setuptools: bug in run_script In-Reply-To: <4336EE17.5080200@colorstudy.com> References: <4336E5B6.9020000@colorstudy.com> <4336EA5F.6060501@colorstudy.com> <4336EE17.5080200@colorstudy.com> Message-ID: <433703ED.4090105@colorstudy.com> Ian Bicking wrote: > The __requires__ variable causes pkg_resources to require that package. > So later when .run_script() is called, the package has already been > loaded, and .requires() returns [], and it fails. I actually don't know > what situation it would work in. But I also realize setuptools has had > a hard time updating my scripts, so I didn't notice it myself. Well, okay, maybe not quite that. Anyway, I was able to confirm the problem after doing a more insulated install, and the attached diff against pkg_resources.py fixes it. It would be nice if there was a require-and-find method, whenever I want to get a distribution I find I have to do: pkg_resources.require('Package') dist = pkg_resources.working_set.find(pkg_resources.Require('Package')) at least that's the best combination I've figured out so far. From ianb at colorstudy.com Sun Sep 25 22:16:15 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 25 Sep 2005 15:16:15 -0500 Subject: [Distutils] Non-root install for testing Message-ID: <4337058F.6070700@colorstudy.com> I've created a script to set up a non-root installation of Python, suitable for testing installation procedures. It follows the instructions located here: http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation The script is at: http://svn.colorstudy.com/home/ianb/non_root_python.py To do testing I've created a new user on my system, so I don't mix things up with my normal development, and so I have nothing to lose ;) I use it like: /usr/bin/python non_root_python.py --no-site-packages --clear I doubt it will work on a Windows system, but it should work on a Mac. You can run it with --clear to reset the system quickly, so you can restart your installation process from zero. --no-site-packages doesn't copy anything from the standard site-packages dir (unlike in the instructions), so you can start out even more bare. But you can use that or not, it shouldn't cause a problem either way. (Well, it *can* cause problems, but interesting problems; like I have cheetah installed with a Debian package, so if I copy site-packages I get an error related to that -- I'm not sure what to do about that at this point). Just make sure ~/bin is first in your $PATH. Ian From pje at telecommunity.com Sun Sep 25 23:23:01 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 25 Sep 2005 17:23:01 -0400 Subject: [Distutils] setuptools: bug in run_script Message-ID: <5.1.1.6.0.20050925172301.0370ce70@mail.telecommunity.com> At 01:00 PM 9/25/2005 -0500, Ian Bicking wrote: >Ben Bangert encountered a problem where >pkg_resources.WorkingSet.run_script didn't work; from line 407: > > self.require(requires)[0].run_script(script_name, ns) > >If all the packages are already loaded for some reason, then >self.require() returns []. > >I don't really understand how we got in this state -- that apparently >packages are available before they are being required. That's actually normal, for packages that are already on sys.path (via easy-install.pth). What's not normal is that require() should never return []. As it turns out, it's a bug in an optimization I added for 0.6a3 that avoids creating an Environment if all requirements can be satisfied from the working set. It ends up not adding the relevant packages to the list it returns. Here's a patch to fix it until I get a bugfix release out: Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.72 diff -u -r1.72 pkg_resources.py --- pkg_resources.py 24 Sep 2005 19:48:28 -0000 1.72 +++ pkg_resources.py 25 Sep 2005 18:57:07 -0000 @@ -463,7 +463,7 @@ requirements = list(requirements)[::-1] # set up the stack processed = {} # set of processed requirements - best = dict([(d.key,d) for d in self]) # key -> dist + best = {} # key -> dist to_activate = [] while requirements: @@ -471,20 +471,20 @@ if req in processed: # Ignore cyclic or redundant dependencies continue - dist = best.get(req.key) if dist is None: # Find the best distribution and add it to the map - if env is None: - env = Environment(self.entries) - dist = best[req.key] = env.best_match(req, self, installer) + dist = self.by_key.get(req.key) if dist is None: - raise DistributionNotFound(req) # XXX put more info here + if env is None: + env = Environment(self.entries) + dist = best[req.key] = env.best_match(req, self, installer) + if dist is None: + raise DistributionNotFound(req) # XXX put more info here to_activate.append(dist) elif dist not in req: # Oops, the "best" so far conflicts with a dependency raise VersionConflict(dist,req) # XXX put more info here - requirements.extend(dist.requires(req.extras)[::-1]) processed[req] = True From pje at telecommunity.com Sun Sep 25 23:26:28 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 25 Sep 2005 17:26:28 -0400 Subject: [Distutils] setuptools: bug in run_script In-Reply-To: <433703ED.4090105@colorstudy.com> References: <4336EE17.5080200@colorstudy.com> <4336E5B6.9020000@colorstudy.com> <4336EA5F.6060501@colorstudy.com> <4336EE17.5080200@colorstudy.com> Message-ID: <5.1.1.6.0.20050925172340.037124d8@mail.telecommunity.com> At 03:09 PM 9/25/2005 -0500, Ian Bicking wrote: >Ian Bicking wrote: > > The __requires__ variable causes pkg_resources to require that package. > > So later when .run_script() is called, the package has already been > > loaded, and .requires() returns [], and it fails. I actually don't know > > what situation it would work in. But I also realize setuptools has had > > a hard time updating my scripts, so I didn't notice it myself. > >Well, okay, maybe not quite that. Anyway, I was able to confirm the >problem after doing a more insulated install, and the attached diff >against pkg_resources.py fixes it. There wasn't a diff attached, but in the meantime I've sent a diff that fixes the problem. >It would be nice if there was a require-and-find method, whenever I want >to get a distribution I find I have to do: > >pkg_resources.require('Package') >dist = pkg_resources.working_set.find(pkg_resources.Require('Package')) > >at least that's the best combination I've figured out so far. See: http://peak.telecommunity.com/DevCenter/PkgResources#getting-or-creating-distributions The correct API is get_distribution(), but you need a correctly working pkg_resources, which sadly is not possible with 0.6a3. Downgrade to a2 or use the patch I posted. From pje at telecommunity.com Mon Sep 26 04:11:11 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 25 Sep 2005 22:11:11 -0400 Subject: [Distutils] setuptools 0.6a4 bug fix release Message-ID: <5.1.1.6.0.20050925221017.021a95b0@mail.telecommunity.com> I've released 0.6a4, to fix the require() bug in 0.6a3. From rkern at ucsd.edu Mon Sep 26 06:28:11 2005 From: rkern at ucsd.edu (Robert Kern) Date: Sun, 25 Sep 2005 21:28:11 -0700 Subject: [Distutils] Non-root install for testing In-Reply-To: <4337058F.6070700@colorstudy.com> References: <4337058F.6070700@colorstudy.com> Message-ID: Ian Bicking wrote: > I've created a script to set up a non-root installation of Python, > suitable for testing installation procedures. It follows the > instructions located here: > http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation I really dislike this approach. I'm afraid that I don't understand what problem it's trying to solve. On OS X, the instructions tell you to install to ~/Library/Python/2.4/site-packages . Why do the general UNIX instructions tell you to make a ton of symlinks? IMHO, better advice would be (thanks in part to Prabhu Ramachandran): Make a directory to hold your EasyInstall'ed packages (e.g. ~/lib/python2.4/site-packages). easy_install uses .pth tricks. Therefore, the directory where eggs are installed require that .pth files work there. On non-system wide installs, this can be achieved using a sitecustomize.py somewhere inside PYTHONPATH with the following lines: import site, os site.addsitedir(os.path.expanduser('~/lib/python2.4/site-packages')) easy_install needs to be told to use this directory and that it is okay to use. Add the following options to your ~/.pydistutils.cfg file: [install] install_lib=~/lib/python$py_version_short/site-packages install_scripts=~/bin [easy_install] site_dirs=~/lib/python$py_version_short/site-packages At this point in time, I think this approach is cleaner. Eventually it might run into conflicts with a distribution-provided easy_install, but AFAICT there aren't any of those yet. In any case, neither of these approaches is going to be tenable in the long term. Having the non-root user recreate the whole python distribution via symlinks isn't a reasonable approach. Controlling the installation/activation of eggs via .pth files was a cool approach, but it appears that it's not terribly flexible because of the way that Python inserts them into sys.path. A more general way forward might be to have sitecustomize.py (or site.py if your distro is kind enough to take care of such things for you) read data files that list the eggs that are activated. sitecustomize.py could then insert the eggs wherever it sees fit into sys.path. On Debian, for example, there would be a file in /usr/lib/pythonX.Y/site-packages/ for eggs that are provided by real dpkg-installed .deb's. There would be another file in /usr/local/lib/pythonX.Y/site-packages/ for eggs that are installed by root but aren't real Debian packages. There might be a final one somewhere canonical in a user's home directory (~/.easy_install perhaps) for eggs that are local to the user (and probably installed to ~/lib/pythonX.Y/site-packages/ but not necessarily). These data files would be searched in reverse order: ~/, /usr/local/, /usr/. It would be nice if the data files could specify that certain eggs in the later directories are to be ignored. Does this sound like a workable idea? On a similar note: easy_install does a lot of sanity checks to determine if it ought to install to the given install_lib directory. Unfortunately, those sanity checks can be too restrictive. For example, when I use Debian, I manage my /usr/local/ tree with GNU Stow. I want to put everything managed by easy_install under the prefix /usr/local/stow/easy_install/ (with subdirectories bin/, lib/python2.4/site-packages/, and so on). When I run Stow, everything will be symlinked into /usr/local/. Because Debian was thoughtful, /usr/local/lib/python2.4/site-packages/ is already on the PYTHONPATH and is .pth-enabled so everything would work just fine. Unfortunately, /usr/local/stow/easy_install/lib/python2.4/site-packages/ isn't on the PYTHONPATH and easy_install will refuse to install there even when it has been specified as a valid --site-dir. Paranoia is healthy, but we need a way to override the paranoia. I am requesting a --force option to override this sanity check. Preferably there would be one --force option per sanity check and one overall --force that entails all of the others, but I'll take what I can get. I'm willing to work on a patch if given a little guidance as to where all of the sanity checks are. Thank you for your attention. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From ianb at colorstudy.com Mon Sep 26 07:26:08 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 26 Sep 2005 00:26:08 -0500 Subject: [Distutils] Non-root install for testing In-Reply-To: References: <4337058F.6070700@colorstudy.com> Message-ID: <43378670.4080402@colorstudy.com> Robert Kern wrote: > Ian Bicking wrote: > >>I've created a script to set up a non-root installation of Python, >>suitable for testing installation procedures. It follows the >>instructions located here: >>http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation > > > I really dislike this approach. I'm afraid that I don't understand what > problem it's trying to solve. On OS X, the instructions tell you to > install to ~/Library/Python/2.4/site-packages . Why do the general UNIX > instructions tell you to make a ton of symlinks? IMHO, better advice > would be (thanks in part to Prabhu Ramachandran): > > Make a directory to hold your EasyInstall'ed packages (e.g. > ~/lib/python2.4/site-packages). easy_install uses .pth tricks. > Therefore, the directory where eggs are installed require that .pth > files work there. On non-system wide installs, this can be achieved > using a sitecustomize.py somewhere inside PYTHONPATH with the > following lines: > > import site, os > site.addsitedir(os.path.expanduser('~/lib/python2.4/site-packages')) > > easy_install needs to be told to use this directory and that it is > okay to use. Add the following options to your ~/.pydistutils.cfg > file: > > [install] > install_lib=~/lib/python$py_version_short/site-packages > install_scripts=~/bin > > [easy_install] > site_dirs=~/lib/python$py_version_short/site-packages This is what I am doing to get easy_install to install in /usr/local, and other private directories on the system. However, I find the non-root install useful for testing, as it is easy to recreate, and avoids side-effects from the rest of the system. Much of the time you actually want side effects, which is what you get with the other procedure. Anyway, I think it would be a good idea to have this in the setuptools documentation (it's a wiki, so if you want to add it I think you should feel free). My own description of the process is at: http://blog.ianbicking.org/alternate-python-install-dir.html One issue with a number of these options, is that I don't think easy_install expands ~ or $py_version_short in all its variables. I think that can just be classified a bug, but in practice it will cause problems right now. > At this point in time, I think this approach is cleaner. Eventually it > might run into conflicts with a distribution-provided easy_install, but > AFAICT there aren't any of those yet. > > In any case, neither of these approaches is going to be tenable in the > long term. Having the non-root user recreate the whole python > distribution via symlinks isn't a reasonable approach. Controlling the > installation/activation of eggs via .pth files was a cool approach, but > it appears that it's not terribly flexible because of the way that > Python inserts them into sys.path. > > A more general way forward might be to have sitecustomize.py (or site.py > if your distro is kind enough to take care of such things for you) read > data files that list the eggs that are activated. In practice, easy_install.pth is a list of eggs that are activated. Though it's a little fuzzy, because as eggs they aren't yet activated, even though as Python packages they are available. And I think some of the issue PJE was having with how scripts should install, is what do you do when the egg on sys.path isn't the egg you want to activate. (Can that entry simply be removed from sys.path, and sys.modules checked to make sure nothing from that egg has been imported yet?) > sitecustomize.py could > then insert the eggs wherever it sees fit into sys.path. On Debian, for > example, there would be a file in /usr/lib/pythonX.Y/site-packages/ for > eggs that are provided by real dpkg-installed .deb's. There would be > another file in /usr/local/lib/pythonX.Y/site-packages/ for eggs that > are installed by root but aren't real Debian packages. A .pth file can point to paths anywhere, so a single .pth file (even in, say, /etc/python/packages.pth) could point anywhere in the system, including /usr/lib and /usr/local/lib. It couldn't point to per-user installs, though... > There might be a > final one somewhere canonical in a user's home directory > (~/.easy_install perhaps) for eggs that are local to the user (and > probably installed to ~/lib/pythonX.Y/site-packages/ but not > necessarily). These data files would be searched in reverse order: ~/, > /usr/local/, /usr/. It would be nice if the data files could specify > that certain eggs in the later directories are to be ignored. It would be nice if site(customize).py added ~/lib/pythonX.Y to the path as well, and any .pth files there as well. It would also be nice if the list of directories where .pth files were read was kept somewhere, like say site.sitedirs. These are fairly low-impact changes, I think. In some ways, I would go so far as to say that distribution maintainers have simply ignored these issues, rather than made any explicit (correct or incorrect) choice. For instance, Debian should have a distutils.cfg that installs packages into /usr/local/lib/pythonX.Y. Maybe I should submit that as a bug, because without it distutils simply doesn't meet Debian policy. So, while these are changes to system layout, I think there's little reason for the status quo in many cases. > Does this sound like a workable idea? > > On a similar note: easy_install does a lot of sanity checks to determine > if it ought to install to the given install_lib directory. > Unfortunately, those sanity checks can be too restrictive. For example, > when I use Debian, I manage my /usr/local/ tree with GNU Stow. I want to > put everything managed by easy_install under the prefix > /usr/local/stow/easy_install/ (with subdirectories bin/, > lib/python2.4/site-packages/, and so on). When I run Stow, everything > will be symlinked into /usr/local/. Because Debian was thoughtful, > /usr/local/lib/python2.4/site-packages/ is already on the PYTHONPATH and > is .pth-enabled so everything would work just fine. Unfortunately, > /usr/local/stow/easy_install/lib/python2.4/site-packages/ isn't on the > PYTHONPATH and easy_install will refuse to install there even when it > has been specified as a valid --site-dir. There's also an issue where easy_install has a site_dir option, but no other installation methods do (e.g., setup.py install, setup.py develop). > Paranoia is healthy, but we need a way to override the paranoia. I am > requesting a --force option to override this sanity check. Preferably > there would be one --force option per sanity check and one overall > --force that entails all of the others, but I'll take what I can get. > I'm willing to work on a patch if given a little guidance as to where > all of the sanity checks are. I'd like the whole thing to be more expansive. That is, not to have it as an easy_install option, but as an option that applies to all of distutils (or at least all of setuptools). In this case, both a global option for site_dirs, and a global option for the sys.path (in this case you say it should treat your stow directory as being on sys.path). From trentm at ActiveState.com Mon Sep 26 19:16:49 2005 From: trentm at ActiveState.com (Trent Mick) Date: Mon, 26 Sep 2005 10:16:49 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: <5.1.1.6.0.20050923201036.02e5f8a8@mail.telecommunity.com> References: <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050923201036.02e5f8a8@mail.telecommunity.com> Message-ID: <20050926171649.GC30808@ActiveState.com> [Phillip J. Eby wrote] > > With Perl and Ruby, for example, scripts from a > >3rd party package will be installed next to the main interpreter binary > >(i.e. on the PATH) on all platforms. Would having setuptools (and > >changing distutils) to install scripts next to python.exe wreak > >unwarranted havoc? > > Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless > you put it there yourself. It gets on the PATH with ActivePython . I'd recommend that the python.org Python installer do the same, but I seem to remember MvL (and others?) stating their preference not to do so (don't recall why). In any case, I'd say it is extremely *common* for Windows users to put the Python install dir on their PATH -- just as it is common for Windows Perl and Ruby users to put the interpreter dir on their PATH. I'd then say that the obvious place to put user scripts installed via distutils is that same directory. > Second, the python.exe directory is on > sys.path, so it would turn your scripts into modules, conflicting with any > same-named modules. That never occured to me that that could be a problem. I have a few scripts that I've been distributing for a while and haven't had a particular issue... because I tend to write so that my scripts are (somewhat) useful as modules as well. But you are right, some people could get surprised by this. Hrm. Either (1) this behaviour (Python putting the interpreter dir on sys.path) is just broken or (2) it was an intentional design and script-writers should be expected to ensure their scripts either work reasonably or at least don't cause damage when imported. Since the directory with the interpreter is NOT on sys.path on other platforms (Linux, Un*x) then I'd say this is just broken behaviour. Anyone know, off hand, *why* the Python interpreter dir is on sys.path on Windows? Trent -- Trent Mick TrentM at ActiveState.com From theller at python.net Mon Sep 26 19:22:45 2005 From: theller at python.net (Thomas Heller) Date: Mon, 26 Sep 2005 19:22:45 +0200 Subject: [Distutils] easy_install question References: <5.1.1.6.0.20050923164945.02ae0af8@mail.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 09:38 PM 9/23/2005 +0200, Thomas Heller wrote: > >>- When I run 'easy_install ctypes', the behaviour depends on the current >>directory. If I run this when the current dir contains a 'ctypes' >>subdirectory, it doesn't search pypi^H^H^H^HCheeseShop for this name, >>instead it seems to attempt to install from source in that directory. >>Is that intended? > > Yes. EasyInstall's arguments may be a URL, an existing local filename, or > a "requirement" string, and the precedence is in that order. (i.e., it's > checked to see if it's a URL, then if it's a filename, and finally parsing > as a requirement string is done.) > > >> Can I override this behaviour? > > Add a version specifier, e.g. "ctypes==4.0", or use an absolute URL. Hm. A big advantage is that I don't have to remember the URL. >>- When I install a package which requires another package is there a way >>to specify a download url for the required package? > > The --find-links option can include download links as well as URLs of pages > containing download links. If you pass a URL that ends with a recognizable > distutils filename, EasyInstall will use it if it matches the version > requirement(s). > > A question: are you embedding EasyInstall in something else, or are these > questions about direct use? No, direct use. The use case is a little bit weird, because I was trying to install comtypes (via the pypi link), and comtypes requires an 'inofficial' ctypes version which is available on the comtypes SF files area. Thomas From theller at python.net Mon Sep 26 19:49:11 2005 From: theller at python.net (Thomas Heller) Date: Mon, 26 Sep 2005 19:49:11 +0200 Subject: [Distutils] Improving distutils' script and GUI app handling References: <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050915104237.01b707f0@mail.telecommunity.com> <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050923201036.02e5f8a8@mail.telecommunity.com> <20050926171649.GC30808@ActiveState.com> Message-ID: Trent Mick writes: > [Phillip J. Eby wrote] >> > With Perl and Ruby, for example, scripts from a >> >3rd party package will be installed next to the main interpreter binary >> >(i.e. on the PATH) on all platforms. Would having setuptools (and >> >changing distutils) to install scripts next to python.exe wreak >> >unwarranted havoc? >> >> Sadly, yes. First of all, python.exe isn't *on* the PATH on Windows unless >> you put it there yourself. > > It gets on the PATH with ActivePython . I'd recommend that the > python.org Python installer do the same, but I seem to remember MvL (and > others?) stating their preference not to do so (don't recall why). > > In any case, I'd say it is extremely *common* for Windows users to put > the Python install dir on their PATH -- just as it is common for Windows > Perl and Ruby users to put the interpreter dir on their PATH. I'd then > say that the obvious place to put user scripts installed via distutils > is that same directory. Well, I don't think so. It *may* be common for windows users who use the command line ;-), but I would guess that's a minority anyway. Even though I *use* the command line, I've never put python.exe on my PATH. It's easier to type 'myscript' or 'myscript.py' than 'python myscript.py' anyway. Then, I have multiple versions installed on my machine, and to select between 2.3.5, 2.4.1, and the CVS version I can always use 'py23 myscript.py', 'py24 myscript.py', 'py myscript.py' and 'py_d myscript.py' - the latter to run it with the debug compiled CVS version. I had created batchfiles py23.bat and so on which specify the directory of the python.exe or python_d.exe files. And this also explains why I'm still waiting for the extented version of the '-m' command line option and did not bother to put the PythonXY\Scripts directory on the PATH - I would loose the ability to easily switch versions. Thomas From dangoor at gmail.com Tue Sep 27 14:03:36 2005 From: dangoor at gmail.com (Kevin Dangoor) Date: Tue, 27 Sep 2005 08:03:36 -0400 Subject: [Distutils] How use Distutils for plugins ? In-Reply-To: <32986.81.242.157.40.1127643079.squirrel@mail.opensource4you.com> References: <32986.81.242.157.40.1127643079.squirrel@mail.opensource4you.com> Message-ID: <3f085ecd05092705031531f4d4@mail.gmail.com> On 9/25/05, william at opensource4you.com wrote: > I've discovered the Distutils mailing list after my post to comp.lang.python > > http://groups.google.be/group/comp.lang.python/browse_thread/thread/b66d6ea597df4dbf/da984e4f31a8f59f?hl=en#da984e4f31a8f59f > > Does any one have an idea how to use distutils to install sub-elements > (f.e. plugins) of a main packages (already installed). You probably want to use setuptools http://peak.telecommunity.com/DevCenter/setuptools Python Eggs let you break up a package into multiple subpackages, specify dependencies, specify optional parts to install at installation time, and even have a way to dynamically locate which plugins satisfy which features. Kevin -- Kevin Dangoor Author of the Zesty News RSS newsreader email: kid at blazingthings.com company: http://www.BlazingThings.com blog: http://www.BlueSkyOnMars.com From trentm at ActiveState.com Tue Sep 27 18:21:12 2005 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 27 Sep 2005 09:21:12 -0700 Subject: [Distutils] Improving distutils' script and GUI app handling In-Reply-To: References: <20050915165838.GA3219@ActiveState.com> <20050916201043.GA7448@ActiveState.com> <79990c6b050916140067c88bd2@mail.gmail.com> <5.1.1.6.0.20050917131150.02bee368@mail.telecommunity.com> <79990c6b05092303171a27cdd3@mail.gmail.com> <5.1.1.6.0.20050923114148.01b79330@mail.telecommunity.com> <5.1.1.6.0.20050923201036.02e5f8a8@mail.telecommunity.com> <20050926171649.GC30808@ActiveState.com> Message-ID: <20050927162112.GA14004@ActiveState.com> [Trent] > > In any case, I'd say it is extremely *common* for Windows users to put > > the Python install dir on their PATH -- just as it is common for Windows > > Perl and Ruby users to put the interpreter dir on their PATH. I'd then > > say that the obvious place to put user scripts installed via distutils > > is that same directory. [Thomas Heller wrote] > Well, I don't think so. It *may* be common for windows users who use > the command line ;-), but I would guess that's a minority anyway. A minority of Windows users of Python use the command line? Really? Anyone coming from Unix will immediately go there. Last I remember "Learning Python" and the Python tutorial and other intro texts encourage the user to open the interactive shell at the command line to play around. You may be right, though. Even if it is "a minority" I suspect that is still a big number. Even for that set of Windows users that *do* use the command line, having Python get on their PATH automatically just makes sense: It allows other systems that depend on Python to be able to just work after an installation. For example, the Mozilla build system on Windows requires an installation of Perl. The build instructions can just say "install ActivePerl" (which puts Perl on the user's PATH) and thereafter the build system can easily find the Perl is needs. I bet that systems out there requiring Python have more difficulty. It allows tutorials/introductory-texts to have a lowest common denominator that just works for newbies on all platforms: 1. install Python 2. type "python script.py" Then they can get on to all the subtle platform-specific bells and whistles, viz: > Even though I *use* the command line, I've never put python.exe on my > PATH. It's easier to type 'myscript' or 'myscript.py' than 'python > myscript.py' anyway. That is presuming that the Windows installer set up a .py file association and put .py;.pyc;... in the PATHEXT env. var. Does putting an installed Python on the PATH cause damage in any way? > Then, I have multiple versions installed on my > machine, and to select between 2.3.5, 2.4.1, and the CVS version I can > always use 'py23 myscript.py', 'py24 myscript.py', 'py myscript.py' and > 'py_d myscript.py' - the latter to run it with the debug compiled CVS > version. I had created batchfiles py23.bat and so on which specify the > directory of the python.exe or python_d.exe files. A week or so ago I asked you what you thought of the idea of having the Windows build (and installers) have a "pythonXY.exe" or "pythonX.Y.exe" along with the usual "python.exe". This would match what is done on other platforms to allow multiple versions on Python on the PATH and still be able pick a specific one. (I'm not sure if you replied.) I do something similar to your "py23", "py24", ... thing. A common mechanism for this would be nice. > And this also explains why I'm still waiting for the extented version of > the '-m' command line option and did not bother to put the > PythonXY\Scripts directory on the PATH - I would loose the ability to > easily switch versions. And gain a subtle python-specific way to launch a script. I realize I am ranting above, but I just think it is crazy to have the situation these days where, for example, I download and install pychecker and then try to run it: C:\trentm\tmp>pychecker 'pychecker' is not recognized as an internal or external command, operable program or batch file. I understand your wanting a mechanism to easily be able to run any Python scripts/modules you have with any Python installation/build you have, but the common user case is just: (1) I have my latest Python installation, (2) I want to run something installed with "python setup.py install" Trent -- Trent Mick TrentM at ActiveState.com From ianb at colorstudy.com Thu Sep 29 00:16:55 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 28 Sep 2005 17:16:55 -0500 Subject: [Distutils] pkg_resources: feature request Message-ID: <433B1657.7080702@colorstudy.com> I just added a WSGI app in paste.urlparser.PkgResourceParser (http://svn.pythonpaste.org/Paste/branches/mainline-refactor/paste/urlparser.py) to serve static files using pkg_resources's resource API. Two new methods would be useful there: resource_mtime and resource_size. Alternately, resource_stat, which would return an os.stat-compatible object. But a lot of what os.stat returns doesn't apply well to a zip file entry, even if mtime and size do. I'm not sure if mtime should be the mtime of the zip file itself, or the date_time of the entry. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Thu Sep 29 01:11:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 19:11:46 -0400 Subject: [Distutils] pkg_resources: feature request In-Reply-To: <433B1657.7080702@colorstudy.com> Message-ID: <5.1.1.6.0.20050928190732.01f704c0@mail.telecommunity.com> At 05:16 PM 9/28/2005 -0500, Ian Bicking wrote: >I just added a WSGI app in paste.urlparser.PkgResourceParser >(http://svn.pythonpaste.org/Paste/branches/mainline-refactor/paste/urlparser.py) > >to serve static files using pkg_resources's resource API. Two new >methods would be useful there: resource_mtime and resource_size. >Alternately, resource_stat, which would return an os.stat-compatible >object. But a lot of what os.stat returns doesn't apply well to a zip >file entry, even if mtime and size do. I'm not sure if mtime should be >the mtime of the zip file itself, or the date_time of the entry. FYI, if you use resource_filename() and stat the file, you'll get the datetime of the file as it was in the zip, as well as the size. If you've opened the file for reading using resource_stream(), then you can try using seek() and tell() to find the file size. I believe StringIO (used for zip files) and regular file objects support that. I'll think about adding more direct ways to query those things, but in the meantime those are some good ways to get the information from the existing implementation. I realize you may not want to unpack the zipfile to disk (as resource_filename does), but it's cached on disk thereafter, and with some servers you could potentially take advantage of static file serving that way. And in the case of an egg that was installed unzipped (or a project that's under development via "setup.py develop"), the file will already be on disk and resource_filename() is just telling you where it is. From pje at telecommunity.com Thu Sep 29 01:23:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 19:23:38 -0400 Subject: [Distutils] Recursive package data In-Reply-To: <5614D45E-3B8B-4C80-950B-5EA69C5F7D80@groovie.org> References: <5.1.1.6.0.20050927124414.021a7508@mail.telecommunity.com> <5.1.1.6.0.20050926213045.021b1dc0@mail.telecommunity.com> <5.1.1.6.0.20050926213045.021b1dc0@mail.telecommunity.com> <5.1.1.6.0.20050927124414.021a7508@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050928191742.03887e18@mail.telecommunity.com> At 01:25 PM 9/28/2005 -0700, Ben Bangert wrote: >As I'm looking at setuptools to handle distribution and usage of web >application's, being able to package arbitrarily deep static content >that belongs with the webapp package (of various extension types) >starts cropping up. Having no way to just say, "everything under this >dir is package data, keep it" becomes a real killer in this context. > >I think I heard somewhere that this feature is being discussed/ >implemented, when will it be available? (or is it not being developed?) I originally thought I could just slap in detection via revision control metadata, but it turned out to be problematic. Doing "sdist" this way is not a problem, because the generated sdist doesn't need to know how its contents were generated - you don't generally build another sdist from an sdist. But doing this for data files would mean the information used to generate the list would be lost to source distributions, which would be Very Bad, because you then couldn't generate a bdist_egg from an sdist. So, the current status is that I'm a bit stumped as to how to implement the feature sanely, and was hoping for some type of brainstorm that would make it easier. :) Really, I suppose one way to do it would be to just include *everything* in package directories (minus CVS/.svn subdirs or other specialized exclusions), but this quickly seems to be duplicating the MANIFEST and MANIFEST.in mechanism of the distutils. In short, for right now I recommend simply writing your own routine(s) to generate file lists or a package_data dictionary, and I'd be interested in seeing different approaches and trying to get some kind of synthesis together in setuptools, assuming I don't come up with a clean way to unify the manifest stuff with this. From ianb at colorstudy.com Thu Sep 29 01:29:48 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 28 Sep 2005 18:29:48 -0500 Subject: [Distutils] Recursive package data In-Reply-To: <5.1.1.6.0.20050928191742.03887e18@mail.telecommunity.com> References: <5.1.1.6.0.20050927124414.021a7508@mail.telecommunity.com> <5.1.1.6.0.20050926213045.021b1dc0@mail.telecommunity.com> <5.1.1.6.0.20050926213045.021b1dc0@mail.telecommunity.com> <5.1.1.6.0.20050927124414.021a7508@mail.telecommunity.com> <5.1.1.6.0.20050928191742.03887e18@mail.telecommunity.com> Message-ID: <433B276C.4070005@colorstudy.com> Phillip J. Eby wrote: > At 01:25 PM 9/28/2005 -0700, Ben Bangert wrote: > >>As I'm looking at setuptools to handle distribution and usage of web >>application's, being able to package arbitrarily deep static content >>that belongs with the webapp package (of various extension types) >>starts cropping up. Having no way to just say, "everything under this >>dir is package data, keep it" becomes a real killer in this context. >> >>I think I heard somewhere that this feature is being discussed/ >>implemented, when will it be available? (or is it not being developed?) > > > I originally thought I could just slap in detection via revision control > metadata, but it turned out to be problematic. Doing "sdist" this way is > not a problem, because the generated sdist doesn't need to know how its > contents were generated - you don't generally build another sdist from an > sdist. But doing this for data files would mean the information used to > generate the list would be lost to source distributions, which would be > Very Bad, because you then couldn't generate a bdist_egg from an sdist. Isn't that information in MANIFEST? Couldn't every package that gets installed have every file under it that is also in MANIFEST be considered a package_data file? Is there stuff people distribute that shouldn't be installed? I guess tests might count (I personally have come to prefer keeping them outside of the packages, so it's not a problem). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ashleywalsh at gmail.com Thu Sep 29 01:53:43 2005 From: ashleywalsh at gmail.com (Ashley Walsh) Date: Thu, 29 Sep 2005 09:53:43 +1000 Subject: [Distutils] setuptools unittests on Mac OS X Message-ID: Just a small fix to make the setuptools.tests.test_resources.DistroTests.testCollection pass on OSX (and other non-win32 platforms): diff -r1.25 test_resources.py 26c26 < ad = Environment([], python=None) --- > ad = Environment([], platform='win32', python=None) Without this the Environment uses the get_platform() result, and Distribution.from_filename("FooPkg-1.4-py2.4-win32.egg") is not added to the environment. (Maybe some can_add() tests would help?) Cheers, Ashley From pobrien at orbtech.com Thu Sep 29 18:26:02 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Thu, 29 Sep 2005 11:26:02 -0500 Subject: [Distutils] cli.exe error Message-ID: <433C159A.9020400@orbtech.com> I'm getting the following error and can't seem to find any information about what might be causing it. Any help would be greatly appreciated. C:\Code\1_schevo>python ez_setup.py . Reading http://orbtech.com/static/eggs/ Reading http://peak.telecommunity.com/snapshots/ Processing . Running setup.py -q bdist_egg --dist-dir C:\Code\1_schevo\egg-dist-tmp-syg_za schevo 1.90 is already the active version in easy-install.pth Installing db_create.py script to C:\Python24\Scripts error: c:\python24\lib\site-packages\setuptools-0.6a4-py2.4.egg\setuptools\cli.exe: No such file or directory There is no cli.exe file in that directory. I tried removing and reinstalling setuptools, which appeared to install successfully: C:\Code\1_schevo>python ez_setup.py Downloading http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6a4-py2.4.egg Reading http://orbtech.com/static/eggs/ Reading http://peak.telecommunity.com/snapshots/ Processing setuptools-0.6a4-py2.4.egg creating c:\python24\lib\site-packages\setuptools-0.6a4-py2.4.egg Extracting setuptools-0.6a4-py2.4.egg to c:\python24\lib\site-packages Adding setuptools 0.6a4 to easy-install.pth file Installing easy_install.py script to C:\Python24\Scripts error: None But still, no cli.exe, no love... :-( -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From pje at telecommunity.com Thu Sep 29 18:41:31 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:41:31 -0400 Subject: [Distutils] setuptools unittests on Mac OS X In-Reply-To: Message-ID: <5.1.1.6.0.20050929124113.02271ea8@mail.telecommunity.com> At 09:53 AM 9/29/2005 +1000, Ashley Walsh wrote: >Just a small fix to make the >setuptools.tests.test_resources.DistroTests.testCollection pass on >OSX (and other non-win32 platforms): > >diff -r1.25 test_resources.py >26c26 >< ad = Environment([], python=None) >--- > > ad = Environment([], platform='win32', python=None) > >Without this the Environment uses the get_platform() result, and >Distribution.from_filename("FooPkg-1.4-py2.4-win32.egg") is not added >to the environment. (Maybe some can_add() tests would help?) platform=None will also work. I'll fix in the next release. From pje at telecommunity.com Thu Sep 29 18:43:56 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:43:56 -0400 Subject: [Distutils] cli.exe error In-Reply-To: <433C159A.9020400@orbtech.com> Message-ID: <5.1.1.6.0.20050929124134.0399ad00@mail.telecommunity.com> At 11:26 AM 9/29/2005 -0500, Patrick K. O'Brien wrote: >But still, no cli.exe, no love... :-( setup.py is broken. It should say '*.exe' on line 39, not 'launcher.exe'. I goofed. Bugfix release to follow. :( From pje at telecommunity.com Thu Sep 29 18:57:02 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:57:02 -0400 Subject: [Distutils] setuptools 0.6a5 brown bag release :( Message-ID: <5.1.1.6.0.20050929125523.0355f448@mail.telecommunity.com> I've updated setuptools to fix the missing launcher .exe files for Windows. Sorry about that. :( From pobrien at orbtech.com Thu Sep 29 20:52:48 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Thu, 29 Sep 2005 13:52:48 -0500 Subject: [Distutils] setuptools 0.6a5 brown bag release :( In-Reply-To: <5.1.1.6.0.20050929125523.0355f448@mail.telecommunity.com> References: <5.1.1.6.0.20050929125523.0355f448@mail.telecommunity.com> Message-ID: <433C3800.6000009@orbtech.com> Phillip J. Eby wrote: > I've updated setuptools to fix the missing launcher .exe files for > Windows. Sorry about that. :( No problem, and no need to apologize. I greatly admire and appreciate the effort you are putting into making setuptools a success. Keep up the great work. :-) -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From stephen.langer at nist.gov Fri Sep 30 19:57:03 2005 From: stephen.langer at nist.gov (Stephen Langer) Date: Fri, 30 Sep 2005 13:57:03 -0400 Subject: [Distutils] linking modules to a shared library Message-ID: Hi -- I have a problem that is similar to one discussed in a thread from a year ago, http://mail.python.org/pipermail/distutils-sig/2004-September/ 004160.html, but that thread doesn't quite have a resolution in the archives. I'd like to use distutils to build a bunch of python modules that all link to the same shared library. The shared library is all C++ code. On Linux, without distutils, I can build the libraries like this: g++ -o libBase.so -shared base.C # builds the common shared library g++ -o _mymodule.so -shared mymodule.C -lBase # builds a python module # linking to the shared lib. g++ -o _othermodule.so -shared othermodule.C -lBase # and so on _mymodule.so and _othermodule.so can be imported by Python, and share a single copy of the code in libBase.so. On the Mac I can get the same effect with this: g++ -dynamiclib base.C -o libBase.dylib g++ -o _mymodule.so mymodule.C -lBase -bundle -undefined dynamic_lookup -flat_namespace g++ -o _othermodule.so othermodule.C -lBase -bundle -undefined dynamic_lookup -flat_namespace Is it possible to create libBase portably with distutils? It's possible to do it on Linux by subclassing build_ext.build_ext and explicitly using self.compiler.compile() and self.compiler.link_shared_lib() to build the shared library before calling build_ext.build_ext.build_extensions(). But the same thing on Mac OS X only creates libBase.so, whereas I need it to create libBase.dylib. If it matters, I'm using OS X 10.4.2 and gcc 3.3, with Python 2.3.5. Many thanks, Steve -- -- EMail: stephen.langer at nist.gov Phone: (301) 975-5423 -- -- WWW: http://math.nist.gov/mcsd/Staff/SLanger/ Fax: (301) 990-4127 -- -- Mail: NIST; 100 Bureau Drive -- Stop 8910; Gaithersburg, Md 20899-8910 -- -- "I don't think this will work. That's why it's science." -- -- Naomi Langer, 17 Feb 2003 --