From doko at cs.tu-berlin.de Fri Nov 2 15:51:25 2007 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Fri, 2 Nov 2007 15:51:25 +0100 Subject: [Distutils] setuptools -- passing options to subcommands? Message-ID: <18219.14701.930253.374768@gargle.gargle.HOWL> I'd like to omit the python version in the .egg-info directory name when using the --single-version-externally-managed option. In this case I know that the module is installed in a patch specific to the python version. How can this option be propagated to the subcommands 'install_egg_info' and 'install_scripts'? thanks, Matthias From pje at telecommunity.com Fri Nov 2 16:48:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Nov 2007 11:48:15 -0400 Subject: [Distutils] setuptools -- passing options to subcommands? In-Reply-To: <18219.14701.930253.374768@gargle.gargle.HOWL> References: <18219.14701.930253.374768@gargle.gargle.HOWL> Message-ID: <20071102154532.697FC3A40B1@sparrow.telecommunity.com> At 03:51 PM 11/2/2007 +0100, Matthias Klose wrote: >I'd like to omit the python version in the .egg-info directory name >when using the --single-version-externally-managed option. In this >case I know that the module is installed in a patch specific to the >python version. How can this option be propagated to the subcommands >'install_egg_info' and 'install_scripts'? install_egg_info does not have an option to omit the python version. From doko at cs.tu-berlin.de Fri Nov 2 17:22:31 2007 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Fri, 2 Nov 2007 17:22:31 +0100 Subject: [Distutils] setuptools -- passing options to subcommands? In-Reply-To: <20071102154532.697FC3A40B1@sparrow.telecommunity.com> References: <18219.14701.930253.374768@gargle.gargle.HOWL> <20071102154532.697FC3A40B1@sparrow.telecommunity.com> Message-ID: <18219.20167.100444.992896@gargle.gargle.HOWL> Phillip J. Eby writes: > At 03:51 PM 11/2/2007 +0100, Matthias Klose wrote: > >I'd like to omit the python version in the .egg-info directory name > >when using the --single-version-externally-managed option. In this > >case I know that the module is installed in a patch specific to the > >python version. How can this option be propagated to the subcommands > >'install_egg_info' and 'install_scripts'? > > install_egg_info does not have an option to omit the python version. sorry, maybe I was not clear; I know that such an option doesn't exist yet. I was trying to add such an option, therefore the question how to pass this option to the subcommand. Matthias From pje at telecommunity.com Fri Nov 2 20:21:45 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Nov 2007 15:21:45 -0400 Subject: [Distutils] setuptools -- passing options to subcommands? In-Reply-To: <18219.20167.100444.992896@gargle.gargle.HOWL> References: <18219.14701.930253.374768@gargle.gargle.HOWL> <20071102154532.697FC3A40B1@sparrow.telecommunity.com> <18219.20167.100444.992896@gargle.gargle.HOWL> Message-ID: <20071102191857.4D9F03A40AE@sparrow.telecommunity.com> At 05:22 PM 11/2/2007 +0100, Matthias Klose wrote: >Phillip J. Eby writes: > > At 03:51 PM 11/2/2007 +0100, Matthias Klose wrote: > > >I'd like to omit the python version in the .egg-info directory name > > >when using the --single-version-externally-managed option. In this > > >case I know that the module is installed in a patch specific to the > > >python version. How can this option be propagated to the subcommands > > >'install_egg_info' and 'install_scripts'? > > > > install_egg_info does not have an option to omit the python version. > >sorry, maybe I was not clear; I know that such an option doesn't exist >yet. I was trying to add such an option, therefore the question how to >pass this option to the subcommand. Well, at least I knew an answer to the old question. Now I have no idea what you're asking. :) From ziade.tarek at gmail.com Sun Nov 4 22:07:23 2007 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 4 Nov 2007 22:07:23 +0100 Subject: [Distutils] zc.buildout : Playing with environment in cfg files Message-ID: <94bdd2610711041307w4be285efvfff78923481646b5@mail.gmail.com> Does buildout has something that would allow me to write something like this in my .cfg: eggs = #ifdef win32 pywin32 #endif That would be nice indeed. Or maybe a way to set up an environement before buildout starts reading the config file ? thanks Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20071104/55a3cd53/attachment.htm From jim at zope.com Sun Nov 4 23:42:13 2007 From: jim at zope.com (Jim Fulton) Date: Sun, 4 Nov 2007 17:42:13 -0500 Subject: [Distutils] zc.buildout : Playing with environment in cfg files In-Reply-To: <94bdd2610711041307w4be285efvfff78923481646b5@mail.gmail.com> References: <94bdd2610711041307w4be285efvfff78923481646b5@mail.gmail.com> Message-ID: On Nov 4, 2007, at 4:07 PM, Tarek Ziad? wrote: > Does buildout has something that would allow me to write something > like this in my .cfg: > > eggs = > #ifdef win32 > pywin32 > #endif > > That would be nice indeed. No. Note that you could have a win32.cfg that you run on Windows that does something special. I've considered having a syntax for platform-specific options or sections. We haven't needed it up to now. > Or maybe a way to set up an environement before buildout starts > reading the config file ? You can set up as many environments as you want. :) Jim -- Jim Fulton Zope Corporation From benhoyt at gmail.com Wed Nov 7 04:32:55 2007 From: benhoyt at gmail.com (Ben Hoyt) Date: Wed, 7 Nov 2007 16:32:55 +1300 Subject: [Distutils] httplib.py chunked Transfer-Encoding bug In-Reply-To: <7c93bf1e0711061930x49286441jdf0ddb3c0cc52efa@mail.gmail.com> References: <7c93bf1e0711061930x49286441jdf0ddb3c0cc52efa@mail.gmail.com> Message-ID: <7c93bf1e0711061932v2ede5b70xb150086aae13c3b1@mail.gmail.com> Hi Greg, I just ran into that bug in httplib.py when a server sends bad input using chunked Transfer-Encoding, and noticed there's already an exact issue for that, and that you've made a patch: http://bugs.python.org/issue1486335 I don't know anything about Python bug reporting and when/how such things get included in the main Python branch, but I'm curious as to why your patch (or something like it) hasn't been included yet. At present I'm fixing it in my code with a sys.exc_info() hack to make sure the ValueError's coming from _read_chunked(), otherwise re-raising. :-) Cheers, Ben. -- Ben Hoyt, http://benhoyt.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20071107/98de5a17/attachment.htm From dpgrote at lbl.gov Thu Nov 8 19:52:49 2007 From: dpgrote at lbl.gov (Dave Grote) Date: Thu, 08 Nov 2007 10:52:49 -0800 Subject: [Distutils] find_on_path permission denied Message-ID: <47335B01.8040504@lbl.gov> Hello All, I am using a package (scipy) that makes use of pkg_resources, and I am getting the following error during the import... >>> from scipy import optimize Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python2.5/site-packages/scipy/__init__.py", line 18, in import pkg_resources as _pr # activate namespace packages (manipulates __path__) File "/usr/lib/python2.5/site-packages/setuptools-0.6c3-py2.5.egg/pkg_resources.py", line 2470, in File "/usr/lib/python2.5/site-packages/setuptools-0.6c3-py2.5.egg/pkg_resources.py", line 343, in __init__ File "/usr/lib/python2.5/site-packages/setuptools-0.6c3-py2.5.egg/pkg_resources.py", line 358, in add_entry File "/usr/lib/python2.5/site-packages/setuptools-0.6c3-py2.5.egg/pkg_resources.py", line 1577, in find_on_path OSError: [Errno 13] Permission denied: '/work/home/dave/scriptsnew' This error is similar to the error reported in http://mail.python.org/pipermail/distutils-sig/2006-July/006551.html. In a response to that message, http://mail.python.org/pipermail/distutils-sig/2006-July/006552.html, a comment was made that permission problems would be handled more gracefully. Has this not yet been implemented? I am using setuptools-0.6c3-py2.5.egg, python2.5 on FC7. This is of course easy enough for me to get around, but I thought this would be good to report. Dave From pje at telecommunity.com Thu Nov 8 20:17:27 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 08 Nov 2007 14:17:27 -0500 Subject: [Distutils] find_on_path permission denied In-Reply-To: <47335B01.8040504@lbl.gov> References: <47335B01.8040504@lbl.gov> Message-ID: <20071108191933.386933A40B1@sparrow.telecommunity.com> At 10:52 AM 11/8/2007 -0800, Dave Grote wrote: >This error is similar to the error reported in >http://mail.python.org/pipermail/distutils-sig/2006-July/006551.html. In >a response to that message, >http://mail.python.org/pipermail/distutils-sig/2006-July/006552.html, a >comment was made that permission problems would be handled more >gracefully. Has this not yet been implemented? Apparently not. ;) > I am using >setuptools-0.6c3-py2.5.egg, python2.5 on FC7. This is of course easy >enough for me to get around, but I thought this would be good to report. Thanks for the reminder. From zooko at zooko.com Sat Nov 10 00:57:26 2007 From: zooko at zooko.com (zooko) Date: Fri, 9 Nov 2007 16:57:26 -0700 Subject: [Distutils] setuptools bug report: include_package_data is on by default if there is a find-files plugin? Message-ID: <68234720-6FB3-485A-95A3-50B703D0B56B@zooko.com> Folks: If my setuptools_darcs plugin [1] is installed, then all files under revision control get included in packages, even if include_package_data=True was not passed. I'm going to leave my README.txt claiming that this argument is necessary, under the hypothesis that this is a bug. Regards, Zooko [1] http://pypi.python.org/pypi/setuptools_darcs From pje at telecommunity.com Sat Nov 10 02:56:46 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Nov 2007 20:56:46 -0500 Subject: [Distutils] setuptools bug report: include_package_data is on by default if there is a find-files plugin? In-Reply-To: <68234720-6FB3-485A-95A3-50B703D0B56B@zooko.com> References: <68234720-6FB3-485A-95A3-50B703D0B56B@zooko.com> Message-ID: <20071110015649.8C03F3A405B@sparrow.telecommunity.com> At 04:57 PM 11/9/2007 -0700, zooko wrote: >Folks: > >If my setuptools_darcs plugin [1] is installed, then all files under >revision control get included in packages, even if >include_package_data=True was not passed. By "included in packages", do you mean included in eggs and installed, or do you mean included in source distributions? From piyush.verma at gmail.com Sat Nov 10 13:12:51 2007 From: piyush.verma at gmail.com (Piyush Verma) Date: Sat, 10 Nov 2007 17:42:51 +0530 Subject: [Distutils] Extending the distutils module Message-ID: <200711101742.51304.piyush.verma@gmail.com> The RPM support in distutils as per i came to use wasn't that very effective, and recently i wrote extensions to the default module to produce multiple rpm's from a same spec file for my local use. The extension read from a configuration type script and used it to produce multiple rpms. and secondly by default on uninstalling the rpm, it didn delete the directories from {_sitelib} . Adding these by default, are they worth it? OR we have already got these implemented and i wasnt able to locate it. -- Regards, Piyush Verma Email:piyush.verma at gmail.com From fijall at gmail.com Sat Nov 10 21:47:11 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 10 Nov 2007 21:47:11 +0100 Subject: [Distutils] Easy install and zipimporter Message-ID: <693bc9ab0711101247h3d0f1c2h9c6794c72e738a83@mail.gmail.com> Hello, I've been implementing zipimport module recently for the pypy project and I came to the following problem: When zipimport is there, easy install is mangling with zipimport._zip_directory_cache, which is a) undocumented in importer protocol b) starts with an _ c) explodes when I compile pypy and import site.py (providing that I've got easy_install installed) Any propositions what to do here? Just providing _zip_directory_cache is not really an option, since it has no documentation and no unittests, which means it might change at any moment. Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20071110/112b462c/attachment.htm From pje at telecommunity.com Sat Nov 10 23:44:39 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Nov 2007 17:44:39 -0500 Subject: [Distutils] Easy install and zipimporter In-Reply-To: <693bc9ab0711101247h3d0f1c2h9c6794c72e738a83@mail.gmail.com > References: <693bc9ab0711101247h3d0f1c2h9c6794c72e738a83@mail.gmail.com> Message-ID: <20071110224552.A96AC3A4069@sparrow.telecommunity.com> At 09:47 PM 11/10/2007 +0100, Maciej Fijalkowski wrote: >Hello, > >I've been implementing zipimport module recently for the pypy >project and I came to the following problem: > >When zipimport is there, easy install is mangling with >zipimport._zip_directory_cache, which is >a) undocumented in importer protocol >b) starts with an _ >c) explodes when I compile pypy and import site.py (providing that >I've got easy_install installed) > >Any propositions what to do here? Just providing >_zip_directory_cache is not really an option, since it has no documentation >>> import zipimport >>> help(zipimport) Help on built-in module zipimport: NAME zipimport - zipimport provides support for importing Python modules from Zip archives. FILE (built-in) MODULE DOCS http://www.python.org/doc/current/lib/module-zipimport.html DESCRIPTION This module exports three objects: - zipimporter: a class; its constructor takes a path to a Zip archive. - ZipImporterError: exception raised by zipimporter objects. It's a subclass of ImportError, so it can be caught as ImportError, too. - _zip_directory_cache: a dict, mapping archive paths to zip directory info dicts, as used in zipimporter._files. >and no unittests, Well, the tests in test_zipimport *clear* it. >which means it might change at any moment. Why? Note, by the way, that as of Python 2.5, the pkgutil module also makes use of this variable. From rasky at develer.com Wed Nov 14 12:45:03 2007 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 14 Nov 2007 12:45:03 +0100 Subject: [Distutils] Third-party packages inside a project Message-ID: <473ADFBF.6040701@develer.com> Hello, for our Windows-only applications, we use a directory setup (working copy) like this: /lib /a /b /c /src boot.py pkg1/ pkg2/ pkg3/ /lib/{a,b,c} are third-party standard distutils packages downloaded from Internet. We stuff them within the working copy because we don't want any external dependency besides Python itself (basically, if you install Python and do a checkout, you must have everything needed to run the application). /src contains our own application source code. boot.py is basically the entry point, and the code is contained within several sub-packages. Third-party packages are installed into /src. Basically, after compilation, you end up with something like: /src boot.py a/ [installed] b/ [installed] c/ [installed] pkg1/ pkg2/ pkg3/ If you run boot.py from /src, this works fine (and has worked fine for years). In the past few months, we realized that this layout is probably going to break in the future (moving to newer python versions and making the application cross-platform) because: 1) It doesnt't work for third-party packages which use setuptools. Setuptools always creates and installs a .pth file (to the best of my understanding), and .pth files are not process inside /src, because it's not part of PYTHONPATH. I can't find a viable fix for this: * Creating a sitecustomize.py inside /src (which does something like site.addsitedir(os.getcwd()) does not work anymore since Python 2.5. It looks like it never worked under *NIX, and was "fixed" under Windows lately. sitecustomize.py is not imported from the current directory, probably for security reason (code injection). * Pointing PYTHONPATH to /src is a PITA, since developers always move among projects and different branches of the same project (so different working copy), and asking them to remember to change PYTHONPATH everytime is simply a recipe for disaster. * Using a "Custom Installation Locations" as suggested by EasyInstall docs is not a solution, because those are "tricks" to install python packages into an user-specific directory (instead of the system directory). I need a solution for a project-specific directory. 2) In Python 2.4, we were able to directly run a script in a subpackages (eg: a test). Basically, from /src, you could run "python pkg1/test_foo.py", and that would work correctly (= absolute imports like "import pkg2" were working as expected). In Python 2.5, this does not work anymore. I was wondering if someone could suggest a fix, especially for #1: I can't believe there's no way to ask distutils/setuptools to install a project-specific package (I am aware of the chapter "Custom Installation Locations" in EasyInstall docs, but that's more about installing packages at user-level instead of system-level; it doesn't cover the project-level which I need). Sorry for the long mail (I reckon it's better to err on the side of verbosity to make sure everything is clear). -- Giovanni Bajo From pje at telecommunity.com Wed Nov 14 13:19:04 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Nov 2007 07:19:04 -0500 Subject: [Distutils] Third-party packages inside a project In-Reply-To: <473ADFBF.6040701@develer.com> References: <473ADFBF.6040701@develer.com> Message-ID: <20071114122045.B4FC13A40A8@sparrow.telecommunity.com> At 12:45 PM 11/14/2007 +0100, Giovanni Bajo wrote: >I was wondering if someone could suggest a fix, especially for #1: I >can't believe there's no way to ask distutils/setuptools to install a >project-specific package (I am aware of the chapter "Custom Installation >Locations" in EasyInstall docs, but that's more about installing >packages at user-level instead of system-level; it doesn't cover the >project-level which I need). setup.py install --single-version-externally-managed --record=somefile ... The above incantation (along with whatever other options you want) will let you install setuptools-based packages "the old fashioned way". You will be responsible for installing the packages' dependencies in the same way, however. You'll also be responsible for any uninstallation prior to upgrading versions of those packages. Various .egg-info directories will be installed alongside the packages; you will need to leave these alone, until/unless you are uninstalling the corresponding package. From rasky at develer.com Wed Nov 14 14:37:08 2007 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 14 Nov 2007 14:37:08 +0100 Subject: [Distutils] Third-party packages inside a project In-Reply-To: <20071114122045.B4FC13A40A8@sparrow.telecommunity.com> References: <473ADFBF.6040701@develer.com> <20071114122045.B4FC13A40A8@sparrow.telecommunity.com> Message-ID: <473AFA04.30004@develer.com> On 11/14/2007 1:19 PM, Phillip J. Eby wrote: >> I was wondering if someone could suggest a fix, especially for #1: I >> can't believe there's no way to ask distutils/setuptools to install a >> project-specific package (I am aware of the chapter "Custom Installation >> Locations" in EasyInstall docs, but that's more about installing >> packages at user-level instead of system-level; it doesn't cover the >> project-level which I need). > > setup.py install --single-version-externally-managed --record=somefile ... > > The above incantation (along with whatever other options you want) will > let you install setuptools-based packages "the old fashioned way". You > will be responsible for installing the packages' dependencies in the > same way, however. You'll also be responsible for any uninstallation > prior to upgrading versions of those packages. That's fine since it's exactly how it was working before. I'm happy there is a "backward-compatibility" option, but it's a pity there's no way to fix it in a way that still allows to fully use setuptools. -- Giovanni Bajo From pje at telecommunity.com Wed Nov 14 15:00:47 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Nov 2007 09:00:47 -0500 Subject: [Distutils] Third-party packages inside a project In-Reply-To: <473AFA04.30004@develer.com> References: <473ADFBF.6040701@develer.com> <20071114122045.B4FC13A40A8@sparrow.telecommunity.com> <473AFA04.30004@develer.com> Message-ID: <20071114140104.9170C3A40AA@sparrow.telecommunity.com> At 02:37 PM 11/14/2007 +0100, Giovanni Bajo wrote: >On 11/14/2007 1:19 PM, Phillip J. Eby wrote: > >>>I was wondering if someone could suggest a fix, especially for #1: I >>>can't believe there's no way to ask distutils/setuptools to install a >>>project-specific package (I am aware of the chapter "Custom Installation >>>Locations" in EasyInstall docs, but that's more about installing >>>packages at user-level instead of system-level; it doesn't cover the >>>project-level which I need). >>setup.py install --single-version-externally-managed --record=somefile ... >>The above incantation (along with whatever other options you want) >>will let you install setuptools-based packages "the old fashioned >>way". You will be responsible for installing the packages' >>dependencies in the same way, however. You'll also be responsible >>for any uninstallation prior to upgrading versions of those packages. > >That's fine since it's exactly how it was working before. I'm happy >there is a "backward-compatibility" option, but it's a pity there's >no way to fix it in a way that still allows to fully use setuptools. Well, there is such a thing, I just assumed you didn't want it. The -m option to easy_install can install packages without a .pth file, but then to use them, your package must also be using setuptools and declare its dependencies, or explicitly use require() to add the packages to sys.path at runtime. From rasky at develer.com Wed Nov 14 16:02:49 2007 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 14 Nov 2007 16:02:49 +0100 Subject: [Distutils] Third-party packages inside a project In-Reply-To: <20071114140104.9170C3A40AA@sparrow.telecommunity.com> References: <473ADFBF.6040701@develer.com> <20071114122045.B4FC13A40A8@sparrow.telecommunity.com> <473AFA04.30004@develer.com> <20071114140104.9170C3A40AA@sparrow.telecommunity.com> Message-ID: <473B0E19.4080803@develer.com> On 11/14/2007 3:00 PM, Phillip J. Eby wrote: > At 02:37 PM 11/14/2007 +0100, Giovanni Bajo wrote: >> On 11/14/2007 1:19 PM, Phillip J. Eby wrote: >> >>>> I was wondering if someone could suggest a fix, especially for #1: I >>>> can't believe there's no way to ask distutils/setuptools to install a >>>> project-specific package (I am aware of the chapter "Custom >>>> Installation >>>> Locations" in EasyInstall docs, but that's more about installing >>>> packages at user-level instead of system-level; it doesn't cover the >>>> project-level which I need). >>> setup.py install --single-version-externally-managed >>> --record=somefile ... >>> The above incantation (along with whatever other options you want) >>> will let you install setuptools-based packages "the old fashioned >>> way". You will be responsible for installing the packages' >>> dependencies in the same way, however. You'll also be responsible >>> for any uninstallation prior to upgrading versions of those packages. >> >> That's fine since it's exactly how it was working before. I'm happy >> there is a "backward-compatibility" option, but it's a pity there's no >> way to fix it in a way that still allows to fully use setuptools. > > Well, there is such a thing, I just assumed you didn't want it. The -m > option to easy_install can install packages without a .pth file, but > then to use them, your package must also be using setuptools and declare > its dependencies, or explicitly use require() to add the packages to > sys.path at runtime. You're right. --single-version-externally-managed seems like a better fit for my usecase. After all, I don't have to manage multiple versions of the same package within the same project. Thanks! -- Giovanni Bajo From zooko at zooko.com Thu Nov 15 01:29:34 2007 From: zooko at zooko.com (zooko) Date: Wed, 14 Nov 2007 17:29:34 -0700 Subject: [Distutils] setuptools bug report: include_package_data is on by default if there is a find-files plugin? In-Reply-To: <20071110015649.8C03F3A405B@sparrow.telecommunity.com> References: <68234720-6FB3-485A-95A3-50B703D0B56B@zooko.com> <20071110015649.8C03F3A405B@sparrow.telecommunity.com> Message-ID: At 04:57 PM 11/9/2007 -0700, zooko wrote: > Folks: > > If my setuptools_darcs plugin [1] is installed, then all files under > revision control get included in packages, even if > include_package_data=True was not passed. On Nov 9, 2007, at 6:56 PM, Phillip J. Eby wrote: > By "included in packages", do you mean included in eggs and > installed, or do you mean included in source distributions? I meant the latter -- I didn't test the former. Regards, Zooko From pje at telecommunity.com Thu Nov 15 02:05:50 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Nov 2007 20:05:50 -0500 Subject: [Distutils] setuptools bug report: include_package_data is on by default if there is a find-files plugin? In-Reply-To: References: <68234720-6FB3-485A-95A3-50B703D0B56B@zooko.com> <20071110015649.8C03F3A405B@sparrow.telecommunity.com> Message-ID: <20071115010551.345E53A408C@sparrow.telecommunity.com> At 05:29 PM 11/14/2007 -0700, zooko wrote: >At 04:57 PM 11/9/2007 -0700, zooko wrote: >>Folks: >> >>If my setuptools_darcs plugin [1] is installed, then all files under >>revision control get included in packages, even if >>include_package_data=True was not passed. > >On Nov 9, 2007, at 6:56 PM, Phillip J. Eby wrote: > >>By "included in packages", do you mean included in eggs and >>installed, or do you mean included in source distributions? > >I meant the latter -- I didn't test the former. That's "as intended" -- a source distribution always includes all the source, whether the files in question will be installed or not. From tseaver at palladion.com Thu Nov 15 17:35:21 2007 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 15 Nov 2007 11:35:21 -0500 Subject: [Distutils] buildout, zc.recipe.testrunner and tests_require In-Reply-To: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> References: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> Message-ID: <473C7549.9060506@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > On Mar 11, 2007, at 11:08 AM, Nathan R. Yergler wrote: > ... >> I tried using zc.recipe.testbrowser, thinking that maybe it'd look at >> the tests_require for the target eggs, but no such luck. > > The tests_require option is only visible to the setup script. It > doesn't cause any data to be included in egg info. It is an > attractive nuisance IMO. I hope the attached patch (against the head of the setuptools 0.6 branch) makes it less of a nuisance. The patch adds an egg-info file entry point which writes the 'tests_require' value to a fill in the .egg-info directory. It would be trivial to add another which dumped the 'test_suite' info as well: perhaps a single ConfigParser-style file with two keys would have more parsimony. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPHVJ+gerLs4ltQ4RAlGHAJ4o4tJrYInQkWB0Xv6KB3oNhAscHQCg0Kjl 9XLihjJaaU6coaBo/dpwkvE= =zSQH -----END PGP SIGNATURE----- -------------- next part -------------- A non-text attachment was scrubbed... Name: setuptools-add_tests_require_file.patch Type: text/x-patch Size: 2715 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20071115/fe0b1903/attachment.bin From tseaver at palladion.com Thu Nov 15 17:35:21 2007 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 15 Nov 2007 11:35:21 -0500 Subject: [Distutils] buildout, zc.recipe.testrunner and tests_require In-Reply-To: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> References: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> Message-ID: <473C7549.9060506@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > On Mar 11, 2007, at 11:08 AM, Nathan R. Yergler wrote: > ... >> I tried using zc.recipe.testbrowser, thinking that maybe it'd look at >> the tests_require for the target eggs, but no such luck. > > The tests_require option is only visible to the setup script. It > doesn't cause any data to be included in egg info. It is an > attractive nuisance IMO. I hope the attached patch (against the head of the setuptools 0.6 branch) makes it less of a nuisance. The patch adds an egg-info file entry point which writes the 'tests_require' value to a fill in the .egg-info directory. It would be trivial to add another which dumped the 'test_suite' info as well: perhaps a single ConfigParser-style file with two keys would have more parsimony. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPHVJ+gerLs4ltQ4RAlGHAJ4o4tJrYInQkWB0Xv6KB3oNhAscHQCg0Kjl 9XLihjJaaU6coaBo/dpwkvE= =zSQH -----END PGP SIGNATURE----- -------------- next part -------------- A non-text attachment was scrubbed... Name: setuptools-add_tests_require_file.patch Type: text/x-patch Size: 2715 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20071115/fe0b1903/attachment-0001.bin From pje at telecommunity.com Thu Nov 15 17:47:25 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Nov 2007 11:47:25 -0500 Subject: [Distutils] buildout, zc.recipe.testrunner and tests_require In-Reply-To: <473C7549.9060506@palladion.com> References: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> <473C7549.9060506@palladion.com> Message-ID: <20071115164725.9922A3A40D9@sparrow.telecommunity.com> At 11:35 AM 11/15/2007 -0500, Tres Seaver wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Jim Fulton wrote: > > On Mar 11, 2007, at 11:08 AM, Nathan R. Yergler wrote: > > ... > >> I tried using zc.recipe.testbrowser, thinking that maybe it'd look at > >> the tests_require for the target eggs, but no such luck. > > > > The tests_require option is only visible to the setup script. It > > doesn't cause any data to be included in egg info. It is an > > attractive nuisance IMO. > >I hope the attached patch (against the head of the setuptools 0.6 >branch) makes it less of a nuisance. The patch adds an egg-info file >entry point which writes the 'tests_require' value to a fill in the >.egg-info directory. Interesting, and worth considering. Meanwhile, note that you can also distribute this as a plugin, or register the entry point in zc.buildout. That is, you don't need this to be a patch in order to get the same behavior in a given Python installation. From tseaver at palladion.com Thu Nov 15 18:10:28 2007 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 15 Nov 2007 12:10:28 -0500 Subject: [Distutils] buildout, zc.recipe.testrunner and tests_require In-Reply-To: <20071115164725.9922A3A40D9@sparrow.telecommunity.com> References: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> <473C7549.9060506@palladion.com> <20071115164725.9922A3A40D9@sparrow.telecommunity.com> Message-ID: <473C7D84.8000609@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 11:35 AM 11/15/2007 -0500, Tres Seaver wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Jim Fulton wrote: >>> On Mar 11, 2007, at 11:08 AM, Nathan R. Yergler wrote: >>> ... >>>> I tried using zc.recipe.testbrowser, thinking that maybe it'd look at >>>> the tests_require for the target eggs, but no such luck. >>> The tests_require option is only visible to the setup script. It >>> doesn't cause any data to be included in egg info. It is an >>> attractive nuisance IMO. >> I hope the attached patch (against the head of the setuptools 0.6 >> branch) makes it less of a nuisance. The patch adds an egg-info file >> entry point which writes the 'tests_require' value to a fill in the >> .egg-info directory. > > Interesting, and worth considering. Meanwhile, note that you can > also distribute this as a plugin, or register the entry point in > zc.buildout. That is, you don't need this to be a patch in order to > get the same behavior in a given Python installation. Thanks, I realized that I could distribute it separately about 3 seconds after hitting "Send". :) I just uploaded 'eggtestinfo-.0.1.tar.gz' to the cheeseshop. It exports both the 'test_suite' and the 'tests_require' information to a single 'test_info.txt' file. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPH2E+gerLs4ltQ4RAmUYAJ9xv6ounUdYV8FY1IIA3aNuGXCulQCfZTrh Y4FIeBAYP+dEGkwAMLuRw6s= =83wG -----END PGP SIGNATURE----- From tseaver at palladion.com Thu Nov 15 18:10:28 2007 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 15 Nov 2007 12:10:28 -0500 Subject: [Distutils] buildout, zc.recipe.testrunner and tests_require In-Reply-To: <20071115164725.9922A3A40D9@sparrow.telecommunity.com> References: <13353241-2CB2-4DEB-8277-E280AF3F77D6@zope.com> <473C7549.9060506@palladion.com> <20071115164725.9922A3A40D9@sparrow.telecommunity.com> Message-ID: <473C7D84.8000609@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 11:35 AM 11/15/2007 -0500, Tres Seaver wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Jim Fulton wrote: >>> On Mar 11, 2007, at 11:08 AM, Nathan R. Yergler wrote: >>> ... >>>> I tried using zc.recipe.testbrowser, thinking that maybe it'd look at >>>> the tests_require for the target eggs, but no such luck. >>> The tests_require option is only visible to the setup script. It >>> doesn't cause any data to be included in egg info. It is an >>> attractive nuisance IMO. >> I hope the attached patch (against the head of the setuptools 0.6 >> branch) makes it less of a nuisance. The patch adds an egg-info file >> entry point which writes the 'tests_require' value to a fill in the >> .egg-info directory. > > Interesting, and worth considering. Meanwhile, note that you can > also distribute this as a plugin, or register the entry point in > zc.buildout. That is, you don't need this to be a patch in order to > get the same behavior in a given Python installation. Thanks, I realized that I could distribute it separately about 3 seconds after hitting "Send". :) I just uploaded 'eggtestinfo-.0.1.tar.gz' to the cheeseshop. It exports both the 'test_suite' and the 'tests_require' information to a single 'test_info.txt' file. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPH2E+gerLs4ltQ4RAmUYAJ9xv6ounUdYV8FY1IIA3aNuGXCulQCfZTrh Y4FIeBAYP+dEGkwAMLuRw6s= =83wG -----END PGP SIGNATURE----- From TParvez at bop.gov Fri Nov 16 17:01:47 2007 From: TParvez at bop.gov (Tania Fahima Parvez) Date: Fri, 16 Nov 2007 11:01:47 -0500 Subject: [Distutils] problem with setuptool install Message-ID: <473D789B020000670000D350@gwmail.bop.gov> Hi, I downloaded the setuptools-0.6c7 version of setupttols. I run "python ez_setup.py" and it comes back with the following message back: "Setuptools version 0.6c7 or greater has been installed." I have downloade the setupttols egg ( setuptools-0.6c7-py2.4.egg ) in this directory also. When I try to run the "python easy_install.py setuptools-0.6c7-py2.4.egg" command , I get the following error in the last lines: File "/downloads/www.python.org/setuptools-0.6c7/setuptools/archive_util.py", line 154, in unpack_zipfile data = z.read(info.filename) File "/usr/local/lib/python2.4/zipfile.py", line 353, in read raise RuntimeError, \ RuntimeError: De-compression requires the (missing) zlib module I have all the zlib libraries installed an in the path. Please help! Thanks. Tania From pje at telecommunity.com Fri Nov 16 17:36:51 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Nov 2007 11:36:51 -0500 Subject: [Distutils] problem with setuptool install In-Reply-To: <473D789B020000670000D350@gwmail.bop.gov> References: <473D789B020000670000D350@gwmail.bop.gov> Message-ID: <20071116163728.381743A40A8@sparrow.telecommunity.com> At 11:01 AM 11/16/2007 -0500, Tania Fahima Parvez wrote: >Hi, > >I downloaded the setuptools-0.6c7 version of setupttols. I run >"python ez_setup.py" and it comes back with the following message >back: "Setuptools version 0.6c7 or greater has been installed." I >have downloade the setupttols egg ( setuptools-0.6c7-py2.4.egg ) in >this directory also. When I try to run the "python easy_install.py >setuptools-0.6c7-py2.4.egg" command , I get the following error in >the last lines: > File > "/downloads/www.python.org/setuptools-0.6c7/setuptools/archive_util.py", > line 154, in unpack_zipfile > data = z.read(info.filename) > File "/usr/local/lib/python2.4/zipfile.py", line 353, in read > raise RuntimeError, \ >RuntimeError: De-compression requires the (missing) zlib module > >I have all the zlib libraries installed an in the path. Is there perhaps a "python-zlib" package provided by your OS? That sounds like what's missing. From pje at telecommunity.com Fri Nov 16 20:51:59 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Nov 2007 14:51:59 -0500 Subject: [Distutils] problem with setuptool install In-Reply-To: <473D8FAB020000670000D360@gwmail.bop.gov> References: <473D789B020000670000D350@gwmail.bop.gov> <20071116163728.381743A40A8@sparrow.telecommunity.com> <473D8FAB020000670000D360@gwmail.bop.gov> Message-ID: <20071116195159.548433A408C@sparrow.telecommunity.com> At 12:40 PM 11/16/2007 -0500, Tania Fahima Parvez wrote: >I downloaded and installed the python-zlib rpm. Stll the same result. > > >>> "Phillip J. Eby" 11/16/2007 11:36 AM >>> >At 11:01 AM 11/16/2007 -0500, Tania Fahima Parvez wrote: > >Hi, > > > >I downloaded the setuptools-0.6c7 version of setupttols. I run > >"python ez_setup.py" and it comes back with the following message > >back: "Setuptools version 0.6c7 or greater has been installed." I > >have downloade the setupttols egg ( setuptools-0.6c7-py2.4.egg ) in > >this directory also. When I try to run the "python easy_install.py > >setuptools-0.6c7-py2.4.egg" command , I get the following error in > >the last lines: > > File > > "/downloads/www.python.org/setuptools-0.6c7/setuptools/archive_util.py", > > line 154, in unpack_zipfile > > data = z.read(info.filename) > > File "/usr/local/lib/python2.4/zipfile.py", line 353, in read > > raise RuntimeError, \ > >RuntimeError: De-compression requires the (missing) zlib module > > > >I have all the zlib libraries installed an in the path. > >Is there perhaps a "python-zlib" package provided by your OS? That >sounds like what's missing. You mean, you still get "De-compression requires the missing zlib module"? Try this: python -c "import zlib" Does that work, or produce an error message? (By the way, please direct your replies to the distutils-sig mailing list, as I don't do off-list support.) From ianb at openplans.org Mon Nov 19 19:15:39 2007 From: ianb at openplans.org (Ian Bicking) Date: Mon, 19 Nov 2007 12:15:39 -0600 Subject: [Distutils] easy_install prefering filename over #egg In-Reply-To: <0000103793@bizling.com> References: <473A2066.5090403@openplans.org> <0000103793@bizling.com> Message-ID: <4741D2CB.70900@openplans.org> We've been building some custom eggs for the lxml trunk and we uploaded the packages to our wiki, which replaced the .'s with -'s. This messed up the filenames, so we added #egg in an effort to correct that: http://www.openplans.org/projects/opencore/dependencies/lxml-2-0alpha5-py2-4-linux-x86_64.egg#egg=lxml-2.0alpha5-py2.4-linux-x86_64 But it seems that easy_install prefers the base name, not the fragment, and so it parsed this as an lxml 2 egg, with 0alpha5blahblahblah as the platform(?)... I'm not sure exactly how it parsed it, but it got it wrong. It seems like easy_install should prefer fragments over the filename itself, as the fragments will only be there if someone deliberately adds them. Ian From pje at telecommunity.com Mon Nov 19 19:18:13 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 19 Nov 2007 13:18:13 -0500 Subject: [Distutils] installation preferences proposal, volunteer to help implementation In-Reply-To: <8928d4e90711190949j54b400f7o1973b881a49a88cf@mail.gmail.co m> References: <8928d4e90711120750g479bd2bjfa62356d34ca52d0@mail.gmail.com> <20071119173152.77BB33A405E@sparrow.telecommunity.com> <8928d4e90711190949j54b400f7o1973b881a49a88cf@mail.gmail.com> Message-ID: <20071119181815.A38813A405E@sparrow.telecommunity.com> At 06:49 PM 11/19/2007 +0100, Martijn Faassen wrote: >Hey, > >On Nov 19, 2007 6:31 PM, Phillip J. Eby wrote: > > > > Yes. What's needed is a way to implement "or" syntax for > > requirements, so that we can say "Foo<1.2|Bar==1.3" -- and therefore > > also say e.g. "B==1.3.1|B>=1.3". > >I'm trying to understand why you'd want to express "Foo < 1.2 | Bar == >1.3" - the library with this dependency would try to import Bar and if >this fails, it'll try importing Foo? > >Why do you use bitwise '|' for this, not 'or' as it used in Python >expressions? Because it's not used by the current syntax, whereas "or" is a valid package name. >One thing that is not very pretty about 'or' syntax for version >preferences is the repetition of the package name: > >zope.interface | zope.interface == 1.2.3 While syntactically valid, the above is meaningless, since the latter requirement implies the former. The "|" is being used to mean "ordered choice" (I suppose "||" would be better in some respects). >I'd like a shortcut syntax for this, something like: > >zope.interface (1.2.3) > >We could let that be sugar that expands to the same underlying 'or' syntax. I'm wary of adding too much syntax to requirements. The meaning of the parentheses in the above is entirely inscrutable. > > In practical terms, this means that what is now Requirement must > > become a VersionSpec or something of that sort, and a Requirement > > become an ordered collection of VersionSpec objects. > > > > Parsing, naturally, will also change a good bit, as will the > > resolution algorithm, since right now there is no provision for > > fallbacks. Finally, easy_install's core searches will change as well. > > > > I don't expect this will be an easy change, even for me. And it's > > definitely not going into the 0.6 line, because dependencies in the > > new syntax won't be readable by older versions of setuptools. > >Yes, I understand this needs to be in 0.7. I do want some basic idea >that there *will* be a 0.7 in the forseeable future if this work >happens, though, on the order of months, not years. Of course it's >understood that this would depend on me actually doing the work. :) > > > You might wonder why we can't implement something simpler that's more > > narrowly tailored to what you need, but in truth it wouldn't be much > > narrowed. The resolution algorithm and easy_install are probably the > > tough nuts to crack (especially without tests); by comparison the > > parsing and object issues should be pretty simple by comparison > > (especially with tests). > >Thanks for the hints. I wasn't expecting it to be easy, and I realize >that a tailored approach wouldn't make it much simpler (even though I >would appreciate a syntax for the particular use case). > >I'm not expecting this to be easy, I just want to give it a shot. I >will be writing tests if it's at all feasible. I like tests. > >Regards, > >Martijn From ianb at colorstudy.com Mon Nov 19 19:18:36 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 19 Nov 2007 12:18:36 -0600 Subject: [Distutils] easy_install prefering filename over #egg In-Reply-To: <4741D2CB.70900@openplans.org> References: <473A2066.5090403@openplans.org> <0000103793@bizling.com> <4741D2CB.70900@openplans.org> Message-ID: <4741D37C.1010208@colorstudy.com> Ian Bicking wrote: > We've been building some custom eggs for the lxml trunk and we uploaded > the packages to our wiki, which replaced the .'s with -'s. This messed > up the filenames, so we added #egg in an effort to correct that: > > http://www.openplans.org/projects/opencore/dependencies/lxml-2-0alpha5-py2-4-linux-x86_64.egg#egg=lxml-2.0alpha5-py2.4-linux-x86_64 > > But it seems that easy_install prefers the base name, not the fragment, > and so it parsed this as an lxml 2 egg, with 0alpha5blahblahblah as the > platform(?)... I'm not sure exactly how it parsed it, but it got it wrong. > > It seems like easy_install should prefer fragments over the filename > itself, as the fragments will only be there if someone deliberately adds > them. Actually, it just occurred to me that there's a link on the page with the #egg, and one without, and probably easy_install was just looking at the link without the fragment. So maybe our plan to use #egg to correct the link just wasn't going to work with the site. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From faassen at startifact.com Mon Nov 19 19:38:12 2007 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 19 Nov 2007 19:38:12 +0100 Subject: [Distutils] installation preferences proposal, volunteer to help implementation In-Reply-To: <20071119181815.A38813A405E@sparrow.telecommunity.com> References: <8928d4e90711120750g479bd2bjfa62356d34ca52d0@mail.gmail.com> <20071119173152.77BB33A405E@sparrow.telecommunity.com> <8928d4e90711190949j54b400f7o1973b881a49a88cf@mail.gmail.com> <20071119181815.A38813A405E@sparrow.telecommunity.com> Message-ID: <8928d4e90711191038p1b3d418aq45806f0815ea1214@mail.gmail.com> Hello, On Nov 19, 2007 7:18 PM, Phillip J. Eby wrote: [snip] > >One thing that is not very pretty about 'or' syntax for version > >preferences is the repetition of the package name: > > > >zope.interface | zope.interface == 1.2.3 > > While syntactically valid, the above is meaningless, since the latter > requirement implies the former. The "|" is being used to mean > "ordered choice" (I suppose "||" would be better in some respects). Ah, I didn't realized you intended this to mean ordered choice. I was thinking about a resolution algorithm that would go for the most specific choice first for any package. I will have to think about this. Since you say ordered choice, would you say it's this? zope.interface == 1.2.3 | zope.interface? Not meaningless, I hope? > >I'd like a shortcut syntax for this, something like: > > > >zope.interface (1.2.3) > > > >We could let that be sugar that expands to the same underlying 'or' syntax. > > I'm wary of adding too much syntax to requirements. The meaning of > the parentheses in the above is entirely inscrutable. I'd suggest the syntactic sugar helps making it clearer what's intended: give me zope.interface, and 1.2.3 I'll guarantee you will work. zope.interface == 1.2.3 | zope.interface is not intuitively clear either in its intent. A special syntax can signal the special intent. Regards, Martijn From pje at telecommunity.com Mon Nov 19 20:00:07 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 19 Nov 2007 14:00:07 -0500 Subject: [Distutils] easy_install prefering filename over #egg In-Reply-To: <4741D2CB.70900@openplans.org> References: <473A2066.5090403@openplans.org> <0000103793@bizling.com> <4741D2CB.70900@openplans.org> Message-ID: <20071119190009.923F63A405E@sparrow.telecommunity.com> At 12:15 PM 11/19/2007 -0600, Ian Bicking wrote: >We've been building some custom eggs for the lxml trunk and we uploaded >the packages to our wiki, which replaced the .'s with -'s. This messed >up the filenames, Stop right there. That's the problem. You also messed up the #egg part, by adding extra junk into the version, but that's irrelevant. The simple fact is that egg files cannot be renamed without affecting their meaning. From gotcha at bubblenet.be Tue Nov 20 16:00:47 2007 From: gotcha at bubblenet.be (Godefroid Chapelle) Date: Tue, 20 Nov 2007 16:00:47 +0100 Subject: [Distutils] bug in archive_util.unpack_archive ? Message-ID: <4742F69F.9060907@bubblenet.be> Hi, I think there is a bug in archive_util.unpack_archive. The following procedure has been tested with setuptools 0.6c7 under MacOSX 10.4.10, Mac OSX 10.5.1 and Ubuntu 7.10 Download archive : http://switch.dl.sourceforge.net/sourceforge/textindexng/TextIndexNG3-3.1.8.tar.gz Test it : tar -xzf TextIndexNG3-3.1.8.tar.gz cd TextIndexNG3 ls -al total 64 drwxr-xr-x 21 gotcha gotcha 714 Mar 11 2006 . drwxr-xr-x 9 gotcha gotcha 306 Nov 20 15:53 .. drwxr-xr-x 5 gotcha gotcha 170 Mar 11 2006 CVS drwxr-xr-x 4 gotcha gotcha 136 Mar 11 2006 Extensions -rw-r--r-- 1 gotcha gotcha 658 Jun 4 2005 Makefile -rw-r--r-- 1 gotcha gotcha 6056 Mar 11 2006 TextIndexNG3.py -rw-r--r-- 1 gotcha gotcha 2226 Sep 21 2005 __init__.py drwxr-xr-x 8 gotcha gotcha 272 Mar 11 2006 adapters -rw-r--r-- 1 gotcha gotcha 3444 Dec 8 2005 browser.py -rw-r--r-- 1 gotcha gotcha 2034 Dec 7 2005 configure.zcml drwxr-xr-x 6 gotcha gotcha 204 Mar 11 2006 doc drwxr-xr-x 10 gotcha gotcha 340 Mar 11 2006 extension_modules -rw-r--r-- 1 gotcha gotcha 320 May 1 2005 header.txt drwxr-xr-x 5 gotcha gotcha 170 Mar 11 2006 i18n drwxr-xr-x 5 gotcha gotcha 170 Mar 11 2006 interfaces drwxr-xr-x 13 gotcha gotcha 442 Mar 11 2006 pt drwxr-xr-x 5 gotcha gotcha 170 Mar 11 2006 skins drwxr-xr-x 4 gotcha gotcha 136 Mar 11 2006 src drwxr-xr-x 8 gotcha gotcha 272 Mar 11 2006 tests -rw-r--r-- 1 gotcha gotcha 6 Mar 11 2006 version.txt drwxr-xr-x 3 gotcha gotcha 102 Mar 11 2006 www Unpack it with setuptools function : python2.4 >>> from setuptools import archive_util >>> archive_util.unpack_archive('TextIndexNG3-3.1.8.tar.gz', 'dest') cd dest/ ls -al TextIndexNG3/ total 8 drwxrwxr-x 6 gotcha gotcha 204 Nov 20 15:57 . drwxrwxrwx 3 gotcha gotcha 102 Nov 20 15:57 .. -rw-rw-r-- 1 gotcha gotcha 3444 Dec 8 2005 browser.py drwxrwxr-x 8 gotcha gotcha 272 Nov 20 15:57 extension_modules drwxrwxr-x 5 gotcha gotcha 170 Nov 20 15:57 i18n drwxrwxr-x 13 gotcha gotcha 442 Nov 20 15:57 pt As you can see the output with unpack_archive in uncomplete. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be From gotcha at bubblenet.be Tue Nov 20 16:16:18 2007 From: gotcha at bubblenet.be (Godefroid Chapelle) Date: Tue, 20 Nov 2007 16:16:18 +0100 Subject: [Distutils] bug in archive_util.unpack_archive ? In-Reply-To: <4742F69F.9060907@bubblenet.be> References: <4742F69F.9060907@bubblenet.be> Message-ID: <4742FA42.20202@bubblenet.be> Godefroid Chapelle wrote: > Hi, > > I think there is a bug in archive_util.unpack_archive. > > The following procedure has been tested with setuptools 0.6c7 > under MacOSX 10.4.10, Mac OSX 10.5.1 and Ubuntu 7.10 Python version was not very explicit in the message. The bug actually happens with 2.4 and not with 2.5. We found out it is a bug in python tarfile module itself in 2.4, which seems to be fixed in 2.5. Sorry for the noise, (might be helpful through archives searches though :-S) -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be From dangoor at gmail.com Mon Nov 26 17:40:21 2007 From: dangoor at gmail.com (Kevin Dangoor) Date: Mon, 26 Nov 2007 11:40:21 -0500 Subject: [Distutils] setuptools dependency_links in dependent packages Message-ID: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> I just tried installing one of the many, cool, interrelated zc.* packages, zc.authorizedotnet. That package depends on zc.ssl. zc.ssl has the following in setup.py: dependency_links = ['http://download.zope.org/distribution/'], Running easy_install zc.authorizedotnet fails because it doesn't have that entry, and the installation of zc.ssl does not pick it up. Installing zc.ssl *and then* zc.authorizedotnet works fine. I'd call that a bug, because zc.authorizedotnet should not need to know that one of its required packages has a special home for its code. Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20071126/a1c1c260/attachment.htm From fdrake at acm.org Tue Nov 27 06:42:07 2007 From: fdrake at acm.org (Fred Drake) Date: Tue, 27 Nov 2007 00:42:07 -0500 Subject: [Distutils] setuptools dependency_links in dependent packages In-Reply-To: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> References: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> Message-ID: <51A83F6E-F664-4F99-AE58-6E98D509D48E@acm.org> On Nov 26, 2007, at 11:40 AM, Kevin Dangoor wrote: > That package depends on zc.ssl. zc.ssl has the following in setup.py: > > dependency_links = ['http://download.zope.org/distribution/'], ... > I'd call that a bug, because zc.authorizedotnet should not need to > know that one of its required packages has a special home for its > code. Are you suggesting that any use of dependency_links is a bug? I'd be quite sympathetic. -Fred -- Fred Drake From pje at telecommunity.com Tue Nov 27 23:05:14 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 27 Nov 2007 17:05:14 -0500 Subject: [Distutils] setuptools dependency_links in dependent packages In-Reply-To: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> References: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> Message-ID: <20071127221557.6CA453A405E@sparrow.telecommunity.com> At 11:40 AM 11/26/2007 -0500, Kevin Dangoor wrote: >I just tried installing one of the many, cool, interrelated zc.* >packages, zc.authorizedotnet. > >That package depends on zc.ssl. zc.ssl has the following in setup.py: > > >dependency_links = >['http://download.zope.org/distribution/'], > > > > >Running easy_install zc.authorizedotnet fails because it doesn't >have that entry, and the installation of zc.ssl does not pick it up. >Installing zc.ssl *and then* zc.authorizedotnet works fine. > >I'd call that a bug, because zc.authorizedotnet should not need to >know that one of its required packages has a special home for its code. I must confess I don't even understand what you're saying is happening. Could you explain at least what you mean by "fails" and "does not pick it up"? The console output from the commands (and the steps to reproduce it) would also be helpful. From tseaver at palladion.com Tue Nov 27 23:51:44 2007 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 27 Nov 2007 17:51:44 -0500 Subject: [Distutils] setuptools dependency_links in dependent packages In-Reply-To: <20071127221557.6CA453A405E@sparrow.telecommunity.com> References: <86A7AE1C-C41B-4B4A-83A2-325AC8F47A8B@gmail.com> <20071127221557.6CA453A405E@sparrow.telecommunity.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 11:40 AM 11/26/2007 -0500, Kevin Dangoor wrote: >> I just tried installing one of the many, cool, interrelated zc.* >> packages, zc.authorizedotnet. >> >> That package depends on zc.ssl. zc.ssl has the following in setup.py: >> >> >> dependency_links = >> ['http://download.zope.org/distribution/'], >> >> >> >> >> Running easy_install zc.authorizedotnet fails because it doesn't >> have that entry, and the installation of zc.ssl does not pick it up. >> Installing zc.ssl *and then* zc.authorizedotnet works fine. >> >> I'd call that a bug, because zc.authorizedotnet should not need to >> know that one of its required packages has a special home for its code. > > I must confess I don't even understand what you're saying is > happening. Could you explain at least what you mean by "fails" and > "does not pick it up"? The console output from the commands (and the > steps to reproduce it) would also be helpful. I read the report as saying that 'dependency_links' mentioned by distributions satisfying user-specified requirements were consulted, but that those mentioned by dependencies of those distributions were ignored. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHTJ+A+gerLs4ltQ4RAl0EAKCIKGz6XmI9A4Bif8CKM9+ySu8MRQCfYdmI 9Xu/HSIuf1E8vqVFhiaOTuE= =NPDd -----END PGP SIGNATURE----- From cz at gocept.com Wed Nov 28 18:41:32 2007 From: cz at gocept.com (Christian Zagrodnick) Date: Wed, 28 Nov 2007 18:41:32 +0100 Subject: [Distutils] zc.buildout: Build python in buildout Message-ID: Hey, building python with buildout is not very hard (see below). When installing sections buildout apparently uses the built python to get dependencies and compile the C extensions if any. Only the dependencies of the recipes are fetched via the bootstrap python. But with C extensions this is the problem. For instance zc.zodbrecipes depend on ZODB3 which is then compiled with the wrong python. This would not be much of a problem but I then get those errors: ImportError: /tmp/test/eggs/ZODB3-3.8.0b4-py2.4-linux-x86_64.egg/persistent/cPersistence.so: undefined symbol: PyUnicodeUCS4_AsEncodedString Any clue what to do? I suppose the only way is to make the two pythons eat the same .so files? This does the trick of building python: [buildout] python = python [python] recipe = zc.recipe.cmmi url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2 executable = ${buildout:directory}/parts/python/bin/python2.4 -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From jim at zope.com Wed Nov 28 21:48:13 2007 From: jim at zope.com (Jim Fulton) Date: Wed, 28 Nov 2007 15:48:13 -0500 Subject: [Distutils] zc.buildout: Build python in buildout In-Reply-To: References: Message-ID: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> On Nov 28, 2007, at 12:41 PM, Christian Zagrodnick wrote: > Hey, > > building python with buildout is not very hard (see below). > > When installing sections buildout apparently uses the built python to > get dependencies and compile the C extensions if any. > > Only the dependencies of the recipes are fetched via the bootstrap > python. But with C extensions this is the problem. For instance > zc.zodbrecipes depend on ZODB3 which is then compiled with the wrong > python. > > This would not be much of a problem but I then get those errors: > > ImportError: > /tmp/test/eggs/ZODB3-3.8.0b4-py2.4-linux-x86_64.egg/persistent/ > cPersistence.so: > undefined symbol: PyUnicodeUCS4_AsEncodedString Ouch. I guess the Python used to run the buildout and the Python built by the buildout are the same Python version. > Any clue what to do? I have clues, although I'm not sure they'll be useful. If the Python used to run the buildout and the Python built by the buildout were different versions, then you'd be OK. I suspect that a better solution would be to find a way to bootstrap the buildout in a way that included building Python as part of the bootstrapping process. > I suppose the only way is to make the two pythons > eat the same .so files? I don't know how you'd do that. Jim -- Jim Fulton Zope Corporation From cz at gocept.com Thu Nov 29 08:04:06 2007 From: cz at gocept.com (Christian Zagrodnick) Date: Thu, 29 Nov 2007 08:04:06 +0100 Subject: [Distutils] zc.buildout: Build python in buildout References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: On 2007-11-28 21:48:13 +0100, Jim Fulton said: > > On Nov 28, 2007, at 12:41 PM, Christian Zagrodnick wrote: > >> Hey, >> >> building python with buildout is not very hard (see below). >> >> When installing sections buildout apparently uses the built python to >> get dependencies and compile the C extensions if any. >> >> Only the dependencies of the recipes are fetched via the bootstrap >> python. But with C extensions this is the problem. For instance >> zc.zodbrecipes depend on ZODB3 which is then compiled with the wrong >> python. >> >> This would not be much of a problem but I then get those errors: >> >> ImportError: >> /tmp/test/eggs/ZODB3-3.8.0b4-py2.4-linux-x86_64.egg/persistent/ >> cPersistence.so: >> undefined symbol: PyUnicodeUCS4_AsEncodedString > > Ouch. > > I guess the Python used to run the buildout and the Python built by > the buildout are the same Python version. Yes. > >> Any clue what to do? > > I have clues, although I'm not sure they'll be useful. If the Python > used to run the buildout and the Python built by the buildout were > different versions, then you'd be OK. Yes. That was my first guess, too. The problem is that there is only a sort of broken python2.4 available there. So I had to come up with something else. :/ > > I suspect that a better solution would be to find a way to bootstrap > the buildout in a way that included building Python as part of the > bootstrapping process. That would certainly the best approach. Only that I have no idea how to do that. :( > >> I suppose the only way is to make the two pythons >> eat the same .so files? > > I don't know how you'd do that. Ubuntu's Python is compiled with --enable-uncide=ucs4 which leads to the incompatibiliby. When compiling the Python in buildout with this option everything works fine (so far anyway). So, no, there is no general approach to fix the incompatibility, yet. Only on specific for this one Server. Anyway, thanks for the ideas. Regards, -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From jim at zope.com Thu Nov 29 16:47:28 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 29 Nov 2007 10:47:28 -0500 Subject: [Distutils] zc.buildout: Build python in buildout In-Reply-To: References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: <0D5642E2-DFCA-44BD-9316-EEA9815930BF@zope.com> On Nov 29, 2007, at 2:04 AM, Christian Zagrodnick wrote: > On 2007-11-28 21:48:13 +0100, Jim Fulton said: >> >> I suspect that a better solution would be to find a way to bootstrap >> the buildout in a way that included building Python as part of the >> bootstrapping process. > > That would certainly the best approach. Only that I have no idea how > to > do that. :( Write a variation on the standard bootstrap script that first builds Python. It could do so directly, or run run buildout to do it. Then, after building Python, redo the bootstrap using this new Python. You could do this by invoking the buildout with the new Python: path_to_new_python bin/buildout bootstrap Jim -- Jim Fulton Zope Corporation From thomas at thomas-lotze.de Fri Nov 30 06:42:25 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 30 Nov 2007 06:42:25 +0100 Subject: [Distutils] zc.buildout: Build python in buildout References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: Christian Zagrodnick wrote: > On 2007-11-28 21:48:13 +0100, Jim Fulton said: >> I suspect that a better solution would be to find a way to bootstrap the >> buildout in a way that included building Python as part of the >> bootstrapping process. > > That would certainly the best approach. Only that I have no idea how to do > that. :( I'm not so sure. I'd prefer bootstrapping not to look too deeply into the buildout configuration, just to keep it simple. Couldn't buildout be taught to restart itself using a custom Python if it finds that one is used, possibly after it has been rebuilt? -- Thomas From jim at zope.com Fri Nov 30 14:23:46 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 30 Nov 2007 08:23:46 -0500 Subject: [Distutils] zc.buildout: Build python in buildout In-Reply-To: References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: On Nov 30, 2007, at 12:42 AM, Thomas Lotze wrote: > Christian Zagrodnick wrote: > >> On 2007-11-28 21:48:13 +0100, Jim Fulton said: >>> I suspect that a better solution would be to find a way to >>> bootstrap the >>> buildout in a way that included building Python as part of the >>> bootstrapping process. >> >> That would certainly the best approach. Only that I have no idea >> how to do >> that. :( > > I'm not so sure. I'd prefer bootstrapping not to look too deeply > into the > buildout configuration, just to keep it simple. > > Couldn't buildout be taught to restart itself using a custom Python > if it > finds that one is used, possibly after it has been rebuilt? Possibly. So if a buildout python option points to a part (a section with a recipe), then it might re-write its script and restart itself if it isn't already running the Python defined by that part. Jim -- Jim Fulton Zope Corporation From thomas at thomas-lotze.de Fri Nov 30 15:50:59 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 30 Nov 2007 15:50:59 +0100 Subject: [Distutils] zc.buildout: Build python in buildout References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: Jim Fulton wrote: > Possibly. So if a buildout python option points to a part (a section with > a recipe), then it might re-write its script and restart itself if it > isn't already running the Python defined by that part. No, I don't mean for buildout to rewrite its own script. I mean that the unchanged buildout script should call path_to_new_python bin/buildout $options Running the buildout script should always invoke the Python used for bootstrapping. Otherwise I fear it's possible to screw up the whole buildout by misconfiguring the Python part. -- Thomas From ignas.mikalajunas at gmail.com Fri Nov 30 16:41:05 2007 From: ignas.mikalajunas at gmail.com (Ignas Mikalajunas) Date: Fri, 30 Nov 2007 17:41:05 +0200 Subject: [Distutils] zc.buildout: Build python in buildout In-Reply-To: References: <1166F969-3B48-4352-84F8-58F4DCA66AE6@zope.com> Message-ID: > > I suspect that a better solution would be to find a way to bootstrap > > the buildout in a way that included building Python as part of the > > bootstrapping process. > > That would certainly the best approach. Only that I have no idea how to > do that. :( Examples: http://source.schooltool.org/viewcvs/trunk/st-buildout/?rev=7260#dirlist - uses a shell script that builds python using buildout, and then builds the product using the new python http://source.schooltool.org/viewcvs/trunk/st-buildout/ - uses a patched bootstrap.py that builds virtual python and uses the virtual python to bootstrap buildout hope that helps Ignas From carl at personnelware.com Fri Nov 30 23:31:40 2007 From: carl at personnelware.com (Carl Karsten) Date: Fri, 30 Nov 2007 16:31:40 -0600 Subject: [Distutils] Your confirmation is required to join the Distutils-SIG mailing list In-Reply-To: References: Message-ID: <47508F4C.8060304@personnelware.com> distutils-sig-confirm+52ef3753124884ddaef255385440abfcefc80eaa at python.org wrote: > Mailing list subscription confirmation notice for mailing list > Distutils-SIG > > We have received a request from 76.29.25.210 for subscription of your > email address, "carl at personnelware.com", to the > distutils-sig at python.org mailing list. To confirm that you want to be > added to this mailing list, simply reply to this message, keeping the > Subject: header intact. Or visit this web page: > > http://mail.python.org/mailman/confirm/distutils-sig/52ef3753124884ddaef255385440abfcefc80eaa > > > Or include the following line -- and only the following line -- in a > message to distutils-sig-request at python.org: > > confirm 52ef3753124884ddaef255385440abfcefc80eaa > > Note that simply sending a `reply' to this message should work from > most mail readers, since that usually leaves the Subject: line in the > right form (additional "Re:" text in the Subject: is okay). > > If you do not wish to be subscribed to this list, please simply > disregard this message. If you think you are being maliciously > subscribed to the list, or have any other questions, send them to > distutils-sig-owner at python.org. > >