From ianb at colorstudy.com Fri Feb 1 20:20:12 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 01 Feb 2008 13:20:12 -0600 Subject: [Distutils] Setuptools scripts on x64 Windows Message-ID: <47A370EC.3050400@colorstudy.com> We got a report of problems with Setuptools-generated scripts on x64 Windows XP. I'm pretty sure the problem is in Setuptools: http://pylonshq.com/project/pylonshq/ticket/370 (Submitter is copied) Ian From vladimirkotulskiy at gmail.com Sat Feb 2 00:24:22 2008 From: vladimirkotulskiy at gmail.com (Vladimir Kotulskiy) Date: Sat, 2 Feb 2008 02:24:22 +0300 Subject: [Distutils] SetupTools on Python 2.5.1 [MSC v.1400 64 bit (AMD64)] on win32 x64 Message-ID: Hello. I have installed last setuptools using python ez_setup.py and have not get any error messages. When I tried to use easy_install utility I got an error message "Cannot find Python executable E:\Develop\Python\python.exe". Also I have the same problem with the Pylons that was installed using ez_setup.py Is there any way to use easy_install with python for x64 ? Y:\Disk>python Python 2.5.1 (r251:54863, Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. Y:\Disk>E:\Develop\Python\python.exe Python 2.5.1 (r251:54863, Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080202/e8265224/attachment.htm From pje at telecommunity.com Mon Feb 4 00:23:16 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 03 Feb 2008 18:23:16 -0500 Subject: [Distutils] SetupTools on Python 2.5.1 [MSC v.1400 64 bit (AMD64)] on win32 x64 In-Reply-To: References: Message-ID: <20080203232313.E6C513A40AF@sparrow.telecommunity.com> At 02:24 AM 2/2/2008 +0300, Vladimir Kotulskiy wrote: >Hello. >I have installed last setuptools using python ez_setup.py and >have not get any error messages. >When I tried to use easy_install utility I got an error message "Cannot find >Python executable E:\Develop\Python\python.exe". Also I have the >same problem with the Pylons that was installed using ez_setup.py >Is there any way to use easy_install with python for x64 ? > > >Y:\Disk>python >Python 2.5.1 >(r251:54863, >Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32 >Type "help", "copyright", "credits" or "license" for more information. > >Y:\Disk>E:\Develop\Python\python.exe >Python 2.5.1 >(r251:54863, >Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32 >Type "help", "copyright", "credits" or "license" for more information. I'm afraid I don't have an x64 system to test or investigate with. I would guess, however, that the problem has to do with the LoadLibraryEx call in launcher.c: http://svn.python.org/view/sandbox/trunk/setuptools/launcher.c The .exe files compiled from launcher.c are 32-bit, but presumably the python.exe being loaded here is 64-bit, and that is probably causing LoadLibraryEx to fail. Unfortunately, I have no idea how to *fix* it. Perhaps somebody with more Windows/C API experience could chime in here? From pje at telecommunity.com Mon Feb 4 00:27:27 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 03 Feb 2008 18:27:27 -0500 Subject: [Distutils] How to make an egg easy_installable ? In-Reply-To: <479EA214.9020507@ar.media.kyoto-u.ac.jp> References: <4799815A.6010100@ar.media.kyoto-u.ac.jp> <20080127195143.58F0C3A4065@sparrow.telecommunity.com> <479EA214.9020507@ar.media.kyoto-u.ac.jp> Message-ID: <20080203232719.CAABF3A40AF@sparrow.telecommunity.com> At 12:48 PM 1/29/2008 +0900, David Cournapeau wrote: >Phillip J. Eby wrote: >> >>Precisely how did you make it? I downloaded it, and there are some >>peculiarities, not the least of which is that the EGG-INFO >>directory is named "egg-info" (lowercase) instead. >> >> >I simply used the bdist_egg command from setuptools. I rebuilt an >egg from my last version (0.3.4), and when unzipping it, I have >EGG-INFO directory this time. How does easy_install decide where to >look for packages ? http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall By the way, it appears that easy_install correctly locates and downloads your numscons 0.3.4 egg now, so I assume all is well. The egg that you had previously did not appear to be a validly-built egg; specifically, it did not appear to have been built by bdist_egg, or if it was, it appeared to have been altered by some other zipfile tool. From pje at telecommunity.com Mon Feb 4 17:01:28 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 04 Feb 2008 11:01:28 -0500 Subject: [Distutils] SetupTools on Python 2.5.1 [MSC v.1400 64 bit (AMD64)] on win32 x64 In-Reply-To: References: <20080203232313.E6C513A40AF@sparrow.telecommunity.com> Message-ID: <20080204160119.099133A40A5@sparrow.telecommunity.com> At 09:31 AM 2/4/2008 +0300, Vladimir Kotulskiy wrote: >I there any place where I can read how can I compile setuptools by my self ? Setuptools doesn't need to be compiled, but launcher.c does. The directions are at the top of the file. From jeff at taupro.com Tue Feb 5 11:23:25 2008 From: jeff at taupro.com (Jeff Rush) Date: Tue, 05 Feb 2008 04:23:25 -0600 Subject: [Distutils] Question re zc.buildout Name Expansion Rules In-Reply-To: <481CE7E2-FFAB-4C9A-B222-C142738409A1@zope.com> References: <479F3DC0.8000309@taupro.com> <481CE7E2-FFAB-4C9A-B222-C142738409A1@zope.com> Message-ID: <47A8391D.1070609@taupro.com> Jim Fulton wrote: > > On Jan 29, 2008, at 9:52 AM, Jeff Rush wrote: > >> Suppose I have something like: >> >> [FreeTDS_installation] >> recipe = zc.recipe.cmmi >> url = blah >> odbcinst_ini = >> [FreeTDS] >> Driver = ${FreeTDS_installation:location}/lib/libtdsodbc.so >> >> 1) Since the variable expansion is referring to an attribute of its >> own .cfg section, is there a way to omit the section name? >> Something like (but this doesn't work): >> >> Driver = ${location}/lib/libtdsodbc.so >> >> It just seems extra work to keep putting my own section name >> in, and is dangerous if I cut/paste lines to move about. > > Yup. I do plan to add a syntax for this. Thanks. >> 2) The initial case above fails to parse because at the time >> the ${} expression is reached during read-in time, the part >> section has not yet been added to the global registry of >> part sections. Hence I get a "FreeTDS_installation" >> undefined error. >> >> It just seems there ought to be a way for a part section >> to refer to its own location attribute. > > I agree that this would be helpful. (This is going to be tricky to > implement, but that's a separate issue.) Would you mind adding a feature > request at: > > https://blueprints.launchpad.net/zc.buildout/ I have done so. -Jeff From david at ar.media.kyoto-u.ac.jp Tue Feb 5 11:27:08 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 05 Feb 2008 19:27:08 +0900 Subject: [Distutils] How to make an egg easy_installable ? In-Reply-To: <20080203232719.CAABF3A40AF@sparrow.telecommunity.com> References: <4799815A.6010100@ar.media.kyoto-u.ac.jp> <20080127195143.58F0C3A4065@sparrow.telecommunity.com> <479EA214.9020507@ar.media.kyoto-u.ac.jp> <20080203232719.CAABF3A40AF@sparrow.telecommunity.com> Message-ID: <47A839FC.30007@ar.media.kyoto-u.ac.jp> Phillip J. Eby wrote: > At 12:48 PM 1/29/2008 +0900, David Cournapeau wrote: >> Phillip J. Eby wrote: >>> >>> Precisely how did you make it? I downloaded it, and there are some >>> peculiarities, not the least of which is that the EGG-INFO directory >>> is named "egg-info" (lowercase) instead. >>> >>> >> I simply used the bdist_egg command from setuptools. I rebuilt an egg >> from my last version (0.3.4), and when unzipping it, I have EGG-INFO >> directory this time. How does easy_install decide where to look for >> packages ? > > http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall > > I've read this page, but I still don't understand: how is it possible to have both url argument to setup and make easy_install look a Pypi first ? > By the way, it appears that easy_install correctly locates and > downloads your numscons 0.3.4 egg now, so I assume all is well. The > egg that you had previously did not appear to be a validly-built egg; > specifically, it did not appear to have been built by bdist_egg, or if > it was, it appeared to have been altered by some other zipfile tool. Hm, strange, I always used bdist_egg. Maybe I did something wrong at some point, but since I cannot reproduce the bug anyway, this is not too much of a concern :). Thanks, David From cz at gocept.com Thu Feb 7 15:49:31 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Thu, 7 Feb 2008 15:49:31 +0100 Subject: [Distutils] buildout: building shared libraries and using them Message-ID: Hi there, we tend to build libxml2, libxslt and lxml in our buildouts together (see below for the config). While building works without any problem, when we use lxml (in tests for instance) the custom built shared libraries are not used but the system libraries. The zc.recipe.egg documentation is quite a bit missleading: rpath: A new-line separated list of directories to search for dynamic libraries at run time. Runtime is apparently referencing the compile time? How could we use the built shared libraries? Should the scripts buildout creates set those paths? Regards, Christian The buildout config: [libxml2] recipe = zc.recipe.cmmi url = http://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/libxml2-2.6.26.tar.gz extra_options = --without-python [libxslt] recipe = zc.recipe.cmmi url = http://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.16.tar.bz2 extra_options = --with-libxml-prefix=${buildout:directory}/parts/libxml2/ --without-python [lxml] recipe = zc.recipe.egg:custom egg = lxml include-dirs = ${buildout:directory}/parts/libxml2/include/libxml2 ${buildout:directory}/parts/libxslt/include library-dirs = ${buildout:directory}/parts/libxml2/lib ${buildout:directory}/parts/libxslt/lib rpath = ${buildout:directory}/parts/libxml2/lib ${buildout:directory}/parts/libxslt/lib -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From cz at gocept.com Thu Feb 7 16:30:50 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Thu, 7 Feb 2008 16:30:50 +0100 Subject: [Distutils] buildout: building shared libraries and using them References: Message-ID: On 2008-02-07 15:49:31 +0100, Christian Zagrodnick said: > Hi there, > > we tend to build libxml2, libxslt and lxml in our buildouts together > > (see below for the config). While building works without any problem, > > when we use lxml (in tests for instance) the custom built shared > > libraries are not used but the system libraries. > > The zc.recipe.egg documentation is quite a bit missleading: > > rpath: A new-line separated list of directories to search for > > dynamic libraries at run time. > > Runtime is apparently referencing the compile time? > > > How could we use the built shared libraries? Should the scripts > > buildout creates set those paths? Right, it seems I was to fast in writing this. Apparently the dynamic linking works as one would expect. But lxml calls xslt-config to find the library directory. xslt-config is just called from the path which is of course the wrong one. So once I get lxml to do the right thing, we're fine. Thanks for listening :) -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From chris at simplistix.co.uk Thu Feb 7 17:55:31 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 07 Feb 2008 16:55:31 +0000 Subject: [Distutils] setuptools - what if a library is required but is in the standard lib? Message-ID: <47AB3803.2050202@simplistix.co.uk> Hi All, If I mark my package as requiring wsgiref, will it still be downloaded if someone is using python 2.5? If yes, how do I write my setup.py such that it's only required if the installer is using python 2.4 or less? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk -------------- next part -------------- An embedded message was scrubbed... From: Chris Withers Subject: [Web-SIG] wsgiref for python 2.4? Date: Thu, 07 Feb 2008 16:48:04 +0000 Size: 4199 Url: http://mail.python.org/pipermail/distutils-sig/attachments/20080207/03b1a5be/attachment.eml From pje at telecommunity.com Thu Feb 7 19:38:15 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 07 Feb 2008 13:38:15 -0500 Subject: [Distutils] setuptools - what if a library is required but is in the standard lib? In-Reply-To: <47AB3803.2050202@simplistix.co.uk> References: <47AB3803.2050202@simplistix.co.uk> Message-ID: <20080207183806.596813A40AC@sparrow.telecommunity.com> At 04:55 PM 2/7/2008 +0000, Chris Withers wrote: >Hi All, > >If I mark my package as requiring wsgiref, will it still be >downloaded if someone is using python 2.5? Only if someone messed with Python's default build structure. The Python stdlib should contain an .egg-info file marking the presence of wsgiref 0.1.2. Some Linux distributions may remove this file due to a misunderstanding of its nature/purpose. Note that this is specific to wsgiref, and doesn't apply to any other separately-distributed packages bundled with 2.5, such as sqlite or ctypes. For these, you'll need to do version checking as described below. >If yes, how do I write my setup.py such that it's only required if >the installer is using python 2.4 or less? requires = [...] if sys.version < "2.5": requires.append('wsgiref>=0.1.2') setup( ... install_requires = requires ) From ziade.tarek at gmail.com Thu Feb 7 21:42:03 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Thu, 7 Feb 2008 12:42:03 -0800 (PST) Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue Message-ID: <15342830.post@talk.nabble.com> Hello, If I create a buildout that uses subversion links to get some eggs, with a download cache, like this (I can provide a test case): [buildout] find-links = http://svn.example.com/my.package/trunk#egg=my.package http://svn.example.com/my.package2/trunk#egg=my.package2 egg = my.package my.package2 download-cache = /Users/tziade/.buildout/downloads/ It will fail because setuptools will try to extract the egg name from the url, using this code in package_index._download_url: name = filter(None,urlparse.urlparse(url)[2].split('/')) The name will then be "trunk" and the method will fail to work properly on the next download (my.package2) because the "trunk" folder will already be there: IOError: [Errno 21] Is a directory: '/Users/tziade/.buildout/downloads/trunk' That's because the function should extract the egg name from the egg fragment. If I add this code there: if '#egg=' in url: name = url.split('#=egg')[-1].strip() else: ... existing code to get the name.. It will work fine, because it will use unique egg names for folders. I would like to suggest changing the code this way. ++ Tarek -- View this message in context: http://www.nabble.com/setuptools-0.6c7-and-zc.buildout%3A-find-links-egg-fragment-issue-tp15342830p15342830.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From jim at zope.com Thu Feb 7 21:52:24 2008 From: jim at zope.com (Jim Fulton) Date: Thu, 7 Feb 2008 15:52:24 -0500 Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue In-Reply-To: <15342830.post@talk.nabble.com> References: <15342830.post@talk.nabble.com> Message-ID: <68509453-8455-441F-B736-07222BDD5226@zope.com> FWIW, I find this feature to be baroque. I have very little interest in supporting it in buildout. (I wouldn't go out of my way to break it either.) I'd prefer to explore other ways to deal with the underlying use case in the context of buildout. I've tried to ignore this issue broadly, but I'm willing to work on an alternate solution because just seeing the discussion go by is too painful to keep ignoring. Jim On Feb 7, 2008, at 3:42 PM, Tarek Ziad? wrote: > > Hello, > > If I create a buildout that uses subversion links to get some eggs, > with a > download cache, like > this (I can provide a test case): > > [buildout] > > find-links = > http://svn.example.com/my.package/trunk#egg=my.package > http://svn.example.com/my.package2/trunk#egg=my.package2 > > egg = > my.package > my.package2 > > download-cache = /Users/tziade/.buildout/downloads/ > > It will fail because setuptools will try to extract the egg name > from the > url, using > this code in package_index._download_url: > > name = filter(None,urlparse.urlparse(url)[2].split('/')) > > The name will then be "trunk" and the method will fail to work > properly on > the next > download (my.package2) because the "trunk" folder will already be > there: > > IOError: > [Errno 21] Is a directory: '/Users/tziade/.buildout/downloads/trunk' > > > That's because the function should extract the egg name from the egg > fragment. > If I add this code there: > > if '#egg=' in url: > name = url.split('#=egg')[-1].strip() > else: > ... existing code to get the name.. > > It will work fine, because it will use unique egg names for folders. > I would like to suggest changing the code this way. > > ++ > Tarek > > > -- > View this message in context: http://www.nabble.com/setuptools-0.6c7-and-zc.buildout%3A-find-links-egg-fragment-issue-tp15342830p15342830.html > Sent from the Python - distutils-sig mailing list archive at > Nabble.com. > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton Zope Corporation From ziade.tarek at gmail.com Thu Feb 7 22:14:43 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Thu, 7 Feb 2008 13:14:43 -0800 (PST) Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue In-Reply-To: <68509453-8455-441F-B736-07222BDD5226@zope.com> References: <15342830.post@talk.nabble.com> <68509453-8455-441F-B736-07222BDD5226@zope.com> Message-ID: <15343632.post@talk.nabble.com> Jim Fulton wrote: > > FWIW, I find this feature to be baroque. I have very little interest > in supporting it in buildout. (I wouldn't go out of my way to break it > either.) I'd prefer to explore other ways to deal with the underlying > use case in the context of buildout. I've tried to ignore this issue > broadly, but I'm willing to work on an alternate solution because just > seeing the discussion go by is too painful to keep ignoring. > Ok. The use case is : we are working on a bunch of private packages that do not have a public distribution. So we start a buildout like this: [buildout] develop = ../../packages/my.package ../../packages/my.package2 eggs = my.package my.package2 Then we create for each package a tag on the svn. And we want to distribute our buildout. so we change the buildout this way: [buildout] find-links = http://private/packages/my.package/tags/0.1#egg=my.package http://private/packages/my.package2/tags/0.1#egg=my.package2 extensions = lovely.buildouthttp download-cache = downloads eggs = my.package my.package2 ..to build a buildout that will have those eggs built within the folder. Then we add "install-from-cache" and "offline" to provide a installable tarball. Ok, we could release some eggs in some private url to have the proper find-links, but the #egg is quite convenient to avoid this extra step. So if we can find a way to automate this.. 2/ the develop problem Another point that would be great to consider: setting up the develop section can be painful when you share a project among many developers, because they need to svn checkout many packages and make sure they have the same folder structure. So could you consider this feature here : http://pypi.python.org/pypi/gp.svndevelop/0.1 ? We created this to be able to automate the set of develop on developers boxes. I think buildout misses this. 3/ the boostrap.py problem If we want to have a full offline mode (not using your rpm system) with a buildout directory, we need boostrap.py to avoid trying to get setuptools with an openurl call. So we have created a special boostrap.py file that first looks in the eggs folder, to be sure to have a 100% offline mode https://ingeniweb.svn.sourceforge.net/svnroot/ingeniweb/IngeniSkel/trunk/ingeniskel/templates/iw_plone_project/buildout/bootstrap.py Could you consider such a feature as well ? We work a *lot* with zc.buildout here, and those three points are the three things that are missing, in the way we use it. I can spare some time to work on some branches. ++ Tarek -- View this message in context: http://www.nabble.com/setuptools-0.6c7-and-zc.buildout%3A-find-links-egg-fragment-issue-tp15342830p15343632.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From ziade.tarek at gmail.com Thu Feb 7 22:17:52 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Thu, 7 Feb 2008 13:17:52 -0800 (PST) Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue In-Reply-To: <15343632.post@talk.nabble.com> References: <15342830.post@talk.nabble.com> <68509453-8455-441F-B736-07222BDD5226@zope.com> <15343632.post@talk.nabble.com> Message-ID: <15343715.post@talk.nabble.com> Tarek Ziad? wrote: > > We work a *lot* with zc.buildout here, and those three points are the > three things that are missing, > in the way we use it. I can spare some time to work on some branches. > Oups I forgot one point : being able to tell that a given part in the buildout will be run only under a given platform win32. ++ tarek -- View this message in context: http://www.nabble.com/setuptools-0.6c7-and-zc.buildout%3A-find-links-egg-fragment-issue-tp15342830p15343715.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From jim at zope.com Thu Feb 7 23:02:28 2008 From: jim at zope.com (Jim Fulton) Date: Thu, 7 Feb 2008 17:02:28 -0500 Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue In-Reply-To: <15343632.post@talk.nabble.com> References: <15342830.post@talk.nabble.com> <68509453-8455-441F-B736-07222BDD5226@zope.com> <15343632.post@talk.nabble.com> Message-ID: <0ECF1B7B-5132-4530-AC66-A2653336729D@zope.com> On Feb 7, 2008, at 4:14 PM, Tarek Ziad? wrote: > > > Jim Fulton wrote: >> >> FWIW, I find this feature to be baroque. I have very little interest >> in supporting it in buildout. (I wouldn't go out of my way to break >> it >> either.) I'd prefer to explore other ways to deal with the underlying >> use case in the context of buildout. I've tried to ignore this issue >> broadly, but I'm willing to work on an alternate solution because >> just >> seeing the discussion go by is too painful to keep ignoring. >> > > Ok. The use case is : we are working on a bunch of private packages > that do not have a public distribution. Have you considered a private repository? > So we start a buildout like this: > > [buildout] > > develop = > ../../packages/my.package > ../../packages/my.package2 > > eggs = > my.package > my.package2 > > Then we create for each package a tag on the svn. And we want to > distribute our buildout. so we change the buildout this way: Is your intention to distribute it for production deployment? Or for development? > [buildout] > > find-links = > http://private/packages/my.package/tags/0.1#egg=my.package > http://private/packages/my.package2/tags/0.1#egg=my.package2 > > extensions = lovely.buildouthttp > > download-cache = downloads > > eggs = > my.package > my.package2 > > > ..to build a buildout that will have those eggs built within the > folder. > Then we add "install-from-cache" and "offline" to provide a > installable > tarball. Have you looked at zc.sourcerelease? > Ok, we could release some eggs in some private url to have the proper > find-links, but the #egg is quite convenient to avoid this extra step. Just to avoid making a distro? You're already making an tag. Why not just use externals? > So if we can find a way to automate this.. I'm still not clear on what "this" is. Let me see, you want to be able to distribute software that includes develop eggs from svn. Is that right? > 2/ the develop problem > > Another point that would be great to consider: setting up the develop > section can be painful when you share a project among many developers, > because they need to svn checkout many packages and make sure they > have the same folder structure. Unless you use externals. > So could you consider this feature here : > http://pypi.python.org/pypi/gp.svndevelop/0.1 ? > We created this to be able to automate the set of develop on > developers > boxes. > I think buildout misses this. Why not use externals? > 3/ the boostrap.py problem > > If we want to have a full offline mode (not using your rpm system) You mean without zc.sourcerelease? > with a > buildout directory, > we need boostrap.py to avoid trying to get setuptools with an > openurl call. > > So we have created a special boostrap.py file that first looks in > the eggs > folder, > to be sure to have a 100% offline mode > > https://ingeniweb.svn.sourceforge.net/svnroot/ingeniweb/IngeniSkel/trunk/ingeniskel/templates/iw_plone_project/buildout/bootstrap.py > > Could you consider such a feature as well ? Sure. I'm unclear on why you don't want to use zc.sourcerelease? > We work a *lot* with zc.buildout here, and those three points are > the three > things that are missing, > in the way we use it. I can spare some time to work on some branches. Cool. > Oups I forgot one point : being able to tell that a given part in the > buildout will be run > only under a given platform win32. Right, so for other folks benefit, the idea is to be able to say something like: [foo platform=win32] and have the options defined there provide foo options only on win32. I definitely plan to do this. Jim -- Jim Fulton Zope Corporation From zooko at zooko.com Thu Feb 7 23:21:32 2008 From: zooko at zooko.com (zooko) Date: Thu, 7 Feb 2008 15:21:32 -0700 Subject: [Distutils] setuptools - what if a library is required but is in the standard lib? In-Reply-To: <20080207183806.596813A40AC@sparrow.telecommunity.com> References: <47AB3803.2050202@simplistix.co.uk> <20080207183806.596813A40AC@sparrow.telecommunity.com> Message-ID: On Feb 7, 2008, at 11:38 AM, Phillip J. Eby wrote: > Only if someone messed with Python's default build structure. The > Python stdlib should contain an .egg-info file marking the presence > of wsgiref 0.1.2. Some Linux distributions may remove this file due > to a misunderstanding of its nature/purpose. Fedora 9, due to be released April, 2008, marks the first release of Fedora which no longer deletes or otherwise messes up the .egg-info files. Ubuntu Hardy LTS 8.04 -- also due April 2008 -- will handle .egg-info files nicely. I think Ubuntu Gutsy 7.10 -- released October 2007 -- or even Ubuntu Feisty 7.04 -- released April 2007 -- may also have handled .egg-info files correctly, I'm not entirely sure. Regards, Zooko From doko at cs.tu-berlin.de Fri Feb 8 00:16:12 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Fri, 8 Feb 2008 00:16:12 +0100 Subject: [Distutils] setuptools - what if a library is required but is in the standard lib? In-Reply-To: References: <47AB3803.2050202@simplistix.co.uk> <20080207183806.596813A40AC@sparrow.telecommunity.com> Message-ID: <18347.37180.62488.232312@gargle.gargle.HOWL> zooko writes: > On Feb 7, 2008, at 11:38 AM, Phillip J. Eby wrote: > > > Only if someone messed with Python's default build structure. The > > Python stdlib should contain an .egg-info file marking the presence > > of wsgiref 0.1.2. Some Linux distributions may remove this file due > > to a misunderstanding of its nature/purpose. > > > Fedora 9, due to be released April, 2008, marks the first release of > Fedora which no longer deletes or otherwise messes up the .egg-info > files. Ubuntu Hardy LTS 8.04 -- also due April 2008 -- will > handle .egg-info files nicely. I think Ubuntu Gutsy 7.10 -- > released October 2007 -- or even Ubuntu Feisty 7.04 -- released April > 2007 -- may also have handled .egg-info files correctly, I'm not > entirely sure. these should be handled correctly back to 6.10 (Ubuntu edgy). Matthias From ziade.tarek at gmail.com Fri Feb 8 10:14:13 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Fri, 8 Feb 2008 01:14:13 -0800 (PST) Subject: [Distutils] setuptools 0.6c7 and zc.buildout: find-links egg fragment issue In-Reply-To: <0ECF1B7B-5132-4530-AC66-A2653336729D@zope.com> References: <15342830.post@talk.nabble.com> <68509453-8455-441F-B736-07222BDD5226@zope.com> <15343632.post@talk.nabble.com> <0ECF1B7B-5132-4530-AC66-A2653336729D@zope.com> Message-ID: <15351725.post@talk.nabble.com> Jim Fulton wrote: > >> >> Ok. The use case is : we are working on a bunch of private packages >> that do not have a public distribution. > > Have you considered a private repository? > It is in a private repository. Jim Fulton wrote: > >> Then we create for each package a tag on the svn. And we want to >> distribute our buildout. so we change the buildout this way: > > Is your intention to distribute it for production deployment? Or for > development? > For production deployement Jim Fulton wrote: > >> Then we add "install-from-cache" and "offline" to provide a >> installable >> tarball. > > Have you looked at zc.sourcerelease? > We have quite a similar script that builds a tarball, we created before you pointed me to zc.sourcerelease. I guess your script handles the bootstrap part differently if you didn't need to change it. So I will try that. But we need to keep our script because it has more advanced features iirc for our upgrade needs (like making diffs between two buildouts) Jim Fulton wrote: > >> Ok, we could release some eggs in some private url to have the proper >> find-links, but the #egg is quite convenient to avoid this extra step. > > Just to avoid making a distro? You're already making an tag. > > Why not just use externals? > We are juste tagging. Making a distro is an extra step. Setting up a bundle is also an extra step. Jim Fulton wrote: > >> So if we can find a way to automate this.. > > I'm still not clear on what "this" is. Let me see, you want to be > able to distribute software that includes develop eggs from svn. Is > that right? > No, I want to be able to distribute software that includes eggs from svn. Jim Fulton wrote: > >> 2/ the develop problem >> >> Another point that would be great to consider: setting up the develop >> section can be painful when you share a project among many developers, >> because they need to svn checkout many packages and make sure they >> have the same folder structure. > > Unless you use externals. > Yup. but that's an extra step, because you need to build a bundle, then make sure the buildout and the bundle are always synchronized. The buildout was a way to get rid of bundles imho. If I need to work with a bundle, it's a step back in my opinion. If we could point an egg in a svn like we do on a file system: develop = /path/in/the/system/my.egg svn://path/in/the/svn/my.egg2/trunk https://path/in/the/svn/my.egg3/trunk That would avoid maintaining a bundle Jim Fulton wrote: > >> 3/ the boostrap.py problem >> If we want to have a full offline mode (not using your rpm system) > > You mean without zc.sourcerelease? > I guess.. Jim Fulton wrote: > >> Oups I forgot one point : being able to tell that a given part in the >> buildout will be run >> only under a given platform win32. > > Right, so for other folks benefit, the idea is to be able to say > something like: > > [foo platform=win32] > > and have the options defined there provide foo options only on win32. > > I definitely plan to do this. > Great ! ++ Tarek -- View this message in context: http://www.nabble.com/setuptools-0.6c7-and-zc.buildout%3A-find-links-egg-fragment-issue-tp15342830p15351725.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From chris at simplistix.co.uk Fri Feb 8 15:12:23 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 08 Feb 2008 14:12:23 +0000 Subject: [Distutils] alternatives to zc.buildout? Message-ID: <47AC6347.5000007@simplistix.co.uk> Hi All, What are the alternatives to zc.buildout? I'm wondering before I plough into zc.buildout because I'm guessing other big projects have run into the problem of pinning certain versions of eggs, building "instance homes" for projects, using a mixture of "public" eggs and those that are private and for a specific project, etc and otherwise working round limitations in setuptools... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From tseaver at palladion.com Fri Feb 8 15:27:00 2008 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 08 Feb 2008 09:27:00 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47AC6347.5000007@simplistix.co.uk> References: <47AC6347.5000007@simplistix.co.uk> Message-ID: <47AC66B4.3050907@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote: > Hi All, > > What are the alternatives to zc.buildout? > > I'm wondering before I plough into zc.buildout because I'm guessing > other big projects have run into the problem of pinning certain versions > of eggs, building "instance homes" for projects, using a mixture of > "public" eggs and those that are private and for a specific project, etc > and otherwise working round limitations in setuptools... The major competitor for mindshare is probably Ian Bicking's 'virtualenv': it makes using 'easy_install' *really* easy, because you are working in a sandbox. The standard installs for repoze.zope2, repoze.grok, repoze.plone, etc. all assume a virtualenv, and add scripts which turn the vanilla virtualenv into an "instance." Trse. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHrGa0+gerLs4ltQ4RAmTYAKCgqLfrRYrl40phh5RHObr7E9x35wCgqpFl FeAhUe2+382JdEGOz4AMI1Y= =qNfI -----END PGP SIGNATURE----- From marius at pov.lt Fri Feb 8 17:44:20 2008 From: marius at pov.lt (Marius Gedminas) Date: Fri, 8 Feb 2008 18:44:20 +0200 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47AC66B4.3050907@palladion.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> Message-ID: <20080208164420.GA8988@fridge.pov.lt> On Fri, Feb 08, 2008 at 09:27:00AM -0500, Tres Seaver wrote: > Chris Withers wrote: > > Hi All, > > > > What are the alternatives to zc.buildout? > > > > I'm wondering before I plough into zc.buildout because I'm guessing > > other big projects have run into the problem of pinning certain versions > > of eggs, building "instance homes" for projects, using a mixture of > > "public" eggs and those that are private and for a specific project, etc > > and otherwise working round limitations in setuptools... > > The major competitor for mindshare is probably Ian Bicking's > 'virtualenv': it makes using 'easy_install' *really* easy, because you > are working in a sandbox. And then there are people who combine zc.buildout with virtualenv. They solve slightly different problems: virtualenv isolates a sandbox from unknown and possibly broken stuff in your system-wide Python installation; zc.buildout automates downloading and building dependencies (which can be anything from pure Python packages to C libraries). Marius Gedminas -- 99 little bugs in the code, 99 bugs in the code, fix one bug, compile it again... 101 little bugs in the code.... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080208/f2404b62/attachment.pgp From optilude at gmx.net Sat Feb 9 07:14:00 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sat, 09 Feb 2008 06:14:00 +0000 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47AC66B4.3050907@palladion.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> Message-ID: Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Chris Withers wrote: >> Hi All, >> >> What are the alternatives to zc.buildout? >> >> I'm wondering before I plough into zc.buildout because I'm guessing >> other big projects have run into the problem of pinning certain versions >> of eggs, building "instance homes" for projects, using a mixture of >> "public" eggs and those that are private and for a specific project, etc >> and otherwise working round limitations in setuptools... > > The major competitor for mindshare is probably Ian Bicking's > 'virtualenv': it makes using 'easy_install' *really* easy, because you > are working in a sandbox. > > The standard installs for repoze.zope2, repoze.grok, repoze.plone, etc. > all assume a virtualenv, and add scripts which turn the vanilla > virtualenv into an "instance." The problem I have with virtualenv - or rather, with raw setuptools based solutions - is that it's difficult to uninstall eggs. With buildout, once you remove the requirement/egg line and re-run buildout, the egg is no longer installed. That said, I like virtualenv and use it a lot. I just think it solves a slightly different use case from the "repeatable configuration management system" one that buildout tries to solve. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From chrism at plope.com Sat Feb 9 17:06:18 2008 From: chrism at plope.com (Chris McDonough) Date: Sat, 09 Feb 2008 11:06:18 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> Message-ID: <47ADCF7A.8020806@plope.com> Martin Aspeli wrote: > Tres Seaver wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Chris Withers wrote: >>> Hi All, >>> > > >> The standard installs for repoze.zope2, repoze.grok, repoze.plone, etc. >> all assume a virtualenv, and add scripts which turn the vanilla >> virtualenv into an "instance." > > The problem I have with virtualenv - or rather, with raw setuptools > based solutions - is that it's difficult to uninstall eggs. With > buildout, once you remove the requirement/egg line and re-run buildout, > the egg is no longer installed. I usually edit site-packages/easy_install.pth and delete the line that points to the egg. And either delete the egg, or not. > That said, I like virtualenv and use it a lot. I just think it solves a > slightly different use case from the "repeatable configuration > management system" one that buildout tries to solve. It does indeed handle uninstall which lots of other things don't. - C From optilude at gmx.net Sat Feb 9 18:15:10 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sat, 09 Feb 2008 17:15:10 +0000 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47ADCF7A.8020806@plope.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADCF7A.8020806@plope.com> Message-ID: Chris McDonough wrote: > Martin Aspeli wrote: >> Tres Seaver wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Chris Withers wrote: >>>> Hi All, >>>> >> > >>> The standard installs for repoze.zope2, repoze.grok, repoze.plone, etc. >>> all assume a virtualenv, and add scripts which turn the vanilla >>> virtualenv into an "instance." >> The problem I have with virtualenv - or rather, with raw setuptools >> based solutions - is that it's difficult to uninstall eggs. With >> buildout, once you remove the requirement/egg line and re-run buildout, >> the egg is no longer installed. > > I usually edit site-packages/easy_install.pth and delete the line that points to > the egg. And either delete the egg, or not. That file's fairly cryptic and the chances of screwing something up there I think are higher than in the more user-oriented buildout.cfg. Also, does deleting egg A in easy_install.pth cause any eggs that were installed as dependencies of A but are no longer needed to go as well? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From tseaver at palladion.com Sat Feb 9 20:32:32 2008 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 09 Feb 2008 14:32:32 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> Message-ID: <47ADFFD0.3060001@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Aspeli wrote: > Tres Seaver wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Chris Withers wrote: >>> Hi All, >>> >>> What are the alternatives to zc.buildout? >>> >>> I'm wondering before I plough into zc.buildout because I'm guessing >>> other big projects have run into the problem of pinning certain versions >>> of eggs, building "instance homes" for projects, using a mixture of >>> "public" eggs and those that are private and for a specific project, etc >>> and otherwise working round limitations in setuptools... >> The major competitor for mindshare is probably Ian Bicking's >> 'virtualenv': it makes using 'easy_install' *really* easy, because you >> are working in a sandbox. > > >> The standard installs for repoze.zope2, repoze.grok, repoze.plone, etc. >> all assume a virtualenv, and add scripts which turn the vanilla >> virtualenv into an "instance." > > The problem I have with virtualenv - or rather, with raw setuptools > based solutions - is that it's difficult to uninstall eggs. With > buildout, once you remove the requirement/egg line and re-run buildout, > the egg is no longer installed. My major beef with zc.buildout is perhaps actually a problem with the recipes nearly everybody uses: they blow away hand-edited stuff without warning. In particular, changes to things like the zope.conf file get zapped, because buildout (or the recipe) thinks that the file is its own personal property. > That said, I like virtualenv and use it a lot. I just think it solves a > slightly different use case from the "repeatable configuration > management system" one that buildout tries to solve. My sense is that zc.buildout's focus on "production deployment" usecass is the source of a lot of my frustration with it as a developer: I expect to "inhabit" the environment it creates, which is completely irrelvant in a production rollout. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHrf/Q+gerLs4ltQ4RAuWmAJ9BkaMqg5xnantzpinhSEc/IF1KSQCePtFo dA/0O/68QFhZ4KFUlflwW9Y= =PA70 -----END PGP SIGNATURE----- From chrism at plope.com Sat Feb 9 20:40:07 2008 From: chrism at plope.com (Chris McDonough) Date: Sat, 09 Feb 2008 14:40:07 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADCF7A.8020806@plope.com> Message-ID: <47AE0197.7020304@plope.com> Martin Aspeli wrote: > Chris McDonough wrote: >> Martin Aspeli wrote: >>> Tres Seaver wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> Chris Withers wrote: >>>>> Hi All, >>>>> >> I usually edit site-packages/easy_install.pth and delete the line that points to >> the egg. And either delete the egg, or not. > > That file's fairly cryptic It's one path per line. Doesn't get too much simpler than that. .pth files have been around forever (almost literally, they have been around since I started using Python anyway, 7 years ago) and are documented pretty well in the Python docs: http://docs.python.org/lib/module-site.html > and the chances of screwing something up > there I think are higher than in the more user-oriented buildout.cfg. True, at least if you trust the recipe authors. Although deleting a line isn't very hard. > Also, does deleting egg A in easy_install.pth cause any eggs that were > installed as dependencies of A but are no longer needed to go as well? No. - C From optilude at gmx.net Sun Feb 10 03:27:49 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 10 Feb 2008 02:27:49 +0000 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47AE0197.7020304@plope.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADCF7A.8020806@plope.com> <47AE0197.7020304@plope.com> Message-ID: Chris McDonough wrote: > Martin Aspeli wrote: >> Chris McDonough wrote: >>> Martin Aspeli wrote: >>>> Tres Seaver wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> Chris Withers wrote: >>>>>> Hi All, >>>>>> >>> I usually edit site-packages/easy_install.pth and delete the line that points to >>> the egg. And either delete the egg, or not. >> That file's fairly cryptic > > It's one path per line. Doesn't get too much simpler than that. .pth files > have been around forever (almost literally, they have been around since I > started using Python anyway, 7 years ago) and are documented pretty well in the > Python docs: > > http://docs.python.org/lib/module-site.html > >> and the chances of screwing something up >> there I think are higher than in the more user-oriented buildout.cfg. > > True, at least if you trust the recipe authors. Although deleting a line isn't > very hard. Knowing which line to delete may be. ;) >> Also, does deleting egg A in easy_install.pth cause any eggs that were >> installed as dependencies of A but are no longer needed to go as well? > > No. Right. Which means that there are lots of eggs there that you won't know whether you need to delete or not and some very bad things may happen if you delete the wrong one (or bad things may be happening already due to a dependency that got installed by accident and that you need to delete). Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From optilude at gmx.net Sun Feb 10 03:30:43 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 10 Feb 2008 02:30:43 +0000 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47ADFFD0.3060001@palladion.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> Message-ID: Tres Seaver wrote: > My major beef with zc.buildout is perhaps actually a problem with the > recipes nearly everybody uses: they blow away hand-edited stuff without > warning. In particular, changes to things like the zope.conf file get > zapped, because buildout (or the recipe) thinks that the file is its own > personal property. This is an implementation detail of the plone.recipe.zope2instance recipe, which indeed does make that assumption. You do have the option of specifying an alternative zope.conf file with this particular recipe that's not touched, but that's obviously a recipe-specific problem and solution. In some cases, stomping *is* the right solution, enabling you to return to a "known good" state every time. It's more of a cultural and documentation thing than anything else. >> That said, I like virtualenv and use it a lot. I just think it solves a >> slightly different use case from the "repeatable configuration >> management system" one that buildout tries to solve. > > My sense is that zc.buildout's focus on "production deployment" usecass > is the source of a lot of my frustration with it as a developer: I > expect to "inhabit" the environment it creates, which is completely > irrelvant in a production rollout. I find it very useful in a development setting, and find that it makes the tasks of moving from development to staging to production more manageable. That said, it's not as easy as just doing "python setup.py install" in a virtualenv to trial something out. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From jim at zope.com Sun Feb 10 19:53:53 2008 From: jim at zope.com (Jim Fulton) Date: Sun, 10 Feb 2008 13:53:53 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47ADFFD0.3060001@palladion.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> Message-ID: <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> On Feb 9, 2008, at 2:32 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Martin Aspeli wrote: >> Tres Seaver wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Chris Withers wrote: >>>> Hi All, >>>> >>>> What are the alternatives to zc.buildout? >>>> >>>> I'm wondering before I plough into zc.buildout because I'm guessing >>>> other big projects have run into the problem of pinning certain >>>> versions >>>> of eggs, building "instance homes" for projects, using a mixture of >>>> "public" eggs and those that are private and for a specific >>>> project, etc >>>> and otherwise working round limitations in setuptools... >>> The major competitor for mindshare is probably Ian Bicking's >>> 'virtualenv': it makes using 'easy_install' *really* easy, >>> because you >>> are working in a sandbox. >>> >>> The standard installs for repoze.zope2, repoze.grok, repoze.plone, >>> etc. >>> all assume a virtualenv, and add scripts which turn the vanilla >>> virtualenv into an "instance." >> >> The problem I have with virtualenv - or rather, with raw setuptools >> based solutions - is that it's difficult to uninstall eggs. With >> buildout, once you remove the requirement/egg line and re-run >> buildout, >> the egg is no longer installed. > > My major beef with zc.buildout is perhaps actually a problem with the > recipes nearly everybody uses: they blow away hand-edited stuff > without > warning. In particular, changes to things like the zope.conf file get > zapped, because buildout (or the recipe) thinks that the file is its > own > personal property. Which it is. Part of using buildout is understanding that anything built by buildout is controlled by it. I often make temporary hacks to created files, knowing that the changes will be thrown away on the next build. If you want to make a permanent change, you need to update the buildout configuration. There *can* be exceptions to this. We have some recipes that manage checkouts. These don't remove checkouts to avoid losing user data. Similarly, the zc.recipes.filestorage recipe doesn't remove data directories it creates on uninstall. >> That said, I like virtualenv and use it a lot. I just think it >> solves a >> slightly different use case from the "repeatable configuration >> management system" one that buildout tries to solve. > > My sense is that zc.buildout's focus on "production deployment" > usecass > is the source of a lot of my frustration with it as a developer: I > expect to "inhabit" the environment it creates, which is completely > irrelvant in a production rollout. I like buildout for development be because I don't have to "inhabit" the created artifacts. I (can) have a single high-level configuration that lets me control a variety of subsystems in one place. I'm sure this is a matter of taste. Jim -- Jim Fulton Zope Corporation From noah.gift at gmail.com Sun Feb 10 20:01:25 2008 From: noah.gift at gmail.com (Noah Gift) Date: Sun, 10 Feb 2008 14:01:25 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> Message-ID: <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> > > I like buildout for development be because I don't have to "inhabit" > the created artifacts. I (can) have a single high-level configuration > that lets me control a variety of subsystems in one place. > > I'm sure this is a matter of taste. > > Jim > > -- > Jim Fulton > Zope Corporation In reference to this rather long discussion. I would love to get an interview or case study on buildout in the scope of package management for Python for the book I am co-authoring on Python For *nix Systems Administration. If there is an official ambassador, I would love to talk to that person, to make sure I do it justice. That particular chapter will cover eggs, virtualenv, and buildout, so I pretty excited about it, although is the toughest chapter in the book so far. Noah Gift From jim at zope.com Sun Feb 10 20:14:00 2008 From: jim at zope.com (Jim Fulton) Date: Sun, 10 Feb 2008 14:14:00 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> Message-ID: <7231BFD1-737E-4E5F-BC75-BE0B65CB7A98@zope.com> On Feb 10, 2008, at 2:01 PM, Noah Gift wrote: > In reference to this rather long discussion. I would love to get an > interview or case study on buildout in the scope of package > management for Python for the book I am co-authoring on Python For > *nix Systems Administration. If there is an official ambassador, I > would love to talk to that person, to make sure I do it justice. > That particular chapter will cover eggs, virtualenv, and buildout, > so I pretty excited about it, although is the toughest chapter in > the book so far. Cool! Obviously, you can work with me on this. One thing to understand about buildout is that it is oriented toward distributing applications vs easy_install/setuptools with is aimed more at distributing libraries. At ZC, we use buildout to create development environments. We also use buildout to create self-contained source-releases and rpms for deploying out *applications* to out deployment environments in ways that facilitate *operation* by experienced Unix system administrators. Jim -- Jim Fulton Zope Corporation From zooko at zooko.com Sun Feb 10 21:33:01 2008 From: zooko at zooko.com (zooko) Date: Sun, 10 Feb 2008 13:33:01 -0700 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <7231BFD1-737E-4E5F-BC75-BE0B65CB7A98@zope.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> <7231BFD1-737E-4E5F-BC75-BE0B65CB7A98@zope.com> Message-ID: <1D91721A-E501-4395-A072-D3090B22899E@zooko.com> On Feb 10, 2008, at 12:14 PM, Jim Fulton wrote: > > On Feb 10, 2008, at 2:01 PM, Noah Gift wrote: >> In reference to this rather long discussion. I would love to get an >> interview or case study on buildout in the scope of package >> management for Python for the book I am co-authoring on Python For >> *nix Systems Administration. > > Cool! Yeah! > One thing to understand about buildout is that it is oriented toward > distributing applications vs easy_install/setuptools with is aimed > more at distributing libraries. For comparison, at http://allmydata.org, we are using eggs plus py2exe, py2app, etc. to build applications to deploy to end users, but we are using Debian packages to deploy applications to system administrators. Our set of tools is working well enough, but there are clearly places where they aren't mature yet. For example we don't use an automated tool to produce .deb's from our setup.py or from our .egg's, but instead have a separately-maintained Debian build. Likewise py2exe and py2app don't naturally manage our own code (which is built by setuptools) and our dependencies (which are mostly but not all packaged as eggs) as smoothly as we would like, but they work. Regards, Zooko From eperez at yaco.es Mon Feb 11 11:45:44 2008 From: eperez at yaco.es (Enrique Perez) Date: Mon, 11 Feb 2008 11:45:44 +0100 Subject: [Distutils] patch for zc.recipe.cmmi Message-ID: Hi, Sorry if this is not the correct list for this issue. We have patched the zc.recipe.cmmi so that it (optionally) leaves the sources in place, so that other packages can build against it. It just unpacks the archive into the parts dir instead of to a tempfile (if a dev option is set in the buildout), and then builds it in place. I attach the patch. Otherwise we probably would upload it as yaco.recipe.cmmi-devel, would this be correct? -- Enrique P?rez Arnaud Yaco Sistemas S.L. -------------- next part -------------- A non-text attachment was scrubbed... Name: zc.recipe.cmmi-devel.patch Type: text/x-patch Size: 4023 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080211/b91a99ba/attachment.bin From noah.gift at gmail.com Mon Feb 11 13:14:39 2008 From: noah.gift at gmail.com (Noah Gift) Date: Mon, 11 Feb 2008 07:14:39 -0500 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <1D91721A-E501-4395-A072-D3090B22899E@zooko.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> <7231BFD1-737E-4E5F-BC75-BE0B65CB7A98@zope.com> <1D91721A-E501-4395-A072-D3090B22899E@zooko.com> Message-ID: Thanks guys. I have sent you both emails offline. On Feb 10, 2008, at 3:33 PM, zooko wrote: > > On Feb 10, 2008, at 12:14 PM, Jim Fulton wrote: > >> >> On Feb 10, 2008, at 2:01 PM, Noah Gift wrote: >>> In reference to this rather long discussion. I would love to get an >>> interview or case study on buildout in the scope of package >>> management for Python for the book I am co-authoring on Python For >>> *nix Systems Administration. >> >> Cool! > > Yeah! > >> One thing to understand about buildout is that it is oriented toward >> distributing applications vs easy_install/setuptools with is aimed >> more at distributing libraries. > > For comparison, at http://allmydata.org, we are using eggs plus > py2exe, py2app, etc. to build applications to deploy to end users, > but we are using Debian packages to deploy applications to system > administrators. > > Our set of tools is working well enough, but there are clearly > places where they aren't mature yet. For example we don't use an > automated tool to produce .deb's from our setup.py or from > our .egg's, but instead have a separately-maintained Debian build. > Likewise py2exe and py2app don't naturally manage our own code > (which is built by setuptools) and our dependencies (which are > mostly but not all packaged as eggs) as smoothly as we would like, > but they work. > > Regards, > > Zooko Noah Gift / http://noahgift.com [Python+Grok+AJAX+Mashup] From cz at gocept.com Mon Feb 11 15:23:11 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Mon, 11 Feb 2008 15:23:11 +0100 Subject: [Distutils] buildout: setting envirionment variables for zc.recipe.egg:custom? Message-ID: Hi, building lxml requires additional configuration so it finds xslt-config correctly. One possibility is to set the PATH environment variable. The other is to call setup.py --xslt-config=... Using setup.py --xslt-config doesn't fit at all into buildout i think. Setting environment variables before installing an egg on the other hand might be useful. Would it make sense to put this into zc.recipe.egg or would it be better to create some unrelated recipe which only sets envirionment variables? Regards, -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From jim at zope.com Mon Feb 11 18:50:09 2008 From: jim at zope.com (Jim Fulton) Date: Mon, 11 Feb 2008 12:50:09 -0500 Subject: [Distutils] patch for zc.recipe.cmmi In-Reply-To: References: Message-ID: <5D50FAEE-AC2C-47DA-B453-84ED6E55C300@zope.com> On Feb 11, 2008, at 5:45 AM, Enrique Perez wrote: > Sorry if this is not the correct list for this issue. This is the right list. > We have patched the zc.recipe.cmmi so that it (optionally) leaves > the sources in place, so that other packages can build against it. > It just unpacks the archive into the parts dir instead of to a > tempfile (if a dev option is set in the buildout), and then builds > it in place. I attach the patch. This sounds good to me. Ooooh, even a test update! :) I'm not sure "dev" is the best option name. I wonder if anyone else has an opinion. I'd prefer you check it in yourself, if you are a zope contributor. If not, then I or someone else with zope check-in access will apply it. > Otherwise we probably would upload it as yaco.recipe.cmmi-devel, > would this be correct? That would be fine too, although I think this is a fine addition to the cmmi recipe. Jim -- Jim Fulton Zope Corporation From jim at zope.com Mon Feb 11 19:08:56 2008 From: jim at zope.com (Jim Fulton) Date: Mon, 11 Feb 2008 13:08:56 -0500 Subject: [Distutils] buildout: setting envirionment variables for zc.recipe.egg:custom? In-Reply-To: References: Message-ID: On Feb 11, 2008, at 9:23 AM, Christian Zagrodnick wrote: > Hi, > > building lxml requires additional configuration so it finds xslt- > config > correctly. One possibility is to set the PATH environment variable. > The > other is to call setup.py --xslt-config=... > > Using setup.py --xslt-config doesn't fit at all into buildout i think. I don't agree. First, this isn't really so much about buildout as it is about the eggs recipe. The zc.recipe.egg:custom already provides custom build support. It seems reasonable to extend it to allow something like this. > Setting environment variables before installing an egg on the other > hand might be useful. Would that help in *this* case? > Would it make sense to put this into > zc.recipe.egg or would it be better to create some unrelated recipe > which only sets envirionment variables? I suggest updating zc.recipe.egg:custom to accept environment- variable, assuming that setting environment variables is sufficient. I think an option to pass setup custom options would be useful as well. Jim -- Jim Fulton Zope Corporation From eperez at yaco.es Mon Feb 11 19:05:47 2008 From: eperez at yaco.es (=?ISO-8859-1?Q?Enrique_P=E9rez?=) Date: Mon, 11 Feb 2008 19:05:47 +0100 Subject: [Distutils] patch for zc.recipe.cmmi In-Reply-To: <5D50FAEE-AC2C-47DA-B453-84ED6E55C300@zope.com> References: <5D50FAEE-AC2C-47DA-B453-84ED6E55C300@zope.com> Message-ID: <47B08E7B.9040906@yaco.es> Jim Fulton escribi?: > > On Feb 11, 2008, at 5:45 AM, Enrique Perez wrote: > >> We have patched the zc.recipe.cmmi so that it (optionally) leaves the >> sources in place, so that other packages can build against it. It >> just unpacks the archive into the parts dir instead of to a tempfile >> (if a dev option is set in the buildout), and then builds it in >> place. I attach the patch. > > This sounds good to me. Ooooh, even a test update! :) I'm not sure > "dev" is the best option name. I wonder if anyone else has an opinion. keep-sources? > > I'd prefer you check it in yourself, if you are a zope contributor. > If not, then I or someone else with zope check-in access will apply it. I am not a zope contributor, so someone else will have to check it in. -- Enrique P?rez Arnaud Yaco Sistemas S.L. From cz at gocept.com Mon Feb 11 19:57:06 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Mon, 11 Feb 2008 19:57:06 +0100 Subject: [Distutils] buildout: setting envirionment variables for zc.recipe.egg:custom? References: Message-ID: On 2008-02-11 19:08:56 +0100, Jim Fulton said: > > On Feb 11, 2008, at 9:23 AM, Christian Zagrodnick wrote: > >> Hi, >> >> building lxml requires additional configuration so it finds xslt- >> config >> correctly. One possibility is to set the PATH environment variable. >> The >> other is to call setup.py --xslt-config=... >> >> Using setup.py --xslt-config doesn't fit at all into buildout i think. > > I don't agree. First, this isn't really so much about buildout as it > is about the eggs recipe. The zc.recipe.egg:custom already provides > custom build support. It seems reasonable to extend it to allow > something like this. I would be fine with that of course. > >> Setting environment variables before installing an egg on the other >> hand might be useful. > > Would that help in *this* case? Yes, it would. Because lxml looks in the $PATH for xslt-config. So setting it correctly would help. > >> Would it make sense to put this into >> zc.recipe.egg or would it be better to create some unrelated recipe >> which only sets envirionment variables? > > > I suggest updating zc.recipe.egg:custom to accept environment- > variable, assuming that setting environment variables is sufficient. It is. > I think an option to pass setup custom options would be useful as well. Probably. Although apparently there are not so many eggs requiring it. Anyway, environment would be fine, option would be better :) -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From stef.mientki at gmail.com Mon Feb 11 22:16:41 2008 From: stef.mientki at gmail.com (Stef Mientki) Date: Mon, 11 Feb 2008 22:16:41 +0100 Subject: [Distutils] Run a generated executable, without the DOS-box ? Message-ID: <47B0BB39.1030306@gmail.com> hello, I just succeeded in running py2exe for the first time with good effort, with a relative simple program (complex programs gives too many errors). With Inno I build a windows package, and that was also ok. Now when I run the generated exe-function, (which is a GUI application), I get such a ugly black Command-Box (DOS-box). Is there anyway to prevent that Command Box ? thanks, Stef Mientki From robin at alldunn.com Tue Feb 12 02:41:03 2008 From: robin at alldunn.com (Robin Dunn) Date: Mon, 11 Feb 2008 17:41:03 -0800 Subject: [Distutils] Run a generated executable, without the DOS-box ? In-Reply-To: <47B0BB39.1030306@gmail.com> References: <47B0BB39.1030306@gmail.com> Message-ID: <47B0F92F.8020508@alldunn.com> Stef Mientki wrote: > hello, Hi Stef, > > I just succeeded in running py2exe for the first time with good effort, > with a relative simple program (complex programs gives too many errors). > > With Inno I build a windows package, and that was also ok. > > Now when I run the generated exe-function, > (which is a GUI application), > I get such a ugly black Command-Box (DOS-box). > Is there anyway to prevent that Command Box ? > You probably didn't use the windows = ["scriptname.py"] parameter to your setup() call. -- Robin Dunn Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython! From thomas at thomas-lotze.de Tue Feb 12 14:28:21 2008 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Tue, 12 Feb 2008 14:28:21 +0100 Subject: [Distutils] buildout: bin/test --coverage Message-ID: I've noticed a bug which is most probably the fault of zc.recipe.testrunner, and as I'm not sure where to report bugs against this buildout recipe, I'll drop a note here. The problem occurs when checking test coverage using the test runner's --coverage option. I've run bin/test --coverage on a suite of doc tests, and it gave me some 70% of coverage for a particular code file. After adding a test for one of the uncovered pieces of code, I got less than 70% coverage for the same file, everything else being equal. I found that now, one method was reported as uncovered that definitely executed in the test run (as evidenced by putting in a pdb call, for example). I could even reproduce the behaviour. Maybe it helps to know that when I asked about it on #zope3-dev, agroszer noted that the same happened to him, and usually in functional tests. -- Thomas From marius at pov.lt Tue Feb 12 18:14:50 2008 From: marius at pov.lt (Marius Gedminas) Date: Tue, 12 Feb 2008 19:14:50 +0200 Subject: [Distutils] buildout: bin/test --coverage In-Reply-To: References: Message-ID: <20080212171449.GA2765@fridge.pov.lt> On Tue, Feb 12, 2008 at 02:28:21PM +0100, Thomas Lotze wrote: > I've noticed a bug which is most probably the fault of > zc.recipe.testrunner, and as I'm not sure where to report bugs against > this buildout recipe, I'll drop a note here. > > The problem occurs when checking test coverage using the test runner's > --coverage option. I've run bin/test --coverage on a suite of doc tests, > and it gave me some 70% of coverage for a particular code file. After > adding a test for one of the uncovered pieces of code, I got less than 70% > coverage for the same file, everything else being equal. It's not a problem with the recipe, it's a problem with zope.testing, or, more fundamentally, with Python. The sys.settrace hook used to implement coverage tracing is non-recursive. zope.testing tests its own ability to do coverage tracing, and since the test is run in the same Python process as the test runner, the side effect of the test's calling sys.settrace is that the runner is no longer able to trace coverage. There are other ways to reset the sys.settrace hook. My favourite is class SomeClass(object): def __iter__(self): ... def __len__(self): return len(list(self)) list(obj) cals obj.__len__ as an optimization. This results in an infinite recursive loop, but then list(obj) traps that, hides it, and does the brute-force list conversion. As a side effect of sys.settrace, the stack overflow RuntimeError happens inside the tracing hook and Python removes it for misbehaving. Tracing stops from that point on. Python's doctest also had side effects that used to disable tracing. Marius Gedminas -- You can't have megalomania. *I* have megalomania. -- Joe Bednorz -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080212/098daa4a/attachment.pgp From thomas at thomas-lotze.de Tue Feb 12 21:19:52 2008 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Tue, 12 Feb 2008 21:19:52 +0100 Subject: [Distutils] buildout: bin/test --coverage References: <20080212171449.GA2765@fridge.pov.lt> Message-ID: Marius Gedminas wrote: > It's not a problem with the recipe, it's a problem with zope.testing, or, Sorry, that should have been obvious to me. No idea what made me blame it on the recipe... > more fundamentally, with Python. The sys.settrace hook used to > implement coverage tracing is non-recursive. > > zope.testing tests its own ability to do coverage tracing, and since the > test is run in the same Python process as the test runner, the side > effect of the test's calling sys.settrace is that the runner is no > longer able to trace coverage. Ah, this makes some sense since the function that was reported uncovered after the change is tested later in the same doc test. > list(obj) cals obj.__len__ as an optimization. This results in an > infinite recursive loop, but then list(obj) traps that, hides it, and > does the brute-force list conversion. As a side effect of sys.settrace, > the stack overflow RuntimeError happens inside the tracing hook and > Python removes it for misbehaving. Tracing stops from that point on. That's what I call gory details... > Python's doctest also had side effects that used to disable tracing. OK, so it's no longer a subject for this list at least. -- Thomas From ianb at colorstudy.com Wed Feb 13 05:35:46 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 12 Feb 2008 22:35:46 -0600 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> Message-ID: <47B273A2.6010406@colorstudy.com> Noah Gift wrote: > In reference to this rather long discussion. I would love to get an > interview or case study on buildout in the scope of package management > for Python for the book I am co-authoring on Python For *nix Systems > Administration. If there is an official ambassador, I would love to > talk to that person, to make sure I do it justice. That particular > chapter will cover eggs, virtualenv, and buildout, so I pretty excited > about it, although is the toughest chapter in the book so far. I don't know that many people have used this (or know about it), but I added a command to buildutils (http://knowledgetap.com/hg/buildutils/) to do "python setup.py bundle", which takes a package and all its dependencies and puts them together, with a script that adds all the dependencies. It's like what zc.buildout does mechanically, but intended to be used more like py2exe. I think it'd fit the model of managing Python commands and scripts pretty well. Ian From ianb at colorstudy.com Wed Feb 13 05:56:39 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 12 Feb 2008 22:56:39 -0600 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47ADFFD0.3060001@palladion.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> Message-ID: <47B27887.5070008@colorstudy.com> Tres Seaver wrote: > My major beef with zc.buildout is perhaps actually a problem with the > recipes nearly everybody uses: they blow away hand-edited stuff without > warning. In particular, changes to things like the zope.conf file get > zapped, because buildout (or the recipe) thinks that the file is its own > personal property. This was also something that drove me nuts. It's too bad this is still the case. We have a build tool ourselves, very similar in scope to zc.buildout, though I don't really have intentions at this time to make it a legitimate project for other people to use. But, for the record: http://www.openplans.org/projects/fassembler/ One of the core parts of it is filemaker, which does most of the interaction with the system: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py It goes to great length to notice changes, even if they aren't due to edits; I find it's nice to know what exactly is going on. It's also got a bit of support for detecting why things changed, by saving the template, and probably will grow support for detecting user changes so some things can be overwritten without warning. With these build things I don't really care who owns what, since that's mostly an abstract concept that the build user won't know and shouldn't really have to know. (filemaker was generally based on the code from paster create) I also pay a lot of attention to logging, as I hate noisy output and of course not enough output is also a problem. I can't remember how zc.buildout acts in that respect. Ian From m.pricejones at gmail.com Wed Feb 13 11:13:33 2008 From: m.pricejones at gmail.com (Michael Jones) Date: Wed, 13 Feb 2008 10:13:33 +0000 Subject: [Distutils] Empty directories in eggs Message-ID: <8c8775e00802130213r793b9bbdqf64b5e353214ef63@mail.gmail.com> Hi, There was a post a while back about this (I've included it below) but I can't find any follow up. Is this possible? I'd like to include a folder structure in an egg that can then be copied out to make a template for people to start using. I can script it but the structure includes files and it would be so much easier to maintain them in there original form instead of doing the folder creation and copying the files into the right locations. I've had no luck finding the answers else where. Any help would be much appreciated, Cheers, Mike Orginal post ------------------------------ Hi, I tried to build an egg from this tag, using bdist_egg and sdist: http://svn.plone.org/svn/collective/ZopeSkel/tags/0.5/ However, the packaged egg lost several directories, such as: http://svn.plone.org/svn/collective/ZopeSkel/tags/0.5/zopeskel/templates/plone3_buildout/products/ These are empty, but under version control. To get them into the egg (I need it because it's part of a Paste Script template which expects the empty directories to be there), I had to add some mostly-empty (if slightly useful) README.txt files (on the 0.6 tag). Bug or feature? ;-) Martin From zooko at zooko.com Wed Feb 13 14:58:35 2008 From: zooko at zooko.com (zooko) Date: Wed, 13 Feb 2008 06:58:35 -0700 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <47B273A2.6010406@colorstudy.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> <47B273A2.6010406@colorstudy.com> Message-ID: <4E857DAB-8236-4B25-98AC-68CB6C184F14@zooko.com> On Feb 12, 2008, at 9:35 PM, Ian Bicking wrote: > I don't know that many people have used this (or know about it), but I > added a command to buildutils (http://knowledgetap.com/hg/buildutils/) > to do "python setup.py bundle", which takes a package and all its > dependencies and puts them together, with a script that adds all the > dependencies. It's like what zc.buildout does mechanically, but > intended to be used more like py2exe. I think it'd fit the model of > managing Python commands and scripts pretty well. That's interesting. What we do is have our setup.py look in a directory named "misc/dependencies/" and add any tarballs it finds therein to the dependency_links: http://allmydata.org/trac/tahoe/browser/setup.py#L93 Then if we want to bundle some dependencies with our app, we just include the tarball for that dependency in that directory: http://allmydata.org/trac/tahoe/browser/misc/dependencies Regards, Zooko From wolever at cs.toronto.edu Wed Feb 13 16:35:18 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Wed, 13 Feb 2008 10:35:18 -0500 Subject: [Distutils] Bug in setuptools/command/sdist.py: global name 'log' is not defined Message-ID: When I use ``py setup.py develop``, I get this error: Traceback (most recent call last): File "setup.py", line 79, in ? entry_points = { File "/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/develop.py", line 27, in run self.install_for_development() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/develop.py", line 85, in install_for_development self.run_command('egg_info') File "/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/egg_info.py", line 171, in run self.find_sources() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/egg_info.py", line 252, in find_sources mm.run() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/egg_info.py", line 306, in run self.add_defaults() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/egg_info.py", line 333, in add_defaults rcfiles = list(walk_revctrl()) File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/sdist.py", line 45, in walk_revctrl for item in ep.load()(dirname): File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/sdist.py", line 52, in _default_revctrl for path in finder(dirname,path): File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ setuptools/command/sdist.py", line 98, in entries_finder log.warn("unrecognized .svn/entries format in %s", dirname) NameError: global name 'log' is not defined It looks like the version of my .svn/entries file is newer (head - n1 .svn/entries: 9). I fixed this by adding "from distutils import log" at the stop of setuptools/command/sdist.py. The version, as suggested by the stack trace, is 0.6c7. David From ianb at colorstudy.com Wed Feb 13 18:26:35 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 13 Feb 2008 11:26:35 -0600 Subject: [Distutils] alternatives to zc.buildout? In-Reply-To: <4E857DAB-8236-4B25-98AC-68CB6C184F14@zooko.com> References: <47AC6347.5000007@simplistix.co.uk> <47AC66B4.3050907@palladion.com> <47ADFFD0.3060001@palladion.com> <4C9B55FB-88BA-40E8-89D6-2BB0E424F74C@zope.com> <55CE4027-EF22-43AC-AF35-73376FAC4E6B@gmail.com> <47B273A2.6010406@colorstudy.com> <4E857DAB-8236-4B25-98AC-68CB6C184F14@zooko.com> Message-ID: <47B3284B.60109@colorstudy.com> zooko wrote: > On Feb 12, 2008, at 9:35 PM, Ian Bicking wrote: > >> I don't know that many people have used this (or know about it), but I >> added a command to buildutils (http://knowledgetap.com/hg/buildutils/) >> to do "python setup.py bundle", which takes a package and all its >> dependencies and puts them together, with a script that adds all the >> dependencies. It's like what zc.buildout does mechanically, but >> intended to be used more like py2exe. I think it'd fit the model of >> managing Python commands and scripts pretty well. > > That's interesting. What we do is have our setup.py look in a directory > named "misc/dependencies/" and add any tarballs it finds therein to the > dependency_links: > > http://allmydata.org/trac/tahoe/browser/setup.py#L93 That's similar to bundle; bundle just calls easy_install to install all the dependencies into a particular directory, then adds "site.addsitedir(dependency_dir)" to the top of any scripts. But as a result you don't have to call setup.py on the host. Figuring out the location of dependency_dir is less than perfect. Both relative and absolute filenames have their problems. Ian From pje at telecommunity.com Fri Feb 15 19:51:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 13:51:42 -0500 Subject: [Distutils] setuptools 0.6c8 released Message-ID: <20080215185130.E1B333A40AE@sparrow.telecommunity.com> This is the last 0.6 candidate release, barring regressions introduced by changes in this release. In one week, if there are no regressions, 0.6 final will be released, so please report any bugs promptly. 0.6final will be the last OSAF-funded release of setuptools, as it already exceeds the Chandler desktop's current and future requirements, at least for the next year. There remain various known issues and limitations, but these won't be fixed in the 0.6 line. From wolever at cs.toronto.edu Fri Feb 15 21:44:11 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 15 Feb 2008 15:44:11 -0500 Subject: [Distutils] Including directories in .eggs? Message-ID: Hey, I'm wondering how I would go about including an entire directory in the .egg setup.py creates. We are using Dojo, and it automatically builds a directory containing the "stuff" it needs... But just including that directory in package_data doesn't work (as it expects to copy a file, not a directory). Including a glob such as 'js/**/ **' also fails, because directories are listed as part of the '**/ **', and it can't copy those. Any ideas? Thanks, David From pje at telecommunity.com Fri Feb 15 21:47:51 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 15:47:51 -0500 Subject: [Distutils] Bug in setuptools/command/sdist.py: global name 'log' is not defined In-Reply-To: References: Message-ID: <20080215204741.2844B3A4075@sparrow.telecommunity.com> At 10:35 AM 2/13/2008 -0500, David Wolever wrote: >It looks like the version of my .svn/entries file is newer (head - >n1 .svn/entries: 9). Hm. Does it detect the correct files, if you change the 'data.startswith("8")' to 'data.startswith("9")'? From wolever at cs.toronto.edu Fri Feb 15 21:58:34 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 15 Feb 2008 15:58:34 -0500 Subject: [Distutils] Bug in setuptools/command/sdist.py: global name 'log' is not defined In-Reply-To: <20080215204741.2844B3A4075@sparrow.telecommunity.com> References: <20080215204741.2844B3A4075@sparrow.telecommunity.com> Message-ID: <3F7B2BF0-34A6-4187-B91D-7DF73E74AD0E@cs.toronto.edu> On 15-Feb-08, at 3:47 PM, Phillip J. Eby wrote: > At 10:35 AM 2/13/2008 -0500, David Wolever wrote: >> It looks like the version of my .svn/entries file is newer (head - >> n1 .svn/entries: 9). > Hm. Does it detect the correct files, if you change the > 'data.startswith("8")' to 'data.startswith("9")'? If I change the 9 to an 8 in trunk/.svn/entries, seems to get the revision number on the egg correct. From pje at telecommunity.com Fri Feb 15 23:17:48 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 17:17:48 -0500 Subject: [Distutils] Including directories in .eggs? In-Reply-To: References: Message-ID: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> At 03:44 PM 2/15/2008 -0500, David Wolever wrote: >Hey, >I'm wondering how I would go about including an entire directory in >the .egg setup.py creates. We are using Dojo, and it automatically >builds a directory containing the "stuff" it needs... But just >including that directory in package_data doesn't work (as it expects >to copy a file, not a directory). Including a glob such as 'js/**/ >**' also fails, because directories are listed as part of the '**/ >**', and it can't copy those. Use 'include_package_data=True', and put the files under revision control. If that doesn't work, you're not using revision control, or just don't want to put them in revision control, you'll have to list the directory in a MANIFEST.in file: see the distutils docs ("Distributing Python Modules") from python.org for the specific format. From pje at telecommunity.com Fri Feb 15 23:20:52 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 17:20:52 -0500 Subject: [Distutils] Bug in setuptools/command/sdist.py: global name 'log' is not defined In-Reply-To: <3F7B2BF0-34A6-4187-B91D-7DF73E74AD0E@cs.toronto.edu> References: <20080215204741.2844B3A4075@sparrow.telecommunity.com> <3F7B2BF0-34A6-4187-B91D-7DF73E74AD0E@cs.toronto.edu> Message-ID: <20080215222039.903FD3A4075@sparrow.telecommunity.com> At 03:58 PM 2/15/2008 -0500, David Wolever wrote: >On 15-Feb-08, at 3:47 PM, Phillip J. Eby wrote: > > At 10:35 AM 2/13/2008 -0500, David Wolever wrote: > >> It looks like the version of my .svn/entries file is newer (head - > >> n1 .svn/entries: 9). > > Hm. Does it detect the correct files, if you change the > > 'data.startswith("8")' to 'data.startswith("9")'? >If I change the 9 to an 8 in trunk/.svn/entries, seems to get the >revision number on the egg correct. I'm asking if changing the code in setuptools causes it to detect the correct files for inclusion in the .egg-info's SOURCES.txt. The revision number has nothing to do with the error you were getting, and is read using entirely different code. To be sure of getting the right info, delete the SOURCES.txt, then run "setup.py egg_info" and see if the regenerated SOURCES.txt lists all (and only) files that are under revision control in your project. From wolever at cs.toronto.edu Fri Feb 15 23:21:30 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 15 Feb 2008 17:21:30 -0500 Subject: [Distutils] Including directories in .eggs? In-Reply-To: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> References: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> Message-ID: On 15-Feb-08, at 5:17 PM, Phillip J. Eby wrote: > If that doesn't work, you're not using revision control, or just > don't want to put them in revision control, you'll have to list the > directory in a MANIFEST.in file: see the distutils docs > ("Distributing Python Modules") from python.org for the specific > format. Alright, I'll check out this MANIFEST.in file -- it seems silly to put auto-generated files under version control. Thanks for the help, David From wolever at cs.toronto.edu Fri Feb 15 23:30:00 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 15 Feb 2008 17:30:00 -0500 Subject: [Distutils] Bug in setuptools/command/sdist.py: global name 'log' is not defined In-Reply-To: <20080215222039.903FD3A4075@sparrow.telecommunity.com> References: <20080215204741.2844B3A4075@sparrow.telecommunity.com> <3F7B2BF0-34A6-4187-B91D-7DF73E74AD0E@cs.toronto.edu> <20080215222039.903FD3A4075@sparrow.telecommunity.com> Message-ID: <2E7A7A7D-A382-4FE7-AFD0-6A553F1BA73A@cs.toronto.edu> On 15-Feb-08, at 5:20 PM, Phillip J. Eby wrote: > To be sure of getting the right info, delete the SOURCES.txt, then > run "setup.py egg_info" and see if the regenerated SOURCES.txt > lists all (and only) files that are under revision control in your > project. Alright, I've changed sdist.py to: if data.startswith('8') or data.startswith('9'): # subversion 1.4 Deleted SOURCES.txt setup.py egg_info then diffed SOURCES.txt with `svn list -R`, and they seem to be identical. FYI, I'm using svn 1.6.0 -- it's an SVN build which hasn't been released yet. David From wolever at cs.toronto.edu Sat Feb 16 00:26:39 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 15 Feb 2008 18:26:39 -0500 Subject: [Distutils] Including directories in .eggs? In-Reply-To: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> References: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> Message-ID: <55FE0095-1481-4A0B-9567-4A9B508A296C@cs.toronto.edu> On 15-Feb-08, at 5:17 PM, Phillip J. Eby wrote: > If that doesn't work, you're not using revision control, or just > don't want to put them in revision control, you'll have to list the > directory in a MANIFEST.in file: see the distutils docs > ("Distributing Python Modules") from python.org for the specific > format. Alright, I've put in my MANIFEST.in file: recursive-include path/to/dojo/* But when I run ``python setup.py install`` the files don't appear in the egg that is installed to .../site-packages/ The SOURCES.txt file DOES have the correct list of files, though, and when I use ``setup.py sdist`` the correct files are included there too. Is there some subtle difference between an "install" and an "sdist" that I'm missing? From pje at telecommunity.com Sat Feb 16 00:58:34 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 18:58:34 -0500 Subject: [Distutils] Including directories in .eggs? In-Reply-To: <55FE0095-1481-4A0B-9567-4A9B508A296C@cs.toronto.edu> References: <20080215221736.ACCE03A4075@sparrow.telecommunity.com> <55FE0095-1481-4A0B-9567-4A9B508A296C@cs.toronto.edu> Message-ID: <20080215235822.1FC263A4075@sparrow.telecommunity.com> At 06:26 PM 2/15/2008 -0500, David Wolever wrote: >On 15-Feb-08, at 5:17 PM, Phillip J. Eby wrote: >>If that doesn't work, you're not using revision control, or just >>don't want to put them in revision control, you'll have to list the >>directory in a MANIFEST.in file: see the distutils docs >>("Distributing Python Modules") from python.org for the specific >>format. >Alright, I've put in my MANIFEST.in file: >recursive-include path/to/dojo/* > >But when I run ``python setup.py install`` the files don't appear in >the egg that is installed to .../site-packages/ > >The SOURCES.txt file DOES have the correct list of files, though, and >when I use ``setup.py sdist`` the correct files are included there too. Did you set 'include_package_data=True' in your setup? Without that, setuptools doesn't know you want those files to be installed, vs. just shipping with your source distribution. >Is there some subtle difference between an "install" and an "sdist" >that I'm missing? Yes: source distributions can include lots of stuff you don't intend to install, such as readme files, tests, documentation, samples, etc. Eggs only contain the bits marked as needing to be installed with the actual library. From s.mientki at ru.nl Sat Feb 16 12:19:46 2008 From: s.mientki at ru.nl (Stef Mientki) Date: Sat, 16 Feb 2008 12:19:46 +0100 Subject: [Distutils] MatPlotLib + Py2exe, backend problems ... Message-ID: <47B6C6D2.7000103@ru.nl> hello, I'm trying to create a distro with py2exe, including MatPlotLib. After finally resolving the matplotlib datapath problem, I bumped into a new problem concerning backends. I'm using MatPlotLib 0.91.2 I only need the wx-Backend. In Py2exe I tried to exclude as many backends as possible, but the excludes seems to have no effect. 'excludes' : [ '_gtkagg', '_tkagg', '_agg2', '_cairo', '_cocoaagg', '_fltkagg', '_gtk', '_gtkcairo', 'backend_qt', 'backend_qt4', 'backend_qt4agg', 'backend_qtagg', 'backend_cairo', 'backend_cocoaagg', 'Tkconstants', 'Tkinter', 'tcl', "_imagingtk", "PIL._imagingtk", "ImageTk", "PIL.ImageTk", "FixTk" ], The creating of the distro seems to work, but the distro won't run, because of some problems with tk-backends, (which I don't need or even want). The problem might be caused by a crashed installation of tk in the past. And might also be amplified by the fact that I use from pylab import * Now I guess I can effectively remove the unnecessary backends, by removing the following lines from \backends\__init__.py interactive_bk = ['GTK', 'GTKAgg', 'GTKCairo', 'FltkAgg', 'QtAgg', 'Qt4Agg', 'TkAgg', 'WX', 'WXAgg', 'CocoaAgg'] non_interactive_bk = ['Agg2', 'Agg', 'Cairo', 'EMF', 'GDK', 'Pdf', 'PS', 'SVG', 'Template'] I don't know if this solves the problem (haven't tried it yet), but I certainly thinks this is not the proper way to go. Any solution / suggestion would be very welcome (while MatPlotLib is quit essential in my distro) thanks, Stef Mientki From cz at gocept.com Mon Feb 18 11:25:03 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Mon, 18 Feb 2008 11:25:03 +0100 Subject: [Distutils] buildout: setting envirionment variables for zc.recipe.egg:custom? References: Message-ID: On 2008-02-11 19:08:56 +0100, Jim Fulton said: > I suggest updating zc.recipe.egg:custom to accept environment- > variable, assuming that setting environment variables is sufficient. I added setting the environment in r84015. Jim, could you review that? Thanks, -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From cz at gocept.com Mon Feb 18 13:44:25 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Mon, 18 Feb 2008 13:44:25 +0100 Subject: [Distutils] buildout: setting envirionment variables for zc.recipe.egg:custom? References: Message-ID: On 2008-02-18 11:25:03 +0100, Christian Zagrodnick said: > On 2008-02-11 19:08:56 +0100, Jim Fulton said: > >> I suggest updating zc.recipe.egg:custom to accept environment- >> variable, assuming that setting environment variables is sufficient. > > I added setting the environment in r84015. Jim, could you review that? I added some fixes and did some cleanup in r84019. -- Christian Zagrodnick gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale www.gocept.com ? fon. +49 345 12298894 ? fax. +49 345 12298891 From jdh2358 at gmail.com Mon Feb 18 16:17:49 2008 From: jdh2358 at gmail.com (John Hunter) Date: Mon, 18 Feb 2008 09:17:49 -0600 Subject: [Distutils] [Matplotlib-users] MatPlotLib + Py2exe, backend problems ... In-Reply-To: <47B6C6D2.7000103@ru.nl> References: <47B6C6D2.7000103@ru.nl> Message-ID: <88e473830802180717n101870bct43b47ef13d07ebf4@mail.gmail.com> On Feb 16, 2008 5:19 AM, Stef Mientki wrote: > In Py2exe I tried to exclude as many backends as possible, > but the excludes seems to have no effect. > 'excludes' : [ > '_gtkagg', '_tkagg', '_agg2', '_cairo', '_cocoaagg', > '_fltkagg', '_gtk', '_gtkcairo', > 'backend_qt', 'backend_qt4', 'backend_qt4agg', > 'backend_qtagg', > 'backend_cairo', 'backend_cocoaagg', > 'Tkconstants', 'Tkinter', 'tcl', > "_imagingtk", "PIL._imagingtk", "ImageTk", "PIL.ImageTk", "FixTk" > ], I am not sure why the excluding is not working, but it shouldn't affect you app. If you are setting the backend in matplotlibrc, and successfully distributing this file, then matplotlib will only try and load that backend and the presence or abscense of the others shouldn't matter. In recent versions of matplotlib (eg the 91.2 version you are using), you can control the build process with setup.cfg. JDH From s.mientki at ru.nl Mon Feb 18 20:11:30 2008 From: s.mientki at ru.nl (Stef Mientki) Date: Mon, 18 Feb 2008 20:11:30 +0100 Subject: [Distutils] [wxPython-users] Re: [Matplotlib-users] MatPlotLib + Py2exe, backend problems ... In-Reply-To: <88e473830802180717n101870bct43b47ef13d07ebf4@mail.gmail.com> References: <47B6C6D2.7000103@ru.nl> <88e473830802180717n101870bct43b47ef13d07ebf4@mail.gmail.com> Message-ID: <47B9D862.1030406@ru.nl> John Hunter wrote: > On Feb 16, 2008 5:19 AM, Stef Mientki wrote: > > >> In Py2exe I tried to exclude as many backends as possible, >> but the excludes seems to have no effect. >> 'excludes' : [ >> '_gtkagg', '_tkagg', '_agg2', '_cairo', '_cocoaagg', >> '_fltkagg', '_gtk', '_gtkcairo', >> 'backend_qt', 'backend_qt4', 'backend_qt4agg', >> 'backend_qtagg', >> 'backend_cairo', 'backend_cocoaagg', >> 'Tkconstants', 'Tkinter', 'tcl', >> "_imagingtk", "PIL._imagingtk", "ImageTk", "PIL.ImageTk", "FixTk" >> ], >> > > I am not sure why the excluding is not working, but it shouldn't > affect you app. If you are setting the backend in matplotlibrc, and > successfully distributing this file, then matplotlib will only try and > load that backend and the presence or abscense of the others shouldn't > matter. In recent versions of matplotlib (eg the 91.2 version you are > using), you can control the build process with setup.cfg. > What is setup.cfg, where can I find it and where can I get information about it ? thanks, Stef Mientki > JDH > > --------------------------------------------------------------------- > To unsubscribe, e-mail: wxPython-users-unsubscribe at lists.wxwidgets.org > For additional commands, e-mail: wxPython-users-help at lists.wxwidgets.org > > > > From pjenvey at underboss.org Tue Feb 19 05:32:45 2008 From: pjenvey at underboss.org (Philip Jenvey) Date: Mon, 18 Feb 2008 20:32:45 -0800 Subject: [Distutils] setuptools 0.6c8 released In-Reply-To: <20080215185130.E1B333A40AE@sparrow.telecommunity.com> References: <20080215185130.E1B333A40AE@sparrow.telecommunity.com> Message-ID: <2723ED55-FE2A-45C4-AAE2-0615949EEEC3@underboss.org> On Feb 15, 2008, at 10:51 AM, Phillip J. Eby wrote: > This is the last 0.6 candidate release, barring regressions > introduced by changes in this release. In one week, if there are no > regressions, 0.6 final will be released, so please report any bugs > promptly. > > 0.6final will be the last OSAF-funded release of setuptools, as it > already exceeds the Chandler desktop's current and future > requirements, at least for the next year. > > There remain various known issues and limitations, but these won't be > fixed in the 0.6 line. > Did you see my last comment on the Jython thread, about eggs created on Jython? I was hoping you'd comment: http://mail.python.org/pipermail/distutils-sig/2008-January/008715.html Also, I just came across an issue while installing simplejson on Jython, but I noticed it's also been reported on other platforms (native_libs.txt doesn't exist). http://mail.python.org/pipermail/python-list/2007-December/469174.html http://ubuntuforums.org/archive/index.php/t-610534.html It looks like it's due to the fact that simplejson includes an optional C speedup extension. It overrides the distutils build_ext command so it can swallow any compiler errors (and skip building the extension). I guess it must be faking out setuptools into thinking there'll be a native_libs.txt when there won't be. I'm going to investigate it more this week. I also just noticed there's problems with ez_setup.py on jython on Windows -- though this might not be setuptools fault -- just FYI, it's on my radar to investigate that as well before your final release. -- Philip Jenvey From ziade.tarek at gmail.com Tue Feb 19 14:42:58 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Tue, 19 Feb 2008 05:42:58 -0800 (PST) Subject: [Distutils] setuptools, removing installed eggs Message-ID: <15560799.post@talk.nabble.com> Hello, I am trying to write a script that removes all traces of an egg. I am not talking about a clever thing that follows dependencies, but I have faced issues where my fellow developers had a remaining "develop" eggs that had to be removed. So an "easy_remove" script that cleanly erase all occurences of a given egg would be good ihmo. In my understanding, the script should: - scan all entries in pkg_resources.working_set.entries, to get all instances of the given egg (folder, archive and link) - call easy_install -m package - remove all entries founded before. Is that right ? Cheers ++ Tarek -- View this message in context: http://www.nabble.com/setuptools%2C-removing-installed-eggs-tp15560799p15560799.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From pje at telecommunity.com Tue Feb 19 15:16:51 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 19 Feb 2008 09:16:51 -0500 Subject: [Distutils] setuptools, removing installed eggs In-Reply-To: <15560799.post@talk.nabble.com> References: <15560799.post@talk.nabble.com> Message-ID: <20080219141637.77AC53A40A5@sparrow.telecommunity.com> At 05:42 AM 2/19/2008 -0800, Tarek Ziad?? wrote: >Hello, > >I am trying to write a script that removes all traces of an egg. >I am not talking about a clever thing that follows dependencies, >but I have faced issues where my fellow developers had a remaining >"develop" eggs that had to be removed. Develop eggs can (and should!) be uninstalled using "setup.py develop -u". >So an "easy_remove" script that cleanly erase all occurences of a given egg >would be good ihmo. > >In my understanding, the script should: > >- scan all entries in pkg_resources.working_set.entries, to get all >instances > of the given egg (folder, archive and link) > >- call easy_install -m package > >- remove all entries founded before. > >Is that right ? It would probably be simpler to just read and edit a specific easy-install.pth file, using ``pkg_resources.find_distributions(path, True)``. If you find a matching distribution, remove the line from the file, and the egg file/directory if appropriate. (Of course, you will need to skip lines beginning with "import ", and leave them in place.) If you want to remove all eggs for a project, whether active or not, you can indeed scan everything on sys.path. The easiest way to do that is just to do: for dist in pkg_resources.Environment()[project_name]: # rm -rf dist.location if appropriate Of course, you will need to make sure you only delete a distribution (in either case) if ``dist.precedence==pkg_resources.EGG_DIST``. From pje at telecommunity.com Tue Feb 19 16:08:28 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 19 Feb 2008 10:08:28 -0500 Subject: [Distutils] setuptools 0.6c8 released In-Reply-To: <2723ED55-FE2A-45C4-AAE2-0615949EEEC3@underboss.org> References: <20080215185130.E1B333A40AE@sparrow.telecommunity.com> <2723ED55-FE2A-45C4-AAE2-0615949EEEC3@underboss.org> Message-ID: <20080219150814.BF5CC3A4088@sparrow.telecommunity.com> At 08:32 PM 2/18/2008 -0800, Philip Jenvey wrote: >On Feb 15, 2008, at 10:51 AM, Phillip J. Eby wrote: > >>This is the last 0.6 candidate release, barring regressions >>introduced by changes in this release. In one week, if there are no >>regressions, 0.6 final will be released, so please report any bugs >>promptly. >> >>0.6final will be the last OSAF-funded release of setuptools, as it >>already exceeds the Chandler desktop's current and future >>requirements, at least for the next year. >> >>There remain various known issues and limitations, but these won't be >>fixed in the 0.6 line. > >Did you see my last comment on the Jython thread, about eggs created >on Jython? I was hoping you'd comment: > >http://mail.python.org/pipermail/distutils-sig/2008-January/008715.html Yeah, I don't really know what to say about that. >Also, I just came across an issue while installing simplejson on >Jython, but I noticed it's also been reported on other platforms >(native_libs.txt doesn't exist). > >http://mail.python.org/pipermail/python-list/2007-December/469174.html > >http://ubuntuforums.org/archive/index.php/t-610534.html > >It looks like it's due to the fact that simplejson includes an >optional C speedup extension. It overrides the distutils build_ext >command so it can swallow any compiler errors (and skip building the >extension). I guess it must be faking out setuptools into thinking >there'll be a native_libs.txt when there won't be. I'm going to >investigate it more this week. Hrm. It's actually a setuptools bug. The bdist_egg command either creates or deletes native_libs.txt in the working .egg-info directory, then copies to the built egg's EGG-INFO. However, there's no reason for it to be in .egg-info, since native_libs.txt is only relevant for zipped .egg files. The specific problem is that the source distribution of simplejson contains a native_libs.txt, which is therefore listed in the SOURCES.txt... which is then used by the metadata copier to copy it into EGG-INFO... right after it gets deleted by bdist_egg. So, basically, bdist_egg should be writing or delete native_libs.txt only from EGG-INFO, and to fix the problem in existing builds, the egg_info command will need to always delete native_libs from the .egg-info. >I also just noticed there's problems with ez_setup.py on jython on >Windows -- though this might not be setuptools fault -- just FYI, >it's on my radar to investigate that as well before your final release. Ok. From ziade.tarek at gmail.com Wed Feb 20 15:43:23 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Wed, 20 Feb 2008 06:43:23 -0800 (PST) Subject: [Distutils] distutils register and upload improvments Message-ID: <15590437.post@talk.nabble.com> Hello, On next saturday (Python bug day), I will ask for my patch to be included in Python 2.6 distutils. (see http://wiki.python.org/moin/EnhancedPyPI, first part) I have made a package that provides the same features in Python 2.4 and 2.5 in two new commands called "mregister" and "mupload". It is available at the cheeseshop here : http://pypi.python.org/pypi/iw.dist Besides the multiple server approach, it provides a few other improvements: - the "-r" option was not caught in a command line that has several distutils commands - the "HOME" environment variable does not exists under some Windows, so it now uses the proper HOMEDRIVE+HOMEPATH ones in that case - the realm can now be changed for the registering authentication process Please let me know if you use it ! ++ Tarek -- View this message in context: http://www.nabble.com/distutils-register-and-upload-improvments-tp15590437p15590437.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From noah.gift at gmail.com Wed Feb 20 16:42:03 2008 From: noah.gift at gmail.com (Noah Gift) Date: Wed, 20 Feb 2008 10:42:03 -0500 Subject: [Distutils] Creating Dynamic Command Line Options from Python Egg Plugins Message-ID: <345F2028-3623-4520-8085-008DEFF0C889@gmail.com> Has anyone created a command line tool, using optparse, that dynamically creates options from available eggs/plugins? I have a need to create such a tool, and was looking for an example. Noah Gift From jtk at yahoo.com Wed Feb 20 22:33:14 2008 From: jtk at yahoo.com (Jeff Kowalczyk) Date: Wed, 20 Feb 2008 16:33:14 -0500 Subject: [Distutils] Error: Couldn't install when corrupt/incomplete .tar.gz download Message-ID: I recently encountered setuptools errors for buildouts depending a specific Zope package. I later discovered that the package's .tar.gz had been only partially downloaded due a poor internet connection. Could setuptools give a more helpful error on this condition? Thanks. Getting distribution for 'ZConfig'. error: None An error occured when trying to install ZConfig 2.5.1.Look above this message for any errors thatwere output by easy_install. While: Installing test. Getting distribution for 'ZConfig'. Error: Couldn't install: ZConfig 2.5.1 Getting distribution for 'ZConfig>=2.4a5'. error: None An error occured when trying to install ZConfig 2.5.1.Look above this message for any errors thatwere output by easy_install. While: Installing. Getting section demo. Initializing section demo. Installing recipe zc.zope3recipes. Getting distribution for 'ZConfig>=2.4a5'. Error: Couldn't install: ZConfig 2.5.1 $ find ~/.buildout -name 'ZConfig*' ~/.buildout/zope/Zope-2.10.5-final/lib/python/ZConfig ~/.buildout/downloads/ZConfig-2.5.1.tar.gz ~/.buildout/downloads/dist/ZConfig-2.5.1.tar.gz From b3i4old02 at sneakemail.com Thu Feb 21 02:13:00 2008 From: b3i4old02 at sneakemail.com (Michael Hoffman) Date: Thu, 21 Feb 2008 01:13:00 +0000 Subject: [Distutils] Creating Dynamic Command Line Options from Python Egg Plugins In-Reply-To: <345F2028-3623-4520-8085-008DEFF0C889@gmail.com> References: <345F2028-3623-4520-8085-008DEFF0C889@gmail.com> Message-ID: Noah Gift wrote: > Has anyone created a command line tool, using optparse, that > dynamically creates options from available eggs/plugins? I have a > need to create such a tool, and was looking for an example. Yeah, I did something like this, which is somewhat clunky: def parse_options(args): usage = """%prog [OPTION...] MODEL SEQFILE OUTFILE %prog [OPTION...] --driver=DRIVER MODEL SEQFILE [DRIVEROPTIONS...]""" version = "%%prog %s" % __version__ parser = OptionParser(usage=usage, version=version) parser.disable_interspersed_args() with OptionGroup(parser, "Output driver") as group: group.add_option("-d", "--driver", default="DEFAULT", help="set output format") group.add_option("--driver-help", action="store_true", help="get driver-specific help") options, args = parser.parse_args(args) if options.driver_help: driver_class = driver_load(options.driver) driver_class.print_help() sys.exit(0) def driver_load(name): return load_entry_point("myproject", "myproject.drivers", name) And I defined the entry points in my setup.py file: ==== [myproject.drivers] DEFAULT = myproject.driver.hdf5:HDF5Driver hdf5 = myproject.driver.hdf5:HDF5Driver sql = myproject.driver.sql:SQLDriver ==== At some point I pass unprocessed args to the entry point, which parse them with optparse. I could call the program like this: myproject --driver=sql model seqfile --database=server.example.net From noah.gift at gmail.com Thu Feb 21 15:51:01 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 21 Feb 2008 09:51:01 -0500 Subject: [Distutils] Creating Dynamic Command Line Options from Python Egg Plugins In-Reply-To: References: <345F2028-3623-4520-8085-008DEFF0C889@gmail.com> Message-ID: <0670C336-9FB0-485B-BAD9-10772D80EF6C@gmail.com> On Feb 20, 2008, at 8:13 PM, Michael Hoffman wrote: > Noah Gift wrote: >> Has anyone created a command line tool, using optparse, that >> dynamically creates options from available eggs/plugins? I have a >> need to create such a tool, and was looking for an example. > Thanks Michael. That gives me some good ideas to start with! > Yeah, I did something like this, which is somewhat clunky: > > def parse_options(args): > usage = """%prog [OPTION...] MODEL SEQFILE OUTFILE > %prog [OPTION...] --driver=DRIVER MODEL SEQFILE > [DRIVEROPTIONS...]""" > version = "%%prog %s" % __version__ > parser = OptionParser(usage=usage, version=version) > parser.disable_interspersed_args() > > > with OptionGroup(parser, "Output driver") as group: > group.add_option("-d", "--driver", default="DEFAULT", > help="set output format") > > group.add_option("--driver-help", action="store_true", > help="get driver-specific help") > > options, args = parser.parse_args(args) > > if options.driver_help: > driver_class = driver_load(options.driver) > driver_class.print_help() > sys.exit(0) > > def driver_load(name): > return load_entry_point("myproject", "myproject.drivers", name) > > And I defined the entry points in my setup.py file: > > ==== > [myproject.drivers] > DEFAULT = myproject.driver.hdf5:HDF5Driver > hdf5 = myproject.driver.hdf5:HDF5Driver > sql = myproject.driver.sql:SQLDriver > ==== > > At some point I pass unprocessed args to the entry point, which parse > them with optparse. I could call the program like this: > > myproject --driver=sql model seqfile --database=server.example.net > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig Noah Gift / http://noahgift.com [Python+Grok+AJAX+Mashup] From b3i4old02 at sneakemail.com Thu Feb 21 16:19:22 2008 From: b3i4old02 at sneakemail.com (Michael Hoffman) Date: Thu, 21 Feb 2008 15:19:22 +0000 Subject: [Distutils] Creating Dynamic Command Line Options from Python Egg Plugins In-Reply-To: References: <345F2028-3623-4520-8085-008DEFF0C889@gmail.com> Message-ID: Michael Hoffman wrote: > with OptionGroup(parser, "Output driver") as group: > group.add_option("-d", "--driver", default="DEFAULT", > help="set output format") > > group.add_option("--driver-help", action="store_true", > help="get driver-specific help") I forgot that this bit requires making an OptionGroup wrapper. from optparse import OptionGroup as _OptionGroup class OptionGroup(_OptionGroup): def __enter__(self): return self def __exit__(self, *exc_info): self.parser.add_option_group(self) Also I have been thinking about this and I think the way I did it is clunkier than necessary. Better would be to do two passes of optparse--one to figure out what plugins to load, then load their options into the OptionParser, and parse the options again. This would require special handling for help and option conflicts, but would be a better user experience. From pjenvey at underboss.org Sat Feb 23 00:51:30 2008 From: pjenvey at underboss.org (Philip Jenvey) Date: Fri, 22 Feb 2008 15:51:30 -0800 Subject: [Distutils] setuptools 0.6c8 released In-Reply-To: <20080219150814.BF5CC3A4088@sparrow.telecommunity.com> References: <20080215185130.E1B333A40AE@sparrow.telecommunity.com> <2723ED55-FE2A-45C4-AAE2-0615949EEEC3@underboss.org> <20080219150814.BF5CC3A4088@sparrow.telecommunity.com> Message-ID: On Feb 19, 2008, at 7:08 AM, Phillip J. Eby wrote: > At 08:32 PM 2/18/2008 -0800, Philip Jenvey wrote: >> >> It looks like it's due to the fact that simplejson includes an >> optional C speedup extension. It overrides the distutils build_ext >> command so it can swallow any compiler errors (and skip building the >> extension). I guess it must be faking out setuptools into thinking >> there'll be a native_libs.txt when there won't be. I'm going to >> investigate it more this week. > > Hrm. It's actually a setuptools bug. The bdist_egg command either > creates or deletes native_libs.txt in the working .egg-info > directory, then copies to the built egg's EGG-INFO. However, > there's no reason for it to be in .egg-info, since native_libs.txt > is only relevant for zipped .egg files. > > The specific problem is that the source distribution of simplejson > contains a native_libs.txt, which is therefore listed in the > SOURCES.txt... which is then used by the metadata copier to copy > it into EGG-INFO... right after it gets deleted by bdist_egg. > > So, basically, bdist_egg should be writing or delete > native_libs.txt only from EGG-INFO, and to fix the problem in > existing builds, the egg_info command will need to always delete > native_libs from the .egg-info. How does this patch look? I also made the extract_constant tests skip on Jython/IronPython, so every test should pass on these platforms. -------------- next part -------------- A non-text attachment was scrubbed... Name: native_libs-fix-r60849.diff Type: application/octet-stream Size: 3337 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080222/eb9552aa/attachment.obj -------------- next part -------------- > > >> I also just noticed there's problems with ez_setup.py on jython on >> Windows -- though this might not be setuptools fault -- just FYI, >> it's on my radar to investigate that as well before your final >> release. > > Ok. > We're ok for Jython on Windows, the problems I saw were all Jython's fault. What I was particularly worried about was stray open file handles -- they could prevent a file from being os.unlink'd (since Windows doesn't allow you to delete a file when it's still in use). This kind of thing is particularly evil for Jython because Java's gc may not finalize and close our left open file objects as immediately as CPython's does. setuptools didn't have any stray file handles that seemed to cause problems for me, but I went through the codebase and found some anyway, patch #2: -------------- next part -------------- A non-text attachment was scrubbed... Name: close_fp-r60849.diff Type: application/octet-stream Size: 3902 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080222/eb9552aa/attachment-0001.obj -------------- next part -------------- -- Philip Jenvey From optilude at gmx.net Sun Feb 24 23:48:55 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 24 Feb 2008 22:48:55 +0000 Subject: [Distutils] Putting egg-info under version control Message-ID: Hi, We normally put development eggs under version control while people are working on them. However, when they are installed (by buildout, in our case, but I assume "python setup.py develop" is similar), a .egg-info directory is created. Putting this under version control seems problematic, because developers tend to get conflicts in PKG-INFO and similar files. Thus, I tend to set svn ignores on the whole directory. People have to re-do the develop egg installation locally anyway. However, when using Paster with local commands, it writes a paster_plugins.txt file to the egg-info which doesn't seem to be re-created when I re-run buildout (again, I assume "python setup.py develop" would behave in the same way). That means that for Paster local commands to continue to work, I have to set ignores on almost everything in the egg-info directory, but not that file, which is cumbersome and error prone. Am I missing something here? What's the best strategy for putting develop eggs under version control? Is this an issue with Paster or with eggs in general? I'm assuming other things that extend the egg-info would have similar problems. Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From tseaver at palladion.com Sun Feb 24 23:59:07 2008 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 24 Feb 2008 17:59:07 -0500 Subject: [Distutils] Putting egg-info under version control In-Reply-To: References: Message-ID: <47C1F6BB.7010500@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Aspeli wrote: > Hi, > > We normally put development eggs under version control while people are > working on them. However, when they are installed (by buildout, in our > case, but I assume "python setup.py develop" is similar), a > .egg-info directory is created. > > Putting this under version control seems problematic, because developers > tend to get conflicts in PKG-INFO and similar files. Thus, I tend to set > svn ignores on the whole directory. People have to re-do the develop egg > installation locally anyway. > > However, when using Paster with local commands, it writes a > paster_plugins.txt file to the egg-info which doesn't seem to be > re-created when I re-run buildout (again, I assume "python setup.py > develop" would behave in the same way). That means that for Paster local > commands to continue to work, I have to set ignores on almost everything > in the egg-info directory, but not that file, which is cumbersome and > error prone. > > Am I missing something here? What's the best strategy for putting > develop eggs under version control? Is this an issue with Paster or with > eggs in general? I'm assuming other things that extend the egg-info > would have similar problems. IMNSHO, anything in the .egg-info directory should be considered a derived file, and therefore NOT placed under version control (we don't check in .o / .so / .dll files either). If some tool can't recreate the files it needs on demand in that directory, that would be a bug in that tool (or perhaps the mode in which it is invoked). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHwfa7+gerLs4ltQ4RAhUtAJ90mPKhAPJq9jR/nL6ficFZf0dFagCgjw95 D7/1MTffTOShB20sn/ca3Hw= =WCSa -----END PGP SIGNATURE----- From optilude at gmx.net Mon Feb 25 00:31:12 2008 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 24 Feb 2008 23:31:12 +0000 Subject: [Distutils] Putting egg-info under version control In-Reply-To: <47C1F6BB.7010500@palladion.com> References: <47C1F6BB.7010500@palladion.com> Message-ID: Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Martin Aspeli wrote: >> Hi, >> >> We normally put development eggs under version control while people are >> working on them. However, when they are installed (by buildout, in our >> case, but I assume "python setup.py develop" is similar), a >> .egg-info directory is created. >> >> Putting this under version control seems problematic, because developers >> tend to get conflicts in PKG-INFO and similar files. Thus, I tend to set >> svn ignores on the whole directory. People have to re-do the develop egg >> installation locally anyway. >> >> However, when using Paster with local commands, it writes a >> paster_plugins.txt file to the egg-info which doesn't seem to be >> re-created when I re-run buildout (again, I assume "python setup.py >> develop" would behave in the same way). That means that for Paster local >> commands to continue to work, I have to set ignores on almost everything >> in the egg-info directory, but not that file, which is cumbersome and >> error prone. >> >> Am I missing something here? What's the best strategy for putting >> develop eggs under version control? Is this an issue with Paster or with >> eggs in general? I'm assuming other things that extend the egg-info >> would have similar problems. > > IMNSHO, anything in the .egg-info directory should be considered a > derived file, and therefore NOT placed under version control (we don't > check in .o / .so / .dll files either). If some tool can't recreate the > files it needs on demand in that directory, that would be a bug in that > tool (or perhaps the mode in which it is invoked). That was my impression too. If that's a safe assumption to make, then we can continue to say that *.egg-info goes in svn:ignores. I can't figure out from http://pythonpaste.org/script/developer.html how the files are supposed to be created though. Wiping egg-info and re-running buildout certainly doesn't re-create it. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From pje at telecommunity.com Mon Feb 25 02:16:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 24 Feb 2008 20:16:42 -0500 Subject: [Distutils] Putting egg-info under version control In-Reply-To: <47C1F6BB.7010500@palladion.com> References: <47C1F6BB.7010500@palladion.com> Message-ID: <20080225011805.A00203A40B0@sparrow.telecommunity.com> At 05:59 PM 2/24/2008 -0500, Tres Seaver wrote: >IMNSHO, anything in the .egg-info directory should be considered a >derived file, and therefore NOT placed under version control (we don't >check in .o / .so / .dll files either). If some tool can't recreate the >files it needs on demand in that directory, that would be a bug in that >tool (or perhaps the mode in which it is invoked). Your NSHO is incorrect for many projects. Setuptools itself has some .egg-info files under revision control, for example, because they are needed to bootstrap setuptools itself. Other projects have metadata whose official source is under .egg-info. The EggTranslations library used by Chandler, for example, puts i18n/l10n resource files under .egg-info, and as such they are tracked by revision control. From ianb at colorstudy.com Mon Feb 25 08:47:26 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 25 Feb 2008 01:47:26 -0600 Subject: [Distutils] Putting egg-info under version control In-Reply-To: References: <47C1F6BB.7010500@palladion.com> Message-ID: <47C2728E.3000100@colorstudy.com> Martin Aspeli wrote: > That was my impression too. If that's a safe assumption to make, then we > can continue to say that *.egg-info goes in svn:ignores. > > I can't figure out from http://pythonpaste.org/script/developer.html how > the files are supposed to be created though. Wiping egg-info and > re-running buildout certainly doesn't re-create it. paster create normally adds the .egg-info directory, but ignores all the Setuptools-created files, and some Paste packages (including paster create) add files in that directory. I won't say that's a good idea, and I probably wouldn't go in this direction if starting from scratch... putting just one or two files under version control, but ignoring the rest, is quite awkward. I usually set svn:ignore in the egg-info directory to *, and then you can still opt-in specific files if you want to (or I leave egg-info out of version control entirely). Ian From noah.gift at gmail.com Mon Feb 25 14:01:56 2008 From: noah.gift at gmail.com (Noah Gift) Date: Mon, 25 Feb 2008 08:01:56 -0500 Subject: [Distutils] Using eggs as Plugins Message-ID: <150C9471-7943-4AC2-8B12-FEBA1C31DB98@gmail.com> Does anyone have any experiences they would like to share about how they use eggs as plugins in their personal projects and at their job? It would be great to know what people used before, perhaps a simple inspection of __init__.py directories, and how eggs have improved their plugin architecture. If you work at a Python centric company and would like to be profiled in an O'Reilly book on your use of eggs, please contact me as well. Noah Gift From faassen at startifact.com Tue Feb 26 00:12:21 2008 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 26 Feb 2008 00:12:21 +0100 Subject: [Distutils] buildout: trying to test versions/develop interaction Message-ID: Hi there, I'm trying to write a test that tests the integration between the 'versions' line and the 'develop' line in [buildout]. My expectation is to see the following behavior from buildout: if I list a version explicitly in 'versions', and list that package under 'develop', I expect it to pick the version listed in [versions] and ignore the version under 'develop'. To verify this behavior, I tried to write a test. Unfortunately the test failed, and seems to be preferentially getting the version listed under 'develop'. My work is on a branch; svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/faassen-develop Incidentally, when I try running the tests on trunk (without my additions) in Linux, I get a number of test failures. I wonder whether this has anything to do with the Windows-compatibility changes that were added a few days ago? (which I see in SVN) Regards, Martijn From lists at durin42.com Tue Feb 26 05:20:57 2008 From: lists at durin42.com (Augie Fackler) Date: Mon, 25 Feb 2008 22:20:57 -0600 Subject: [Distutils] setuptools Fails with Subversion 1.5 Message-ID: <8FE3CD7E-E1DA-494D-BA12-892F2D1F8B04@durin42.com> Hey all, With Subversion 1.5 entering alpha, I've been using it for more than just a few projects, and I've noticed that setuptools fails to run in a 1.5 working copy. For the short term, I've found that I can just modify the setuptools source to trust that version 9 WCs will work the same as version 8 WCs. The diff for that is attached. Based on what I've read of subversion's README on the format of the entries file, the modification I've made should be just fine - the overall structure of the file is the same, there's just some new tags possible which will already not get stuck in the regex already in place. In my local testing, using setuptools for installing files works just fine with this patch in place. Peace, Augie Fackler -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: setuptools_patch.txt Url: http://mail.python.org/pipermail/distutils-sig/attachments/20080225/b5221847/attachment.txt -------------- next part -------------- From wolever at cs.toronto.edu Tue Feb 26 16:55:15 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Tue, 26 Feb 2008 10:55:15 -0500 Subject: [Distutils] setuptools Fails with Subversion 1.5 In-Reply-To: <8FE3CD7E-E1DA-494D-BA12-892F2D1F8B04@durin42.com> References: <8FE3CD7E-E1DA-494D-BA12-892F2D1F8B04@durin42.com> Message-ID: <66597D62-0263-4078-AFB4-B65049B3E266@cs.toronto.edu> I can confirm that, when this change is made, distutils can still get a list of visioned files (I'd like to say "everything works fine", but I suppose all I can really say is "I haven't had a problem with it yet). On 25-Feb-08, at 11:20 PM, Augie Fackler wrote: > Hey all, > > With Subversion 1.5 entering alpha, I've been using it for more > than just a few projects, and I've noticed that setuptools fails to > run in a 1.5 working copy. For the short term, I've found that I > can just modify the setuptools source to trust that version 9 WCs > will work the same as version 8 WCs. The diff for that is attached. > > Based on what I've read of subversion's README on the format of the > entries file, the modification I've made should be just fine - the > overall structure of the file is the same, there's just some new > tags possible which will already not get stuck in the regex already > in place. In my local testing, using setuptools for installing > files works just fine with this patch in place. > > Peace, > Augie Fackler > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From dodgyville at gmail.com Wed Feb 27 10:27:59 2008 From: dodgyville at gmail.com (Luke Miller) Date: Wed, 27 Feb 2008 20:27:59 +1100 Subject: [Distutils] project distribution Message-ID: Hi, I've been working on a game and engine written in python, and I'm at the stage where I am thinking about release engineering. Obviously distutils has been used up until now for packaging and installation on my test rig. However, I would like to create an iso of the game + source for distribution on CD (or DVD or multiple-CDs) My ideal is something like: python setup.py bdist --formats win32, linux, macOSx --type iso and it would create a "standard" game CD iso (or mutiple isos), where the user puts it in their computer and can install the game in the method traditional to the OS. A bonus would be if the three installs shared data, so there was only one copy of the music, art, game data, etc. Another bonus would be if it handled installing python on the target machine. As far as I can tell distutils "only" goes as far as packaging it up into a neat package. Does anyone have any stories on getting their python project into an iso-style format? Regards, Luke PS I am in awe of distutils. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080227/443ce239/attachment.htm From lists at zopyx.com Wed Feb 27 19:57:03 2008 From: lists at zopyx.com (Andreas Jung) Date: Wed, 27 Feb 2008 19:57:03 +0100 Subject: [Distutils] Introspection on eggs? Message-ID: <8EA4BD347CF40BB18889B9D5@[192.168.0.24]> Hi, I am currently working on a small PyPI replacement for hosting local and private eggs: I would like to some introspection on uploaded eggs - basically I am interested in the long_description and some other metadata without having to unpack the egg and parse it on my own. Is there some API available for performing such a task? Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080227/49668a19/attachment.pgp From pje at telecommunity.com Wed Feb 27 20:09:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 27 Feb 2008 14:09:53 -0500 Subject: [Distutils] Introspection on eggs? In-Reply-To: <8EA4BD347CF40BB18889B9D5@[192.168.0.24]> References: <8EA4BD347CF40BB18889B9D5@[192.168.0.24]> Message-ID: <20080227190940.56F873A4088@sparrow.telecommunity.com> At 07:57 PM 2/27/2008 +0100, Andreas Jung wrote: >Hi, > >I am currently working on a small PyPI replacement for hosting local >and private eggs: > > > >I would like to some introspection on uploaded eggs - basically I am >interested in the long_description and some other metadata without having to >unpack the egg and parse it on my own. Is there some API available >for performing such a task? You can use dist.get_metadata('PKG-INFO') (where dist is a Distribution object (egg) obtained via any of the usual pkg_resources APIs) to obtain the contents of the PKG-INFO file. You would need to parse it to get the long_description and other stuff out, but it's a pretty simple format. From dpeterson at enthought.com Wed Feb 27 20:10:44 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Wed, 27 Feb 2008 13:10:44 -0600 Subject: [Distutils] Introspection on eggs? In-Reply-To: <8EA4BD347CF40BB18889B9D5@[192.168.0.24]> References: <8EA4BD347CF40BB18889B9D5@[192.168.0.24]> Message-ID: <47C5B5B4.9000209@enthought.com> Andreas Jung wrote: > I am currently working on a small PyPI replacement for hosting local > and private eggs: > > > > I would like to some introspection on uploaded eggs - basically I am > interested in the long_description and some other metadata without > having to > unpack the egg and parse it on my own. Is there some API available > for performing such a task? I'm also interested in a response to this, particularly if it is better than what we already do at Enthought. :-) We basically unpack the EGG-INFO/PKG-INFO and EGG-INFO/requires.txt from the egg (you don't need to unpack the whole thing), concatenate those files into a single file, and then put that file up in our egg repository to ensure that tools browsing the repo don't have to download and unpack an egg in order to figure out what the egg is and what its dependencies are. Of course, we hadn't been trying to be a PyPi, but simply a web-accessible directory that stores eggs. -- Dave From pjenvey at underboss.org Thu Feb 28 04:19:15 2008 From: pjenvey at underboss.org (Philip Jenvey) Date: Wed, 27 Feb 2008 19:19:15 -0800 Subject: [Distutils] pkg_resources.declare_namespace inconsistency vs pkgutil.extend_path Message-ID: <9EF1582C-27F5-479E-AA94-0835CB3973E5@underboss.org> Paste uses pkg_resources.declare_namespace: >>> import sys >>> import paste >>> paste.__path__ ['/usr/local/lib/python2.5/site-packages/Paste-1.6-py2.5.egg/paste', '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.3.1-py2.5.egg/ paste', '/usr/local/lib/python2.5/site-packages/PasteScript-1.6.2- py2.5.egg/paste'] >>> del sys.modules['paste'] >>> import paste >>> paste.__path__ ['/usr/local/lib/python2.5/site-packages/Paste-1.6-py2.5.egg/paste'] Whereas pkgutil.extend_path does the right thing and restores paste.__path__ to contain the 3 paths -- Philip Jenvey From jeff at taupro.com Thu Feb 28 11:46:57 2008 From: jeff at taupro.com (Jeff Rush) Date: Thu, 28 Feb 2008 04:46:57 -0600 Subject: [Distutils] Seeking the Essence of the Concept of an Egg Message-ID: <47C69121.4020208@taupro.com> In preparing for a tutorial on Python eggs, I'm wrestling with the *precise* definition of a "Python egg". It appears to be a slippery term from the various documentation: "There are several binary formats that embody eggs, but the most common is '.egg' zipfile format, because it's a convenient one for distributing projects. All of the formats support including package-specific data, project-wide metadata, C extensions, and Python code." Ok, so what binary formats -other than- .egg files are actually eggs? I figure RPMs, .debs, etc. are eggs, as long as they have an EGG-INFO subdirectory and conform to certain basic structural rules. "Eggs are pluggable distributions in one of the three formats currently supported by ``pkg_resources``. There are built eggs, development eggs, and egg links." Hmmm, so it's not just binary distributions that can be eggs, but also egg-links and development eggs. The concept of a "built egg" egg still includes an RPM or .deb though. "Python Eggs are the preferred binary distribution format for EasyInstall, because they are cross-platform (for "pure" packages), directly importable, and contain project metadata including scripts and information about the project's dependencies. They can be simply downloaded and added to sys.path directly, or they can be placed in a directory on sys.path and then automatically discovered by the egg runtime system." Well drat, so RPMs and .debs are -not- eggs, because you don't just place them on sys.path directly to use them. Egg-links don't quite fit that definition either, unless you look at them funny. So what -other- binary format could that first paragraph be talking about that can be dropped onto sys.path and be picked up by Python? -Jeff Python eggs are so very zen; you can only understand them by already understanding them. From jim at zope.com Thu Feb 28 16:13:45 2008 From: jim at zope.com (Jim Fulton) Date: Thu, 28 Feb 2008 10:13:45 -0500 Subject: [Distutils] Seeking the Essence of the Concept of an Egg In-Reply-To: <47C69121.4020208@taupro.com> References: <47C69121.4020208@taupro.com> Message-ID: I think that the confusion comes from the blurring of eggs and setuptools. IMO, when most people use the term "egg", they are really referring to the packaging system provided by setuptools and easy_install (and the other applications, like buildout) that build on setuptools). IMO, the actual distribution format isn't that important. Source distributions have advantages over eggs for distribution in many cases, but what's really important is setuptools. Jim On Feb 28, 2008, at 5:46 AM, Jeff Rush wrote: > In preparing for a tutorial on Python eggs, I'm wrestling with the > *precise* > definition of a "Python egg". It appears to be a slippery term from > the > various documentation: > > "There are several binary formats that embody eggs, but the most > common is '.egg' zipfile format, because it's a convenient one > for distributing projects. All of the formats support including > package-specific data, project-wide metadata, C extensions, and > Python code." > > Ok, so what binary formats -other than- .egg files are actually > eggs? I > figure RPMs, .debs, etc. are eggs, as long as they have an EGG-INFO > subdirectory and conform to certain basic structural rules. > > "Eggs are pluggable distributions in one of the three formats > currently supported by ``pkg_resources``. There are built > eggs, development eggs, and egg links." > > Hmmm, so it's not just binary distributions that can be eggs, but also > egg-links and development eggs. The concept of a "built egg" egg > still > includes an RPM or .deb though. > > "Python Eggs are the preferred binary distribution format for > EasyInstall, because they are cross-platform (for "pure" > packages), directly importable, and contain project metadata > including scripts and information about the project's > dependencies. They can be simply downloaded and added to > sys.path directly, or they can be placed in a directory on > sys.path and then automatically discovered by the egg > runtime system." > > Well drat, so RPMs and .debs are -not- eggs, because you don't just > place them > on sys.path directly to use them. Egg-links don't quite fit that > definition > either, unless you look at them funny. So what -other- binary > format could > that first paragraph be talking about that can be dropped onto > sys.path and be > picked up by Python? > > -Jeff > > Python eggs are so very zen; you can only understand them by already > understanding them. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton Zope Corporation From pje at telecommunity.com Thu Feb 28 16:48:41 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 28 Feb 2008 10:48:41 -0500 Subject: [Distutils] Seeking the Essence of the Concept of an Egg In-Reply-To: <47C69121.4020208@taupro.com> References: <47C69121.4020208@taupro.com> Message-ID: <20080228154826.0DAF73A405C@sparrow.telecommunity.com> At 04:46 AM 2/28/2008 -0600, Jeff Rush wrote: >In preparing for a tutorial on Python eggs, I'm wrestling with the >*precise* definition of a "Python egg". It appears to be a slippery >term from the various documentation: > > "There are several binary formats that embody eggs, but the most > common is '.egg' zipfile format, because it's a convenient one > for distributing projects. All of the formats support including > package-specific data, project-wide metadata, C extensions, and > Python code." > >Ok, so what binary formats -other than- .egg files are actually >eggs? I figure RPMs, .debs, etc. are eggs, as long as they have an >EGG-INFO subdirectory and conform to certain basic structural rules. No - they're not. They *contain* eggs, they aren't eggs themselves. > "Eggs are pluggable distributions in one of the three formats > currently supported by ``pkg_resources``. There are built > eggs, development eggs, and egg links." > >Hmmm, so it's not just binary distributions that can be eggs, but >also egg-links and development eggs. The concept of a "built egg" >egg still includes an RPM or .deb though. Development eggs are .egg-info eggs. This definition *is* a bit slippery though, in that an egg link isn't "really" an egg, except in the sense that a symlink to a directory is a directory. :) If you want a solid definition, I'd say that built eggs (.egg file or directory) and development eggs (.egg-info eggs) are the only "real" eggs, as they are directly importable (as described below). > "Python Eggs are the preferred binary distribution format for > EasyInstall, because they are cross-platform (for "pure" > packages), directly importable, and contain project metadata > including scripts and information about the project's > dependencies. They can be simply downloaded and added to > sys.path directly, or they can be placed in a directory on > sys.path and then automatically discovered by the egg > runtime system." > >Well drat, so RPMs and .debs are -not- eggs, because you don't just >place them on sys.path directly to use them. Egg-links don't quite >fit that definition either, unless you look at them funny. So what >-other- binary format could that first paragraph be talking about >that can be dropped onto sys.path and be picked up by Python? There isn't one now - it was phrased that way to leave open future possibilities.