From pje at telecommunity.com Sun Aug 1 01:00:34 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 31 Jul 2010 19:00:34 -0400 Subject: [Distutils] How to customize egg_info behavior In-Reply-To: <12C7AB425F0DD546B6049311F827C74E08A37476C2@VA3DIAXVS141.RE D001.local> References: <12C7AB425F0DD546B6049311F827C74E08A37476C2@VA3DIAXVS141.RED001.local> Message-ID: <20100731230038.EE9153A40D9@sparrow.telecommunity.com> At 12:39 PM 7/31/2010 -0700, Jason R. Coombs wrote: >I'm using setuptools 0.6c11. > >I'd like to programmatically customize the way the egg_info command >is run. That is, in my setup.py, I would like to run some functions >to determine the tag-build, tag-date, and tag-svn-revision >parameters to egg_info. Are there parameters > >How is this task best accomplished? Do I subclass the egg_info >Command? If so, do I just specify >cmdclass={'egg_info':my_custom_egg_info} ? Or would you recommend >another approach? You can just do this in your setup script: setup( ... options = dict(egg_info = dict(tag_build = "whatever", ...)), ... ) Replacing "whatever", of course, with your programmatic calculation. From marius at pov.lt Sun Aug 1 01:19:55 2010 From: marius at pov.lt (Marius Gedminas) Date: Sun, 1 Aug 2010 02:19:55 +0300 Subject: [Distutils] GCC versions and binary egg names In-Reply-To: <4C545169.8020901@simplistix.co.uk> References: <4C4DC004.8060705@simplistix.co.uk> <4C545169.8020901@simplistix.co.uk> Message-ID: <20100731231954.GA10963@fridge.pov.lt> On Sat, Jul 31, 2010 at 05:38:01PM +0100, Chris Withers wrote: > Tres Seaver wrote: > >By never, ever, ever distributing binary eggs, especially for > >Linux. > > That's fine in a world without python packages with C and Fortran > extensions that are a royal PITA to compile. (fragile much?!) In my experience the difficulty is figuring out which non-Python dependencies you need installed (libraries, header files, compilers, build tools). Cheating (of the 'sudo apt-get build-dep python-numpy' kind) helps, if what you're trying to install is a (newer version of a) package that's already been packaged for your distribution. Is that what you experience too, or are there other sources of pain? Marius Gedminas -- Hanlon's Razor: Never attribute to malice that which is adequately explained by stupidity. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: Digital signature URL: From jaraco at jaraco.com Sun Aug 1 07:36:31 2010 From: jaraco at jaraco.com (Jason R. Coombs) Date: Sat, 31 Jul 2010 22:36:31 -0700 Subject: [Distutils] How to customize egg_info behavior In-Reply-To: <20100731230038.EE9153A40D9@sparrow.telecommunity.com> References: <12C7AB425F0DD546B6049311F827C74E08A37476C2@VA3DIAXVS141.RED001.local> <20100731230038.EE9153A40D9@sparrow.telecommunity.com> Message-ID: <12C7AB425F0DD546B6049311F827C74E08A585959D@VA3DIAXVS141.RED001.local> Thanks for the tip, pje. Your suggestion was very helpful. Perhaps I can ask a follow-up. In addition to calculating the tag at package-time (when setup.py is run for sdist/bdist*), I am calculating the version at package-time. The problem I have with this approach is with source distributions (sdists). Since the sdists get the same setup.py file, the calculations for version and tag are done both at packaging time and at install time. I would like to configure my setup.py such that when sdist is run, the version and tag are calculated and added to the egg_info, but when a consumer of that sdist goes to install it, the version and tag are not calculated and just consumed from the egg_info (similar to what happens when installing a bdist_egg). This leads to two questions: 1. Is there a way to determine that I'm installing from an sdist, so I can bypass any package-time calculations and defer to the already-calculated PKG_INFO present in the sdist zip? I can probably elicit that information from the environment, but if there's an indicator in setuptools, that would be helpful. 2. Once the setup script has determined that it is installing from an sdist, how can it call setup() to actually defer to the existing PKG_INFO for installation? I suspect there may be a better approach; any suggestions are appreciated. Using bdist_egg does exactly what I need, but I'd prefer to use sdist to avoid the need to distribute versions for each Python version. I can almost imagine an sdist_egg command that creates a source-only distribution suitable for easy_install. If it helps to have a concrete example, the project I'm working on is [hgtools][1] and here is the [current state of the setup.py script][2]. Thanks! Jason [1] http://bitbucket.org/jaraco/hgtools [2] http://bitbucket.org/jaraco/hgtools/src/6e18ec0cea4a/setup.py > -----Original Message----- > From: P.J. Eby [mailto:pje at telecommunity.com] > Sent: Saturday, 31 July, 2010 19:01 > To: Jason R. Coombs; distutils-sig at python.org > Subject: Re: [Distutils] How to customize egg_info behavior > > At 12:39 PM 7/31/2010 -0700, Jason R. Coombs wrote: > >I'm using setuptools 0.6c11. > > > >I'd like to programmatically customize the way the egg_info command is > >run. That is, in my setup.py, I would like to run some functions to > >determine the tag-build, tag-date, and tag-svn-revision parameters to > >egg_info. Are there parameters > > > >How is this task best accomplished? Do I subclass the egg_info Command? > >If so, do I just specify cmdclass={'egg_info':my_custom_egg_info} ? Or > >would you recommend another approach? > > You can just do this in your setup script: > > setup( > ... > options = dict(egg_info = dict(tag_build = "whatever", ...)), > ... > ) > > Replacing "whatever", of course, with your programmatic calculation. > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6448 bytes Desc: not available URL: From schugschug at gmail.com Sun Aug 1 16:27:47 2010 From: schugschug at gmail.com (Eric Schug) Date: Sun, 01 Aug 2010 10:27:47 -0400 Subject: [Distutils] easy_install install file permissions Message-ID: <4C558463.4020409@gmail.com> Hello, I was having problem getting installed egg files to work until I realized that they had the wrong permissions set. It seems that on my machine easy_install is installing files with permissions set to -rw-r-----. How do I change this so that my files are installed with og+r, automatically so that I don't have to go back and fix them? Thanks Eric Schug Email: schugschug at gmail.com http://trilogysci.webfactional.com From pje at telecommunity.com Sun Aug 1 19:07:25 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 01 Aug 2010 13:07:25 -0400 Subject: [Distutils] easy_install install file permissions In-Reply-To: <4C558463.4020409@gmail.com> References: <4C558463.4020409@gmail.com> Message-ID: <20100801170721.EDA093A4093@sparrow.telecommunity.com> At 10:27 AM 8/1/2010 -0400, Eric Schug wrote: >Hello, > >I was having problem getting installed egg files to work until I >realized that they had the wrong permissions set. It seems that on >my machine easy_install is installing files with permissions set to >-rw-r-----. How do I change this so that my files are installed >with og+r, automatically so that I don't have to go back and fix them? That sounds like a problem with your umask. Check your platform's docs for the umask command to find out how to see what it's set to and to change it. From pje at telecommunity.com Sun Aug 1 21:09:28 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 01 Aug 2010 15:09:28 -0400 Subject: [Distutils] How to customize egg_info behavior In-Reply-To: <12C7AB425F0DD546B6049311F827C74E08A585959D@VA3DIAXVS141.RE D001.local> References: <12C7AB425F0DD546B6049311F827C74E08A37476C2@VA3DIAXVS141.RED001.local> <20100731230038.EE9153A40D9@sparrow.telecommunity.com> <12C7AB425F0DD546B6049311F827C74E08A585959D@VA3DIAXVS141.RED001.local> Message-ID: <20100801190923.BC9F13A4758@sparrow.telecommunity.com> At 10:36 PM 7/31/2010 -0700, Jason R. Coombs wrote: >Thanks for the tip, pje. Your suggestion was very helpful. Perhaps I can ask >a follow-up. > >In addition to calculating the tag at package-time (when setup.py is run for >sdist/bdist*), I am calculating the version at package-time. > >The problem I have with this approach is with source distributions (sdists). >Since the sdists get the same setup.py file, the calculations for version >and tag are done both at packaging time and at install time. I would like to >configure my setup.py such that when sdist is run, the version and tag are >calculated and added to the egg_info, but when a consumer of that sdist goes >to install it, the version and tag are not calculated and just consumed from >the egg_info (similar to what happens when installing a bdist_egg). When setuptools builds an sdist, it includes an updated setup.cfg containing all the original egg_info settings, so that part is done for you. What this means is that you can programmatically tell if you're running from an sdist or a checkout by looking for the tag_build setting in setup.cfg. That means that you could potentially just specify version='' in your setup(), and put the *entire* version number in the tag_build option to egg_info. In order for that to work correctly, what you'd need to do is: 1. Try to read setup.cfg using ConfigParser, and read out the 'tag_build' from the 'egg_info' section. 2. If it's found, set the version string to that, and force tag_build to an *empty string* in the options['egg_info'] passed to setup(), so that tag_build doesn't get stuck back in again (doubling-up the version). 3. If it's not found, do a straight-up calculation. The main downside to this approach, however, is that you have to put all this code in setup.py, and you can't import it from anywhere, unless you bundle the code with every project you're using it on. So here's what you do instead. Implement this as a setup() argument, e.g.: def hg_version_calc(dist, attr, value): if value: from ConfigParser import ConfigParser parser = ConfigParser() parser.read('setup.cfg') if parser.has_section('egg_info') and 'tag_build' in parser.options('egg_info'): version = parser.options('egg_info')['tag_build'] else: version = calculate_version() dist.metadata.version = version And register this as a 'distutils.setup_keywords' entry point in your plugin project. You then need only include a 'use_hg_version = True' in your setup() call, along with a setup_requires to ensure the plugin is available. Now, there is one minor hitch with this; as shown above, this code will double-up version strings in certain circumstances; specifically, if an 'egg_info' is done from an sdist -- which is going to happen if you build pretty much anything from an sdist, be it an sdist, egg, RPM, or anything else. There isn't any easy way to work around this at the moment (although I will implement a formal fix in later releases of setuptools). In the meantime, you can monkeypatch thus, in the same plugin where you implement the above function: from setuptools.command.egg_info import egg_info old_tagged_version = egg_info.tagged_version def tagged_version(self): if self.distribution.use_hg_version: return safe_version(self.distribution.get_version()) else: return old_tagged_version(self) egg_info.tagged_version = tagged_version Whew. This was a tough one to figure out, and it's a little hacky. Probably in 0.7 I'll just factor this into a separate entry point group for version calculations, so that it's not as much trouble to do something like this. From reinout at vanrees.org Mon Aug 2 11:21:25 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 02 Aug 2010 11:21:25 +0200 Subject: [Distutils] buildout 2 In-Reply-To: References: Message-ID: On 07/30/2010 03:37 PM, Jim Fulton wrote: > > After 1.5 lands and the dust settles, I intend to look at refactoring > buildout pretty significantly, primarily to make maintenance simpler. a) Sounds good. b) What about existing branches that are waiting for merging? I've got one that's about half a year old. (The one with support for distutils scripts in addition to the console_scripts entry points). Small change, includes tests. Are branches like that supposed to be merged beforehand? Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From m.van.rees at zestsoftware.nl Tue Aug 3 10:38:37 2010 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Tue, 03 Aug 2010 10:38:37 +0200 Subject: [Distutils] Announcing buildout-versions 1.1 In-Reply-To: <4C52F99D.1070701@simplistix.co.uk> References: <4C52F1D6.8070506@simplistix.co.uk> <4C52F99D.1070701@simplistix.co.uk> Message-ID: Op 30-07-10 18:11, Chris Withers schreef: > Chris Withers wrote: >> Hi All, >> >> I'm pleased to announce the first release of buildout_versions. > > After having some trouble with a package with an underscore in it's > name, I've renamed it to buildout-versions and done a 1.1 release :-( > > The PyPI page is now here: > http://pypi.python.org/pypi/buildout-versions > > The documentation is now here: > http://packages.python.org/buildout-versions I don't see a good enough reason for me to use this instead of the buildout.dumppickedversions extension that I am used to. But it does work. Small gripe: it does not report that the buildout-versions package itself is not pinned. Note that buildout.dumppickedversions has the same problem. -- Maurits van Rees Programmer, Zest Software From chris at simplistix.co.uk Tue Aug 3 11:18:52 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 03 Aug 2010 10:18:52 +0100 Subject: [Distutils] Announcing buildout-versions 1.1 In-Reply-To: References: <4C52F1D6.8070506@simplistix.co.uk> <4C52F99D.1070701@simplistix.co.uk> Message-ID: <4C57DEFC.4060104@simplistix.co.uk> Maurits van Rees wrote: > I don't see a good enough reason for me to use this instead of the > buildout.dumppickedversions extension that I am used to. But it does work. That's fine, I just didn't like the starry vomit, even when all packages are pinned, and I didn't like my versions.cfg being stomped all the time (mine often have extends sections, they need appending) > Small gripe: it does not report that the buildout-versions package > itself is not pinned. Note that buildout.dumppickedversions has the > same problem. Yes, I suspect a bit chicken and egg (pardon the pun!). Both extensions monkey patch the loading mechanism, but obviously, by the time the patching code has been run, the package for the extension has already been loaded :-S Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From jaraco at jaraco.com Wed Aug 4 06:51:53 2010 From: jaraco at jaraco.com (Jason R. Coombs) Date: Tue, 3 Aug 2010 21:51:53 -0700 Subject: [Distutils] How to customize egg_info behavior In-Reply-To: <20100801190923.BC9F13A4758@sparrow.telecommunity.com> References: <12C7AB425F0DD546B6049311F827C74E08A37476C2@VA3DIAXVS141.RED001.local> <20100731230038.EE9153A40D9@sparrow.telecommunity.com> <12C7AB425F0DD546B6049311F827C74E08A585959D@VA3DIAXVS141.RED001.local> <20100801190923.BC9F13A4758@sparrow.telecommunity.com> Message-ID: <12C7AB425F0DD546B6049311F827C74E08A5E4538C@VA3DIAXVS141.RED001.local> > -----Original Message----- > From: P.J. Eby [mailto:pje at telecommunity.com] > ... > There isn't any easy way to work around this at the moment (although I will > implement a formal fix in later releases of setuptools). In the meantime, > you can monkeypatch thus, in the same plugin where you implement the > above function: > > from setuptools.command.egg_info import egg_info > old_tagged_version = egg_info.tagged_version > > def tagged_version(self): > if self.distribution.use_hg_version: > return safe_version(self.distribution.get_version()) > else: > return old_tagged_version(self) > > egg_info.tagged_version = tagged_version > > Whew. This was a tough one to figure out, and it's a little hacky. Probably in > 0.7 I'll just factor this into a separate entry point group for version > calculations, so that it's not as much trouble to do something like this. > PJE: Thanks again. That's a huge help. I don't think I would have ever figured out all of those hooks by myself. One problem I found was that even with that code, the tag_build wasn't getting set in setup.cfg, so I added another line to set the egg_info.tag_build in the monkey-patched egg_info.tagged_version. After that, it seems to work exactly as prescribed. It's in hgtools 0.4, so feel free to give it a go. As you prescribed, set use_hg_version=True and setup_requires hgtools>=0.4 and you'll get versions automatically calculated based on your tags. Regards, Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6448 bytes Desc: not available URL: From sridharr at activestate.com Thu Aug 5 20:25:33 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Thu, 05 Aug 2010 11:25:33 -0700 Subject: [Distutils] setuptools: 'console_scripts' does not respect 'extras' requested by easy_install? Message-ID: <4C5B021D.5060503@activestate.com> From setup.py: [...] name='hello' [...] entry_points = { 'console_scripts': [ 'hello=hello:main', 'internal=hello.internal:main [secret]', [...] If I run "easy_install hello" both bin/hello and bin/internal are installed. I expected bin/internal to be installed *only* when "easy_install hello[secret]" is run. Is this anomaly a bug, or by design? I have a real-world use case for this, which is to install 'backend' scripts only when the 'backend' extras is requested. This is to prevent users from installing backend/server related commands when they only need the client commands. When creating sdist, I explicitly exclude src/hello/internal (internal packages), but there is no way to prevent the script creation other than hacking setup.py. -srid From pje at telecommunity.com Thu Aug 5 20:51:46 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 05 Aug 2010 14:51:46 -0400 Subject: [Distutils] setuptools: 'console_scripts' does not respect 'extras' requested by easy_install? In-Reply-To: <4C5B021D.5060503@activestate.com> References: <4C5B021D.5060503@activestate.com> Message-ID: <20100805185136.69A4E3A4156@sparrow.telecommunity.com> At 11:25 AM 8/5/2010 -0700, Sridhar Ratnakumar wrote: > From setup.py: > > [...] > name='hello' > [...] > entry_points = { > 'console_scripts': [ > 'hello=hello:main', > 'internal=hello.internal:main [secret]', > [...] > >If I run "easy_install hello" both bin/hello and bin/internal are >installed. I expected bin/internal to be installed *only* when >"easy_install hello[secret]" is run. > >Is this anomaly a bug, or by design? I don't really consider it a bug; more of the absence of a feature. ;-) I will consider adding it to 0.7. (Though it will more likely be that I will simply issue a warning if the extra(s) aren't specified and the requirements aren't already met; in part it depends on whether I refactor to add a more transactional installation system.) >I have a real-world use case for this, which is to install 'backend' >scripts only when the 'backend' extras is requested. This is to >prevent users from installing backend/server related commands when >they only need the client commands. When creating sdist, I >explicitly exclude src/hello/internal (internal packages), but there >is no way to prevent the script creation other than hacking setup.py. As a way to implement this in the short run, you could place the backend scripts in a separate distribution, and list that in the extras. The other distribution could consist of nothing more than the list of entry points, and a requirement that points back to the original distribution. From chris at simplistix.co.uk Fri Aug 6 21:10:37 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 06 Aug 2010 20:10:37 +0100 Subject: [Distutils] GCC versions and binary egg names In-Reply-To: <20100731231954.GA10963@fridge.pov.lt> References: <4C4DC004.8060705@simplistix.co.uk> <4C545169.8020901@simplistix.co.uk> <20100731231954.GA10963@fridge.pov.lt> Message-ID: <4C5C5E2D.5010604@simplistix.co.uk> Marius Gedminas wrote: > In my experience the difficulty is figuring out which non-Python > dependencies you need installed (libraries, header files, compilers, > build tools). Cheating (of the 'sudo apt-get build-dep python-numpy' > kind) helps, if what you're trying to install is a (newer version of a) > package that's already been packaged for your distribution. Yes, that's one. The other is to be able to use buildout or pip + a requirements file to get the same set of python packages available, no matter what OS you choose to build the project on... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Aug 10 10:00:31 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 10 Aug 2010 09:00:31 +0100 Subject: [Distutils] Announcing buildout_versions 1.2 Message-ID: <4C61071F.9060408@simplistix.co.uk> Hi All, I'm pleased to announce a new release of buildout_versions. The new version re-shuffles the order in which un-pinned versions are reported, making it easier to spot where package dependencies come from. The PyPI page is here: http://pypi.python.org/pypi/buildout_versions The documentation is here: http://packages.python.org/buildout_versions/use.html If you have any problems or suggestions, please let me know on this list! cheers, Chris From chris at simplistix.co.uk Tue Aug 10 11:27:55 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 10 Aug 2010 10:27:55 +0100 Subject: [Distutils] Announcing buildout-versions 1.2 In-Reply-To: <4C61071F.9060408@simplistix.co.uk> References: <4C61071F.9060408@simplistix.co.uk> Message-ID: <4C611B9B.5090306@simplistix.co.uk> Urg, copy'n'paste fail... Chris Withers wrote: > Hi All, > > I'm pleased to announce a new release of buildout_versions. > > The new version re-shuffles the order in which un-pinned versions are > reported, making it easier to spot where package dependencies come from. > > The PyPI page is here: > http://pypi.python.org/pypi/buildout_versions > > The documentation is here: > http://packages.python.org/buildout_versions/use.html Should be: The PyPI page is here: http://pypi.python.org/pypi/buildout-versions The documentation is here: http://packages.python.org/buildout-versions/use.html Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From udono at gmx.net Tue Aug 10 16:27:50 2010 From: udono at gmx.net (Udo Spallek) Date: Tue, 10 Aug 2010 16:27:50 +0200 Subject: [Distutils] zc.buildout: Declare options in a recipe and use it in another recipe Message-ID: <1281450470.13512.768.camel@lisa.localdomain> Hello, like to write a custom recipe. The recipe should be able to introduce a new variable (or option) which can be used in another part of buildout.cfg. I try to catch this with the buildout.cfg and recipe attached below. The recipe is found, installed and called with part1 right before part2. So far so good. My problem is in part2. New_option is always empty and not filled with the result (testresult) in recipe test-new-option. How can I declare and transport options from one recipe script to another? Regards Udo Sample buildout.cfg [buildout] parts = part1 part2 [part1] recipe = recipes:test-new-option [part2] recipe = collective.recipe.template == 1.8 new_option = ${part1:new_option} ... ---- Recipe: test-new-option: class TestNewOption(object): """zc.buildout recipe""" def __init__(self, buildout, name, options): self.buildout, self.name, self.options = buildout, name, options def install(self): """Installer""" # Does not work: self.options['new_option'] = 'testresult' return tuple() def update(self): """Updater""" pass From fdrake at acm.org Tue Aug 10 16:41:07 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 10 Aug 2010 10:41:07 -0400 Subject: [Distutils] zc.buildout: Declare options in a recipe and use it in another recipe In-Reply-To: <1281450470.13512.768.camel@lisa.localdomain> References: <1281450470.13512.768.camel@lisa.localdomain> Message-ID: On Tue, Aug 10, 2010 at 10:27 AM, Udo Spallek wrote: > I try to catch this with the buildout.cfg and recipe attached below. The > recipe is found, installed and called with part1 right before part2. So > far so good. Options initialized by your recipe's constructor will be available to other recipes. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From jim at zope.com Tue Aug 10 16:41:40 2010 From: jim at zope.com (Jim Fulton) Date: Tue, 10 Aug 2010 10:41:40 -0400 Subject: [Distutils] zc.buildout: Declare options in a recipe and use it in another recipe In-Reply-To: <1281450470.13512.768.camel@lisa.localdomain> References: <1281450470.13512.768.camel@lisa.localdomain> Message-ID: On Tue, Aug 10, 2010 at 10:27 AM, Udo Spallek wrote: > Hello, > > like to write a custom recipe. The recipe should be able to introduce a > new variable (or option) which can be used in another part of > buildout.cfg. > I try to catch this with the buildout.cfg and recipe attached below. The > recipe is found, installed and called with part1 right before part2. So > far so good. > > My problem is in part2. ?New_option is always empty and not filled with > the result (testresult) in recipe test-new-option. > > How can I declare and transport options from one recipe script to > another? You must do this in __init__, not in install. Options are resolved before installation methods are called, so for a recipe to create options to be consumed by other recipes, it must do so in the __init__ method. Jim -- Jim Fulton From udono at gmx.net Tue Aug 10 17:37:09 2010 From: udono at gmx.net (udono) Date: Tue, 10 Aug 2010 17:37:09 +0200 Subject: [Distutils] zc.buildout: Declare options in a recipe and use it in another recipe In-Reply-To: References: <1281450470.13512.768.camel@lisa.localdomain> Message-ID: <1281454629.13512.771.camel@lisa.localdomain> Hello Jim, On Tue, Aug 10, 2010 at 10:41 AM, -0400 Jim Fulton wrote: > On Tue, Aug 10, 2010 at 10:27 AM, Udo Spallek wrote: > > How can I declare and transport options from one recipe script to > > another? [...] > You must do this in __init__, not in install. Options are > resolved before installation methods are called, so for > a recipe to create options to be consumed by other > recipes, it must do so in the __init__ method. Yes, thats it. Thanks a lot! Cheers Udo From friedrichromstedt at gmail.com Tue Aug 10 23:08:42 2010 From: friedrichromstedt at gmail.com (Friedrich Romstedt) Date: Tue, 10 Aug 2010 23:08:42 +0200 Subject: [Distutils] Announcing buildout_versions 1.2 In-Reply-To: <4C61071F.9060408@simplistix.co.uk> References: <4C61071F.9060408@simplistix.co.uk> Message-ID: 2010/8/10 Chris Withers : > The PyPI page is here: > http://pypi.python.org/pypi/buildout_versions No, it's here ;-): http://pypi.python.org/pypi/buildout-versions/ Friedrich From marrakis at gmail.com Wed Aug 11 14:57:17 2010 From: marrakis at gmail.com (Mathieu Leduc-Hamel) Date: Wed, 11 Aug 2010 14:57:17 +0200 Subject: [Distutils] Optional requirements with "extra" Message-ID: Hi all, I was reading the documentation of distutils, setuptools and distribution cause I was searching for a way to add some optional dependancies to a project. In fact i would like to add an optional dependency and it will change some components of the a module. I'll explain a little bit more: I have a module called "storelocator", when you have use it without any optional dependency. In the setup script I have the following option: extras_require = { "satchmo" : ["Satchmo"], } If you install the package with the satchmo support like that: storelocator[satchmo] Then I would like be able with pkg_resources or something like to check in the code if the satchmo support has been enabled or not, if yes, i'll change some part of my package or not. Mathieu Leduc-Hamel -------------- next part -------------- An HTML attachment was scrubbed... URL: From potterpavi at gmail.com Wed Aug 11 16:18:18 2010 From: potterpavi at gmail.com (pavi ena) Date: Wed, 11 Aug 2010 19:48:18 +0530 Subject: [Distutils] Issue with Setuptools package name Message-ID: Hello, While working with setuptools [Automatic Script Creation with setuptools] module to build python package naming with "build" as prefix for package name, i found wired issue with setup.py. My setup.py code looks like below. from setuptools import setup, find_packages setup( name=*'build_tools'*, version='0.1.0', packages = find_packages(), entry_points = { 'console_scripts': [ 'test_build = test_build:main', ], }, ) F:\mywork\*build_tools*>Scripts\python.exe setup.py install -vvv [this is how i executed the setup.py from virtual python] After running the above installer, as per my understanding after the successful installation of this package the console entry_points should go in to the "Scripts" directory. The above said scenario is working fine with Linux machine when i tried the same usecase in windows machine, it is not working as expected. Entry point console script is not listing/showing under "Scripts" directory. Please any one suggest fix or explain me the problem. Thanks, Praveen Doddamani. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Wed Aug 11 16:42:44 2010 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Aug 2010 10:42:44 -0400 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: References: Message-ID: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> At 07:48 PM 8/11/2010 +0530, pavi ena wrote: >The above said scenario is working fine with Linux machine when i >tried the same usecase in windows machine, it is not working as expected. >Entry point console script is not listing/showing under "Scripts" directory. Where is it showing up? Note that on Linux, the standard installation pattern is such that scripts are installed in prefix/bin -- i.e., alongside the python executable. On Windows, however, the default installation location for scripts is in a subdirectory called "Scripts" alongside the python executable. Since you did a verbose install, the actual location of the script should be shown in the output from the command. From pje at telecommunity.com Wed Aug 11 16:46:08 2010 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Aug 2010 10:46:08 -0400 Subject: [Distutils] Optional requirements with "extra" In-Reply-To: References: Message-ID: <20100811144614.7FF123A4108@sparrow.telecommunity.com> At 02:57 PM 8/11/2010 +0200, Mathieu Leduc-Hamel wrote: >If you install the package with the satchmo support like that: > >storelocator[satchmo] > >Then I would like be able with pkg_resources or something like to >check in the code if the satchmo support has been enabled or not, if >yes, i'll change some part of my package or not. try: pkg_resources.require('storelocator[satchmo]') except pkg_resources.ResolutionError: pass # failure case else: pass # success case From shimizukawa at gmail.com Thu Aug 12 07:46:34 2010 From: shimizukawa at gmail.com (Takayuki Shimizukawa) Date: Thu, 12 Aug 2010 14:46:34 +0900 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: References: Message-ID: Hi Praveen, Your problem may be connected with below issue: http://bugs.python.org/issue6884 http://mail.python.org/pipermail/distutils-sig/2009-September/013194.html Regards, -- Takayuki SHIMIZUKAWA 2010/8/11 pavi ena : > Hello, > > While working with setuptools [Automatic Script Creation with setuptools] > module to build python package naming with "build" as prefix for package > name, i found wired issue with setup.py. My setup.py code looks like below. > > from setuptools import setup, find_packages > > setup( > ??? name='build_tools', > ??? version='0.1.0', > ??? packages = find_packages(), > ??? entry_points = { > ????????? 'console_scripts': [ > ????????????? 'test_build = test_build:main', > ?????????????? ], > ??????? }, > ) > > F:\mywork\build_tools>Scripts\python.exe setup.py install -vvv? [this is how > i executed the setup.py from virtual python] > > After running the above installer, as per my understanding after the > successful installation of this package the console entry_points should go > in to the "Scripts" directory. > The above said scenario is working fine with Linux machine when i tried the > same usecase in windows machine, it is not working as expected. > Entry point console script is not listing/showing under "Scripts" directory. > > Please any one suggest fix or explain me the problem. > > Thanks, > Praveen Doddamani. > > > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From potterpavi at gmail.com Thu Aug 12 08:06:55 2010 From: potterpavi at gmail.com (pavi ena) Date: Thu, 12 Aug 2010 11:36:55 +0530 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> References: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> Message-ID: Hello sir, Please find the attached batch file out put, which i ran for the installation. The below commands are my windows batch file content doskey clear=cls ls=dir virtualenv --no-site-packages --clear . type setup.py Scripts\python.exe setup.py install -vvv cd Scripts ls -l As per my setup.py after the installation console entry_points should list/install in prefix\scripts, where as it is not installing the entry points in a subdirectory called "Scripts" alongside the python executable. But the same scenario is working fine in Linux platform all the entry_points are listing\installing in prefix/bin subdirectory. Thanks, Praveen On Wed, Aug 11, 2010 at 8:12 PM, P.J. Eby wrote: > At 07:48 PM 8/11/2010 +0530, pavi ena wrote: > >> The above said scenario is working fine with Linux machine when i tried >> the same usecase in windows machine, it is not working as expected. >> Entry point console script is not listing/showing under "Scripts" >> directory. >> > > Where is it showing up? Note that on Linux, the standard installation > pattern is such that scripts are installed in prefix/bin -- i.e., alongside > the python executable. On Windows, however, the default installation > location for scripts is in a subdirectory called "Scripts" alongside the > python executable. > > Since you did a verbose install, the actual location of the script should > be shown in the output from the command. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- E:\praveen\temp_build\build_tools>doskey clear=cls ls=dir E:\praveen\temp_build\build_tools>virtualenv --no-site-packages --clear . Not deleting .\Scripts New python executable in .\Scripts\python.exe Installing setuptools................done. E:\praveen\temp_build\build_tools>type setup.py from setuptools import setup, find_packages setup( name='build_tool', version='0.1.0', install_requires=['pip==0.7.1', 'virtualenv==1.4.9', 'rdopts==0.1.3', 'scripttest', 'nose', ], tests_require=['nose'], packages = find_packages(), entry_points = { 'console_scripts': [ 'pybuild = build_tools.build:main', 'pyfreeze = build_tools.freeze:main', 'pydocs = build_tools.docs:main', ], }, ) E:\praveen\temp_build\build_tools>Scripts\python.exe setup.py install -vvv running install running bdist_egg running egg_info creating build_tool.egg-info writing requirements to build_tool.egg-info\requires.txt writing build_tool.egg-info\PKG-INFO writing top-level names to build_tool.egg-info\top_level.txt writing dependency_links to build_tool.egg-info\dependency_links.txt writing entry points to build_tool.egg-info\entry_points.txt writing manifest file 'build_tool.egg-info\SOURCES.txt' reading manifest file 'build_tool.egg-info\SOURCES.txt' writing manifest file 'build_tool.egg-info\SOURCES.txt' installing library code to build\bdist.win32\egg running install_lib running build_py creating build creating build\lib creating build\lib\build_tools copying build_tools\build.py -> build\lib\build_tools copying build_tools\docs.py -> build\lib\build_tools copying build_tools\freeze.py -> build\lib\build_tools copying build_tools\__init__.py -> build\lib\build_tools creating build\bdist.win32 creating build\bdist.win32\egg creating build\bdist.win32\egg\build_tools copying build\lib\build_tools\build.py -> build\bdist.win32\egg\build_tools copying build\lib\build_tools\docs.py -> build\bdist.win32\egg\build_tools copying build\lib\build_tools\freeze.py -> build\bdist.win32\egg\build_tools copying build\lib\build_tools\__init__.py -> build\bdist.win32\egg\build_tools byte-compiling build\bdist.win32\egg\build_tools\build.py to build.pyc byte-compiling build\bdist.win32\egg\build_tools\docs.py to docs.pyc byte-compiling build\bdist.win32\egg\build_tools\freeze.py to freeze.pyc byte-compiling build\bdist.win32\egg\build_tools\__init__.py to __init__.pyc creating build\bdist.win32\egg\EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist\build_tool-0.1.0.dev_12-py2.5.egg' and adding 'build\bdist.win32\egg' to it removing 'build\bdist.win32\egg' (and everything under it) Processing build_tool-0.1.0.dev_12-py2.5.egg Copying build_tool-0.1.0.dev_12-py2.5.egg to e:\praveen\temp_build\build_tools\lib\site-packages Adding build-tool 0.1.0.dev-12 to easy-install.pth file Installed e:\praveen\temp_build\build_tools\lib\site-packages\build_tool-0.1.0.dev_12-py2.5.egg Processing dependencies for build-tool==0.1.0.dev-12 Finished processing dependencies for build-tool==0.1.0.dev-12 E:\praveen\temp_build\build_tools>cd Scripts E:\praveen\temp_build\build_tools\Scripts>ls -l total 39 -rwxr-xr-x 1 pdoddama Administ 621 Aug 12 16:29 activate.bat -rw-r--r-- 1 pdoddama Administ 1003 Aug 12 16:29 activate_this.py -rwxr-xr-x 1 pdoddama Administ 317 Aug 12 16:29 deactivate.bat -rwxr-xr-x 1 pdoddama Administ 340 Aug 12 16:29 easy_install-2.5-script.py -rwxr-xr-x 1 pdoddama Administ 7168 Aug 12 16:29 easy_install-2.5.exe -rw-r--r-- 1 pdoddama Administ 531 Aug 12 16:29 easy_install-2.5.exe.manifest -rwxr-xr-x 1 pdoddama Administ 332 Aug 12 16:29 easy_install-script.py -rwxr-xr-x 1 pdoddama Administ 7168 Aug 12 16:29 easy_install.exe -rw-r--r-- 1 pdoddama Administ 527 Aug 12 16:29 easy_install.exe.manifest -rwxr-xr-x 1 pdoddama Administ 290 Aug 12 16:29 pip-script.py -rwxr-xr-x 1 pdoddama Administ 7168 Aug 12 16:29 pip.exe -rw-r--r-- 1 pdoddama Administ 518 Aug 12 16:29 pip.exe.manifest -rwxr-xr-x 1 pdoddama Administ 24064 Aug 12 16:29 python.exe -rwxr-xr-x 1 pdoddama Administ 24576 Aug 12 16:29 pythonw.exe From chris at simplistix.co.uk Thu Aug 12 17:01:48 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 12 Aug 2010 16:01:48 +0100 Subject: [Distutils] [buildout] bug related to package name capitalisation Message-ID: <4C640CDC.1050609@simplistix.co.uk> Hi All, The following buildout.cfg demonstrates something that feels buggy to me: [buildout] parts = sqlalchemy migrate versions = versions [versions] SQLAlchemy = 0.6.0 sqlalchemy = 0.6.2 [sqlalchemy] recipe = zc.recipe.egg eggs = SQLAlchemy entry-points = foo=bar:baz [migrate] recipe = zc.recipe.egg eggs = sqlalchemy-migrate When run, you'll get two scripts: bin/migrate: ... sys.path[0:0] = [ '/opt/buildout-eggs/sqlalchemy_migrate-0.6-py2.6.egg', '/opt/buildout-eggs/Tempita-0.4-py2.6.egg', '/opt/buildout-eggs/decorator-3.2.0-py2.6.egg', '/opt/buildout-eggs/SQLAlchemy-0.6.2-py2.6.egg', ... bin/foo: ... sys.path[0:0] = [ '/opt/buildout-eggs/SQLAlchemy-0.6.0-py2.6.egg', ] ... The problem is that sqlalchemy-migrate declares an install_requires dependency on "sqlalchemy", which the package is actually called SQLAlchemy. It could be argued that sqlalchemy-migrate is "wrong" but it remains that all python's packaging tools appear case insensitive to package names. As such, I'd expect buildout's version pinning to be case agnostic as well. What do other people feel about this? Chris From fdrake at acm.org Thu Aug 12 19:22:30 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 12 Aug 2010 13:22:30 -0400 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: <4C640CDC.1050609@simplistix.co.uk> References: <4C640CDC.1050609@simplistix.co.uk> Message-ID: On Thu, Aug 12, 2010 at 11:01 AM, Chris Withers wrote: > As such, I'd expect buildout's version pinning to be case agnostic as well. > What do other people feel about this? A lot of case-insensitivity is present for dealing with case-insensitive file systems; when looking at a distribution, the case of the name often doesn't matter. For importable names, case *does* matter; Python bothers to verify case on case-preserving file systems. I've seen very few cases where case-insensitivity in package names is tolerated (once, I think). There's no reason to spread the case-senseless disease any further. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From benji at benjiyork.com Thu Aug 12 19:55:51 2010 From: benji at benjiyork.com (Benji York) Date: Thu, 12 Aug 2010 13:55:51 -0400 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: References: <4C640CDC.1050609@simplistix.co.uk> Message-ID: On Thu, Aug 12, 2010 at 1:22 PM, Fred Drake wrote: > On Thu, Aug 12, 2010 at 11:01 AM, Chris Withers wrote: >> As such, I'd expect buildout's version pinning to be case agnostic as well. >> What do other people feel about this? > I've seen very few cases where case-insensitivity in package names is > tolerated (once, I think). ?There's no reason to spread the > case-senseless disease any further. +1 Case-insensitivity usually hurts more than it helps. -- Benji York From chris at simplistix.co.uk Thu Aug 12 23:59:53 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 12 Aug 2010 22:59:53 +0100 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: References: <4C640CDC.1050609@simplistix.co.uk> Message-ID: <4C646ED9.4060007@simplistix.co.uk> Benji York wrote: > On Thu, Aug 12, 2010 at 1:22 PM, Fred Drake wrote: >> On Thu, Aug 12, 2010 at 11:01 AM, Chris Withers wrote: >>> As such, I'd expect buildout's version pinning to be case agnostic as well. >>> What do other people feel about this? > >> I've seen very few cases where case-insensitivity in package names is >> tolerated (once, I think). There's no reason to spread the >> case-senseless disease any further. > > +1 Case-insensitivity usually hurts more than it helps. Indeed, but sadly, setuptools is case insensitive as far as distributions go, which means that buildout ends up doing bizarre things when different casings of the same package name are used, particularly when it comes to pinning versions. Yes, the "correct" thing to do would be for buildout to kick out requirements where the case of the distribution name specified didn't match that returned by the distribution that setuptools decided would satisfy that requirement. However, that would mean you'd suddenly have distributions that would work with easy_install and pip but not with buildout. I guess my point is that, by way of being based on setuptools, buildout already *is* case insensitive w.r.t. distribution names and the fact that *only* version pinning *is* case sensitive ends up being a bug... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From benji at benjiyork.com Fri Aug 13 00:05:17 2010 From: benji at benjiyork.com (Benji York) Date: Thu, 12 Aug 2010 18:05:17 -0400 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: <4C646ED9.4060007@simplistix.co.uk> References: <4C640CDC.1050609@simplistix.co.uk> <4C646ED9.4060007@simplistix.co.uk> Message-ID: On Thu, Aug 12, 2010 at 5:59 PM, Chris Withers wrote: > I guess my point is that, by way of being based on setuptools, buildout > already *is* case insensitive w.r.t. distribution names and the fact that > *only* version pinning *is* case sensitive ends up being a bug... I guess that might be construed as a bug. Maybe. This makes me even happier that I always use allow-picked-versions = false. It means that my versions section will always have the correctly capitalized names. -- Benji York From chris at simplistix.co.uk Fri Aug 13 00:10:06 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 12 Aug 2010 23:10:06 +0100 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: References: <4C640CDC.1050609@simplistix.co.uk> <4C646ED9.4060007@simplistix.co.uk> Message-ID: <4C64713E.3060003@simplistix.co.uk> Benji York wrote: > On Thu, Aug 12, 2010 at 5:59 PM, Chris Withers wrote: >> I guess my point is that, by way of being based on setuptools, buildout >> already *is* case insensitive w.r.t. distribution names and the fact that >> *only* version pinning *is* case sensitive ends up being a bug... > > I guess that might be construed as a bug. Maybe. > > This makes me even happier that I always use allow-picked-versions = > false. It means that my versions section will always have the correctly > capitalized names. Yes, which is exactly how I bumped into this ;-) See the example I gave; you end up having to specify the same package with the same version but with all the capitalisation permutations in order for the version pin to stick and allow-picked-versions* not to kick the distribution out... cheers, Chris *I use buildout-versions so as not to have to deal with the cycle of pin, unpinned version kicked out, next unpinned version kicked out, etc... buildout-versions tells me all the problems up front and even keeps the versions.cfg file up to date for me ;-) -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From fdrake at acm.org Fri Aug 13 01:29:33 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 12 Aug 2010 19:29:33 -0400 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: <4C646ED9.4060007@simplistix.co.uk> References: <4C640CDC.1050609@simplistix.co.uk> <4C646ED9.4060007@simplistix.co.uk> Message-ID: On Thu, Aug 12, 2010 at 5:59 PM, Chris Withers wrote: > I guess my point is that, by way of being based on setuptools, buildout > already *is* case insensitive w.r.t. distribution names and the fact that > *only* version pinning *is* case sensitive ends up being a bug... The fix for probably include abandoning setuptools and case-senseless file systems. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From chris at simplistix.co.uk Fri Aug 13 02:02:40 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 13 Aug 2010 01:02:40 +0100 Subject: [Distutils] [buildout] bug related to package name capitalisation In-Reply-To: References: <4C640CDC.1050609@simplistix.co.uk> <4C646ED9.4060007@simplistix.co.uk> Message-ID: <4C648BA0.5060806@simplistix.co.uk> Fred Drake wrote: > On Thu, Aug 12, 2010 at 5:59 PM, Chris Withers wrote: >> I guess my point is that, by way of being based on setuptools, buildout >> already *is* case insensitive w.r.t. distribution names and the fact that >> *only* version pinning *is* case sensitive ends up being a bug... > > The fix for probably include abandoning setuptools and case-senseless > file systems. OK, and back in reality, buildout-versions will likely grow a fix for this within the next 24 hrs... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From pje at telecommunity.com Fri Aug 13 05:05:43 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 12 Aug 2010 23:05:43 -0400 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: References: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> Message-ID: <20100813030553.53C8D3A4758@sparrow.telecommunity.com> At 11:36 AM 8/12/2010 +0530, pavi ena wrote: >Scripts\python.exe setup.py install -vvv Change the above line to: Scripts\python.exe setup.py build -b btmp install -vvv If this change makes it work, then the problem is indeed the bug pointed out by Takayuki SHIMIZUKAWA: http://bugs.python.org/issue6884 Otherwise, I'm not sure what's going on. From chris at simplistix.co.uk Fri Aug 13 17:02:27 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 13 Aug 2010 16:02:27 +0100 Subject: [Distutils] Announcing buildout-versions 1.3 Message-ID: <4C655E83.8020705@simplistix.co.uk> Hi All, I'm pleased to announce a new release of buildout-versions. This release: - Makes Buildout Versions check its own version has been pinned. - Makes Buildout?s version constraints agnostic of the case of distribution names. - Has a bit of a documentation tidy-up. The PyPI page is here: http://pypi.python.org/pypi/buildout-versions The documentation is here: http://packages.python.org/buildout-versions/use.html If you have any problems or suggestions, please let me know on this list or the Simplistix open source google group! cheers, Chris From m.van.rees at zestsoftware.nl Fri Aug 13 22:42:35 2010 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Fri, 13 Aug 2010 22:42:35 +0200 Subject: [Distutils] Announcing buildout-versions 1.3 In-Reply-To: <4C655E83.8020705@simplistix.co.uk> References: <4C655E83.8020705@simplistix.co.uk> Message-ID: Op 13-08-10 17:02, Chris Withers schreef: > Hi All, > > I'm pleased to announce a new release of buildout-versions. > > This release: > > - Makes Buildout Versions check its own version has been pinned. > > - Makes Buildout?s version constraints agnostic of the case of > distribution names. > > - Has a bit of a documentation tidy-up. > > The PyPI page is here: > http://pypi.python.org/pypi/buildout-versions > > The documentation is here: > http://packages.python.org/buildout-versions/use.html > > If you have any problems or suggestions, please let me know on this list > or the Simplistix open source google group! I am used to buildout.dumppickedversions instead, but buildout-versions is growing on me. Perhaps it is just that the name is a few characters smaller. :) I tried 1.3 and I have just one minor gripe; with python2.4 you get a SyntaxError when first installing it in a buildout: Getting distribution for 'buildout-versions'. install_dir /Users/mauritsvanrees/shared-eggs/tmpdGrmyf File "build/bdist.macosx-10.6-i386/egg/buildout_versions/tests/test_constrain.py", line 55 with ShouldRaise(IncompatibleVersionError( ^ SyntaxError: invalid syntax File "/Users/mauritsvanrees/shared-eggs/tmpdGrmyf/buildout_versions-1.3-py2.4.egg/buildout_versions/tests/test_constrain.py", line 55 with ShouldRaise(IncompatibleVersionError( ^ SyntaxError: invalid syntax Got buildout-versions 1.3. I am used to those errors (due to scripts in Zope and Plone packages that use RestrictedPython, which has slightly different syntax) and know they are usually harmless. In this case it is harmless too: it is in a test, so it will not show up during normal use; and with python 2.5 or higher this syntax is fine. So for other users who see this: don't worry, be happy. :-) And to you: perhaps rewrite this one test for python2.4 to avoid possible confusion. Anyway, the extension seems to work fine so far. I think I may sneak it into a next project and see if anyone notices. :-) Thanks for your work, -- Maurits van Rees Programmer, Zest Software From ronaldoussoren at mac.com Tue Aug 17 12:35:39 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 17 Aug 2010 12:35:39 +0200 Subject: [Distutils] Question about testing Message-ID: <93C6E572-8849-45B2-A26A-A68C7C7CBED9@mac.com> Hi, What's the best way to bundle tests into a test archive for a distribution using distribute or setuptools? My requirements: * Tests should not be installed because the testsuite is larger than the actual package and not generally usefull * "python setup.py develop" should also not add the testsuite to sys.path * Some tests use extensions that are part of the testsuite * It should be possible to use "python setup.py test" to run the testsuite My current directory layout is simular to the CPython tree: * toplevel setup.py script * Lib/ contains modules/packages * Modules/ contains extension source * TestSuite/ contains the unittests The setup.py file contains a package_dir declaration: package_dir = { '': 'Lib', 'TestSuite': 'TestSuite' }, I also have a custom build_ext command to ensure that extensions in the testsuite get copied there during the build phase (as if the inplace option is selected for just those extensions). This works but feels rather fragile, and I wonder if there is a cleaner solution for meeting my requirements. (I'm using plain unittest for the tests and will probably switch to unittest2 in the near future) Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From bradallen137 at gmail.com Tue Aug 17 15:37:45 2010 From: bradallen137 at gmail.com (Brad Allen) Date: Tue, 17 Aug 2010 08:37:45 -0500 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: <20100813030553.53C8D3A4758@sparrow.telecommunity.com> References: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> <20100813030553.53C8D3A4758@sparrow.telecommunity.com> Message-ID: I am not sure if this got lost in the shuffle, but Praveen said this problem only occurs if the package name begins with the string "build". The problem doesn't occur if the package is renamed to something else. On Thu, Aug 12, 2010 at 10:05 PM, P.J. Eby wrote: > At 11:36 AM 8/12/2010 +0530, pavi ena wrote: >> >> Scripts\python.exe setup.py install -vvv > > Change the above line to: > > Scripts\python.exe setup.py build -b btmp install -vvv > > If this change makes it work, then the problem is indeed the bug pointed out > by Takayuki SHIMIZUKAWA: > > http://bugs.python.org/issue6884 > > Otherwise, I'm not sure what's going on. > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Tue Aug 17 19:58:01 2010 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 17 Aug 2010 13:58:01 -0400 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: References: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> <20100813030553.53C8D3A4758@sparrow.telecommunity.com> Message-ID: <20100817175807.B9C533A4100@sparrow.telecommunity.com> At 08:37 AM 8/17/2010 -0500, Brad Allen wrote: >I am not sure if this got lost in the shuffle, but Praveen said this >problem only occurs if the package name begins with the string >"build". The problem doesn't occur if the package is renamed to >something else. That's why I gave the instructions that I did. The likely problem is that distutils is doing an exclusion on the "build" directory -- but it uses the "build base" directory to determine the exclusion, i.e.: self.filelist.exclude_pattern(None, prefix=build.build_base) This means you can (probably) work around the issue by adding "build -b btmp", so that the exclusion is applied to files beginning with 'btmp' instead of with 'build'. Hence, the instructions you see below: >On Thu, Aug 12, 2010 at 10:05 PM, P.J. Eby wrote: > > At 11:36 AM 8/12/2010 +0530, pavi ena wrote: > >> > >> Scripts\python.exe setup.py install -vvv > > > > Change the above line to: > > > > Scripts\python.exe setup.py build -b btmp install -vvv > > > > If this change makes it work, then the problem is indeed the bug > pointed out > > by Takayuki SHIMIZUKAWA: > > > > http://bugs.python.org/issue6884 > > > > Otherwise, I'm not sure what's going on. From pje at telecommunity.com Tue Aug 17 20:09:34 2010 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 17 Aug 2010 14:09:34 -0400 Subject: [Distutils] Question about testing In-Reply-To: <93C6E572-8849-45B2-A26A-A68C7C7CBED9@mac.com> References: <93C6E572-8849-45B2-A26A-A68C7C7CBED9@mac.com> Message-ID: <20100817180941.E53843A4100@sparrow.telecommunity.com> At 12:35 PM 8/17/2010 +0200, Ronald Oussoren wrote: >Hi, > >What's the best way to bundle tests into a test archive for a >distribution using distribute or setuptools? > >My requirements: >* Tests should not be installed because the testsuite is larger than >the actual package and not generally usefull >* "python setup.py develop" should also not add the testsuite to sys.path >* Some tests use extensions that are part of the testsuite >* It should be possible to use "python setup.py test" to run the testsuite With the exception of the "not add the testsuite to sys.path", these are trivially achieved by putting your test suite in a package directory alongside your code, and simply not listing that package in your packages to install. Why do you not want the test suite on sys.path? As long as you name it something like "mypackage_tests", it's unlikely to interfere with anything else. >I also have a custom build_ext command to ensure that extensions in >the testsuite get copied there during the build phase >(as if the inplace option is selected for just those extensions). If you use the "setup.py test" command, the custom build_ext is unnecessary; the test command automatically does an in-place build_ext. The one thing I'm not 100% positive on is whether you can build the test extensions without also installing them. If not, then probably what you want is to subclass the *test* command instead, to have it add the test extensions, then fall through to the normal test command implementation. That way, there's no possibility for them to be installed under normal conditions. >This works but feels rather fragile, and I wonder if there is a >cleaner solution for meeting my requirements. > >(I'm using plain unittest for the tests and will probably switch to >unittest2 in the near future) If you are using plain unittest, then making your tests a package and setting: test_suite = 'mypackage_tests', # or whatever the package is called will automatically test all TestCase subclasses in every submodule and subpackage, and also call any functions named "additional_tests()" to get extra test suites. The latter is especially useful for running text-based doctests as part of your overall unittest suite; typically I'll have a few like this: def additional_tests(): import doctest return doctest.DocFileSuite( 'README.txt', 'sometests.txt', optionflags=doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE, ) But of course you can return any sort of unittest.TestSuite. The main point is just that if you have things in a module that you want to test that aren't just TestCase subclasses, you can slap them in an additional_tests() function for the "setup.py test" command to add in to the overall testing process. From reinout at vanrees.org Wed Aug 18 15:11:34 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 18 Aug 2010 15:11:34 +0200 Subject: [Distutils] Announcing buildout-versions 1.3 In-Reply-To: References: <4C655E83.8020705@simplistix.co.uk> Message-ID: On 08/13/2010 10:42 PM, Maurits van Rees wrote: > > Anyway, the extension seems to work fine so far. I think I may sneak it > into a next project and see if anyone notices. :-) I already snug it into several projects without anyone noticing. Not seeing any message when there's nothing to report is way prettier. Only I had to revert it back for one project as I needed to make a tarball out of it with z3c.sourcerelease. Somehow that one did not pick up buildout-versions, even though buildout.dumppickedversions. So there was no buildout-versions egg in the everything-included .tgz! I might have to re-run the test now that 1.3 is out as that pins itself, too. Perhaps it also un-hides itself for z3c.sourcerelease? Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Wed Aug 18 15:48:24 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 18 Aug 2010 15:48:24 +0200 Subject: [Distutils] Announcing buildout-versions 1.3 In-Reply-To: References: <4C655E83.8020705@simplistix.co.uk> Message-ID: On 08/18/2010 03:11 PM, Reinout van Rees wrote: > On 08/13/2010 10:42 PM, Maurits van Rees wrote: >> >> Anyway, the extension seems to work fine so far. I think I may sneak it >> into a next project and see if anyone notices. :-) > > I already snug it into several projects without anyone noticing. > > Not seeing any message when there's nothing to report is way prettier. > > Only I had to revert it back for one project as I needed to make a > tarball out of it with z3c.sourcerelease. Somehow that one did not pick > up buildout-versions, even though buildout.dumppickedversions. So there > was no buildout-versions egg in the everything-included .tgz! > > I might have to re-run the test now that 1.3 is out as that pins itself, > too. Perhaps it also un-hides itself for z3c.sourcerelease? Actually, I was already using 1.3, so I guess this is a bug. I'll look deeper into this when the project is ready. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From chris at simplistix.co.uk Wed Aug 18 15:51:40 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 18 Aug 2010 14:51:40 +0100 Subject: [Distutils] Announcing buildout-versions 1.3 In-Reply-To: References: <4C655E83.8020705@simplistix.co.uk> Message-ID: <4C6BE56C.5040800@simplistix.co.uk> Maurits van Rees wrote: > worry, be happy. :-) And to you: perhaps rewrite this one test for > python2.4 to avoid possible confusion. Patches accepted ;-) (otherwise I'll get to it when I get to it) > Anyway, the extension seems to work fine so far. I think I may sneak it > into a next project and see if anyone notices. :-) Thanks! Chris From chris at simplistix.co.uk Wed Aug 18 15:53:32 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 18 Aug 2010 14:53:32 +0100 Subject: [Distutils] Announcing buildout-versions 1.3 In-Reply-To: References: <4C655E83.8020705@simplistix.co.uk> Message-ID: <4C6BE5DC.3010104@simplistix.co.uk> Reinout van Rees wrote: >> Only I had to revert it back for one project as I needed to make a >> tarball out of it with z3c.sourcerelease. Somehow that one did not pick >> up buildout-versions, even though buildout.dumppickedversions. So there >> was no buildout-versions egg in the everything-included .tgz! >> >> I might have to re-run the test now that 1.3 is out as that pins itself, >> too. Perhaps it also un-hides itself for z3c.sourcerelease? > > Actually, I was already using 1.3, so I guess this is a bug. I'll look > deeper into this when the project is ready. I don't know anything about z3c.sourcerelease, so any pointers you can give would be helpful! (my only guess: you don't have buildout-versions in development mode? The only thing similar to what you described was when I had buildout-versions in develop mode in my ~/.buildout/default.cfg... Chris From potterpavi at gmail.com Thu Aug 19 07:02:13 2010 From: potterpavi at gmail.com (pavi ena) Date: Thu, 19 Aug 2010 10:32:13 +0530 Subject: [Distutils] Issue with Setuptools package name In-Reply-To: <20100813030553.53C8D3A4758@sparrow.telecommunity.com> References: <20100811144251.10FAF3A4108@sparrow.telecommunity.com> <20100813030553.53C8D3A4758@sparrow.telecommunity.com> Message-ID: Hello Eby, I changed the line to "Scripts\python.exe setup.py build -b btmp install -vvv" it works for me. Thanks for suggestion to fix my problem. On Fri, Aug 13, 2010 at 8:35 AM, P.J. Eby wrote: > At 11:36 AM 8/12/2010 +0530, pavi ena wrote: > >> Scripts\python.exe setup.py install -vvv >> > > Change the above line to: > > Scripts\python.exe setup.py build -b btmp install -vvv > > If this change makes it work, then the problem is indeed the bug pointed > out by Takayuki SHIMIZUKAWA: > > > http://bugs.python.org/issue6884 > > Otherwise, I'm not sure what's going on. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From udono at gmx.net Thu Aug 19 14:05:39 2010 From: udono at gmx.net (Udo Spallek) Date: Thu, 19 Aug 2010 14:05:39 +0200 Subject: [Distutils] Change the python interpreter for a recipe Message-ID: <1282219539.14800.140.camel@lisa.localdomain> Hi, having a problem with my buildout and like to request some hints. I have a buildout which creates a simple python interpreter environment like explained in the buildout manual. Additionally I use a custom script. My problem is: How can I tell my_recipe to use the (newly created) mypython interpreter, instead of using the standard python interpreter? Or as a workaround: Is there a way to install the custom interpreter before the buildout, maybe on bootstrap? I like to avoid using virtualenv. TIA and cheers Udo Spallek ------------ buildout.cfg ------------ [buildout] parts = mypython myrecipe [mypython] recipe = zc.recipe.egg interpreter = mypython eggs = relatorio [myrecipe] recipe = my_recipe interpreter = mypython # This doesn't work From jim at zope.com Thu Aug 19 15:09:00 2010 From: jim at zope.com (Jim Fulton) Date: Thu, 19 Aug 2010 09:09:00 -0400 Subject: [Distutils] Change the python interpreter for a recipe In-Reply-To: <1282219539.14800.140.camel@lisa.localdomain> References: <1282219539.14800.140.camel@lisa.localdomain> Message-ID: On Thu, Aug 19, 2010 at 8:05 AM, Udo Spallek wrote: > Hi, > > having a problem with my buildout and like to request some hints. > > I have a buildout which creates a simple python interpreter environment > like explained in the buildout manual. So an interpreter script. > Additionally I use a custom > script. Not sure what that is or why it matters. > > My problem is: How can I tell my_recipe to use the (newly created) > mypython interpreter, instead of using the standard python interpreter? You can't. > Or as a workaround: Is there a way to install the custom interpreter > before the buildout, maybe on bootstrap? The custom interpreter isn't a true Python installation. Generally, you aren't going to be able to bootstrap a buildout with it. Why do you want to do this? What problem are you trying to solve? Jim -- Jim Fulton From udono at gmx.net Thu Aug 19 15:53:13 2010 From: udono at gmx.net (udono) Date: Thu, 19 Aug 2010 15:53:13 +0200 Subject: [Distutils] Change the python interpreter for a recipe (solved) In-Reply-To: References: <1282219539.14800.140.camel@lisa.localdomain> Message-ID: <1282225993.14800.180.camel@lisa.localdomain> Hi Jim, thanks for your reply. I need to re-thing my plan for the builout and found some ideas. Am Donnerstag, den 19.08.2010, 09:09 -0400 schrieb Jim Fulton: > On Thu, Aug 19, 2010 at 8:05 AM, Udo Spallek wrote: > > having a problem with my buildout and like to request some hints. > > I have a buildout which creates a simple python interpreter environment > > like explained in the buildout manual. > So an interpreter script. > > Additionally I use a custom > > script. > Not sure what that is or why it matters. It is a script which use the API of a trytond server (www.tryton.org) to create a database and install several modules into this database. The dependencies of trytond server are installed into the mypython environment. So when I wanted to maintain trytond I need to start it within the mypython environment, for now. > > My problem is: How can I tell my_recipe to use the (newly created) > > mypython interpreter, instead of using the standard python interpreter? > You can't. > > Or as a workaround: Is there a way to install the custom interpreter > > before the buildout, maybe on bootstrap? > The custom interpreter isn't a true Python installation. Generally, > you aren't going to be able to bootstrap a buildout with it. > Why do you want to do this? What problem are you trying to solve? Problem is to buildout a complete deployment of trytond including a database and installation of all needed modules. The buildout has these coarse-grained stages/parts: 1. Install dependencies into mypython as eggs 2. Install trytond and all tryton-modules needed 3. Create Database/Install Modules: (my custom recipe) import trytond create_database if not exist install_modules into database 4. Debian system works (init.d-script, links to /var/log, /var/run,...) Part 3 is the problem, because for now, import trytond needs to be started in the mypython environment, because of the dependencies. I think I rewrite this part and use the xmlrpc API of trytond and start the server in a prior part with a system call to an init.d-script. Then I do the steps from 3. inside the buildout via xmlrpc. This way I do not need the mypython environment in my recipe. Thanks a lot! Cheers Udo From chris at simplistix.co.uk Thu Aug 19 19:29:06 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 19 Aug 2010 18:29:06 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? Message-ID: <4C6D69E2.90606@simplistix.co.uk> Hi All, Given a distribution name (ie: one of those things setuptools uses, 'name' parameter to setup.py's setup call, etc) of a dsitrbution that has been installed with easy_install, buildout, pip, etc, how can I find out what python packages (ie: dotted names that I can import) are provided by that package? I'm trying to write a scanner than looks for SQLAlchemy table objects and it would be great to be able to pass a distribution name rather than a set of dotted package names... cheers, Chris From pje at telecommunity.com Thu Aug 19 20:48:28 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 19 Aug 2010 14:48:28 -0400 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <4C6D69E2.90606@simplistix.co.uk> References: <4C6D69E2.90606@simplistix.co.uk> Message-ID: <20100819184838.7708C3A4100@sparrow.telecommunity.com> At 06:29 PM 8/19/2010 +0100, Chris Withers wrote: >Hi All, > >Given a distribution name (ie: one of those things setuptools uses, >'name' parameter to setup.py's setup call, etc) of a dsitrbution >that has been installed with easy_install, buildout, pip, etc, how >can I find out what python packages (ie: dotted names that I can >import) are provided by that package? > >I'm trying to write a scanner than looks for SQLAlchemy table >objects and it would be great to be able to pass a distribution name >rather than a set of dotted package names... get_distribution('projectname').get_metadata('top_level.txt').splitlines() will return you a list of top-level package or module names provided by the package. Subpackages you'll have to find yourself; the pkgutil module in 2.5+ has a walk_packages() that might be helpful. (Note that if there are packages installed in "flat" form (e.g. pip-installed or system-installed), you may see overlapping packages in the same directory, so simply walking packages in a distribution object's base directory may include packages and modules from another project.) From chris at simplistix.co.uk Fri Aug 20 00:32:26 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 19 Aug 2010 23:32:26 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <20100819184838.7708C3A4100@sparrow.telecommunity.com> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> Message-ID: <4C6DB0FA.3040409@simplistix.co.uk> P.J. Eby wrote: >> I'm trying to write a scanner than looks for SQLAlchemy table objects >> and it would be great to be able to pass a distribution name rather >> than a set of dotted package names... > > get_distribution('projectname').get_metadata('top_level.txt').splitlines() > will return you a list of top-level package or module names provided by > the package. Ah, okay. I assume this will only work for packages that use setuptools? > Subpackages you'll have to find yourself; the pkgutil module in 2.5+ has > a walk_packages() that might be helpful. > > (Note that if there are packages installed in "flat" form (e.g. > pip-installed or system-installed), you may see overlapping packages in > the same directory, so simply walking packages in a distribution > object's base directory may include packages and modules from another > project.) I assume this is only a problem for namespace packages? (ie: if so packages each supply part of a namespace, they will both have the namespace listed in top_level.txt, right?) How do static resources returned with pkg_resources.resource_filename and friends fare when packages are installed in a "flat" form? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From pje at telecommunity.com Fri Aug 20 01:14:52 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 19 Aug 2010 19:14:52 -0400 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <4C6DB0FA.3040409@simplistix.co.uk> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> Message-ID: <20100819231503.A7D343A4100@sparrow.telecommunity.com> At 11:32 PM 8/19/2010 +0100, Chris Withers wrote: >P.J. Eby wrote: >>>I'm trying to write a scanner than looks for SQLAlchemy table >>>objects and it would be great to be able to pass a distribution >>>name rather than a set of dotted package names... >>get_distribution('projectname').get_metadata('top_level.txt').splitlines() >>will return you a list of top-level package or module names >>provided by the package. > >Ah, okay. I assume this will only work for packages that use setuptools? Or distribute. Or pip (I assume). Anything but plain distutils, basically. >I assume this is only a problem for namespace packages? >(ie: if so packages each supply part of a namespace, they will both >have the namespace listed in top_level.txt, right?) Correct. >How do static resources returned with >pkg_resources.resource_filename and friends fare when packages are >installed in a "flat" form? They go in the same relative location as they would in an egg, it's just that in flat layouts everyone's files are in the same namespace. Sort of a "scrambled egg", if you will. ;-) From chris at simplistix.co.uk Fri Aug 20 01:20:07 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 20 Aug 2010 00:20:07 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <20100819231503.A7D343A4100@sparrow.telecommunity.com> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> Message-ID: <4C6DBC27.1060006@simplistix.co.uk> P.J. Eby wrote: >> Ah, okay. I assume this will only work for packages that use setuptools? > > Or distribute. Or pip (I assume). Anything but plain distutils, > basically. Well, the packages import setup from setuptools for all of the above, apart from plain distutils ;-) >> How do static resources returned with pkg_resources.resource_filename >> and friends fare when packages are installed in a "flat" form? > > They go in the same relative location as they would in an egg, it's just > that in flat layouts everyone's files are in the same namespace. > > Sort of a "scrambled egg", if you will. ;-) Okay, so if we have two distributions, x.a and x.b both define template folders called 'templates', and each has a template called 'master', what ends up on disk? what does pkg_resources.resource_filename('x.a','templates/master') return? It should be different to pkg_resources.resource_filename('x.b','templates/master'), but will it be? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From pje at telecommunity.com Fri Aug 20 01:40:55 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 19 Aug 2010 19:40:55 -0400 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <4C6DBC27.1060006@simplistix.co.uk> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> Message-ID: <20100819234105.8DABA3A4100@sparrow.telecommunity.com> At 12:20 AM 8/20/2010 +0100, Chris Withers wrote: >P.J. Eby wrote: >>>Ah, okay. I assume this will only work for packages that use setuptools? >>Or distribute. Or pip (I assume). Anything but plain distutils, basically. > >Well, the packages import setup from setuptools for all of the >above, apart from plain distutils ;-) If you use any of those tools to do the installing, it doesn't matter whether the package itself imports setuptools or distutils. >>>How do static resources returned with >>>pkg_resources.resource_filename and friends fare when packages are >>>installed in a "flat" form? >>They go in the same relative location as they would in an egg, it's >>just that in flat layouts everyone's files are in the same namespace. >>Sort of a "scrambled egg", if you will. ;-) > >Okay, so if we have two distributions, x.a and x.b both define >template folders called 'templates', and each has a template called >'master', what ends up on disk? what does >pkg_resources.resource_filename('x.a','templates/master') return? >It should be different to >pkg_resources.resource_filename('x.b','templates/master'), but will it be? You seem to be confusing distributions and packages. If 'x.a' and 'x.b' are package names, then each will have its own directory ('x/a/templates' or 'x/b/templates'). If they used install_data to specify non-package data, OTOH, it'll be mixed. That's why the recommendation is to use package_data or include_package_data; if both projects included their data using one of those two setup() options, they'll be fine. From reinout at vanrees.org Fri Aug 20 10:03:15 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Fri, 20 Aug 2010 10:03:15 +0200 Subject: [Distutils] Change the python interpreter for a recipe (solved) In-Reply-To: <1282225993.14800.180.camel@lisa.localdomain> References: <1282219539.14800.140.camel@lisa.localdomain> <1282225993.14800.180.camel@lisa.localdomain> Message-ID: On 08/19/2010 03:53 PM, udono wrote: > Problem is to buildout a complete deployment of trytond including a > database and installation of all needed modules. The buildout has these > coarse-grained stages/parts: > > 1. Install dependencies into mypython as eggs > 2. Install trytond and all tryton-modules needed > 3. Create Database/Install Modules: (my custom recipe) > import trytond > create_database if not exist > install_modules into database > 4. Debian system works (init.d-script, links to /var/log, /var/run,...) > > Part 3 is the problem, because for now, import trytond needs to be > started in the mypython environment, because of the dependencies. You could make a recipe that just calls bin/mypython with the options you need. os.system() (or rather the subprocess.Popen() call, look at zc.buildout's easyinstall.py around line 147 for an example). Perhaps make the assumption that you've got a [trytond] part. You can then grab the ${trytond:interpreter} option from within your recipe and grab the bin directory from the buildout options. Assuming the [trytond] part runs before your recipe you're pretty much set, I'd guess. Bonus points if your recipe can detect whether it actually needs to run by checking whether a database file exists or something like that. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From chris at simplistix.co.uk Fri Aug 20 11:20:11 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 20 Aug 2010 10:20:11 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <20100819234105.8DABA3A4100@sparrow.telecommunity.com> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> <20100819234105.8DABA3A4100@sparrow.telecommunity.com> Message-ID: <4C6E48CB.9030302@simplistix.co.uk> P.J. Eby wrote: >>>> How do static resources returned with >>>> pkg_resources.resource_filename and friends fare when packages are >>>> installed in a "flat" form? >>> They go in the same relative location as they would in an egg, it's >>> just that in flat layouts everyone's files are in the same namespace. >>> Sort of a "scrambled egg", if you will. ;-) >> >> Okay, so if we have two distributions, x.a and x.b both define >> template folders called 'templates', and each has a template called >> 'master', what ends up on disk? what does >> pkg_resources.resource_filename('x.a','templates/master') return? >> It should be different to >> pkg_resources.resource_filename('x.b','templates/master'), but will it >> be? > > You seem to be confusing distributions and packages. Potentially, is the first argument to resource_filename a distributions name or a dotted package named? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Aug 20 12:49:39 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 20 Aug 2010 11:49:39 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <20100819234105.8DABA3A4100@sparrow.telecommunity.com> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> <20100819234105.8DABA3A4100@sparrow.telecommunity.com> Message-ID: <4C6E5DC3.9070003@simplistix.co.uk> P.J. Eby wrote: >> Okay, so if we have two distributions, x.a and x.b both define >> template folders called 'templates', and each has a template called >> 'master', what ends up on disk? what does >> pkg_resources.resource_filename('x.a','templates/master') return? >> It should be different to >> pkg_resources.resource_filename('x.b','templates/master'), but will it >> be? > > You seem to be confusing distributions and packages. If 'x.a' and 'x.b' > are package names, then each will have its own directory > ('x/a/templates' or 'x/b/templates'). If they used install_data to > specify non-package data, OTOH, it'll be mixed. Now that I'm somewhere I can read the docs, I can be clearer. 'x.a' and 'x.b' were distribution names, such distributions names are quite common. However, for absolute clarity, assuming we have: - A distribution called 'XA', that contains an 'x.a' package but also a 'templates' directory at the root of the distribution, containing a 'master' template - A distribution called 'XB', that contains an 'x.b' package but also a 'templates' directory at the root of the distribution, containing a 'master' template. Which template do I get if I do: pkg_resources.resource_string(tuple(pkg_resources.parse_requirements('XA'))[0],'templates/master') ...in the context of pip or another flat-structure install? Would it work okay if each distribution was unpacked in its own folder (as buildout and easy_install do)? cheers, Chris From pje at telecommunity.com Fri Aug 20 16:17:12 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 20 Aug 2010 10:17:12 -0400 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <4C6E5DC3.9070003@simplistix.co.uk> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> <20100819234105.8DABA3A4100@sparrow.telecommunity.com> <4C6E5DC3.9070003@simplistix.co.uk> Message-ID: <20100820141724.BC5DD3A4119@sparrow.telecommunity.com> At 11:49 AM 8/20/2010 +0100, Chris Withers wrote: >P.J. Eby wrote: >>>Okay, so if we have two distributions, x.a and x.b both define >>>template folders called 'templates', and each has a template >>>called 'master', what ends up on disk? what does >>>pkg_resources.resource_filename('x.a','templates/master') return? >>>It should be different to >>>pkg_resources.resource_filename('x.b','templates/master'), but will it be? >>You seem to be confusing distributions and packages. If 'x.a' and >>'x.b' are package names, then each will have its own directory >>('x/a/templates' or 'x/b/templates'). If they used install_data to >>specify non-package data, OTOH, it'll be mixed. > >Now that I'm somewhere I can read the docs, I can be clearer. >'x.a' and 'x.b' were distribution names, such distributions names >are quite common. However, for absolute clarity, assuming we have: > >- A distribution called 'XA', that contains an 'x.a' package but >also a 'templates' directory at the root of the distribution, >containing a 'master' template > >- A distribution called 'XB', that contains an 'x.b' package but >also a 'templates' directory at the root of the distribution, >containing a 'master' template. > >Which template do I get if I do: > >pkg_resources.resource_string(tuple(pkg_resources.parse_requirements('XA'))[0],'templates/master') > >...in the context of pip or another flat-structure install? You get whatever was installed last. >Would it work okay if each distribution was unpacked in its own >folder (as buildout and easy_install do)? Yep. And of course, it'll also work if they put their templates under x/a or x/b, like they really ought to be doing in the first place, for precisely this reason. (Alternatively, they can put their stuff under the project's egg-info directory, the way EggTranslations does.) From chris at simplistix.co.uk Fri Aug 20 16:19:38 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 20 Aug 2010 15:19:38 +0100 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <20100820141724.BC5DD3A4119@sparrow.telecommunity.com> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> <20100819234105.8DABA3A4100@sparrow.telecommunity.com> <4C6E5DC3.9070003@simplistix.co.uk> <20100820141724.BC5DD3A4119@sparrow.telecommunity.com> Message-ID: <4C6E8EFA.3070703@simplistix.co.uk> P.J. Eby wrote: >> Which template do I get if I do: >> >> pkg_resources.resource_string(tuple(pkg_resources.parse_requirements('XA'))[0],'templates/master') >> >> >> ...in the context of pip or another flat-structure install? > > You get whatever was installed last. Is pkg_resources.resource_string(,..) *ever* useful or sane then? It doesn't sound like it, in which case, could it be deprecated and removed please? > (Alternatively, they can put their stuff under the project's egg-info > directory, the way EggTranslations does.) What's EggTranslations and where can I see its code? Chris From gary.poster at canonical.com Fri Aug 20 17:22:31 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 20 Aug 2010 11:22:31 -0400 Subject: [Distutils] Buildout release news Message-ID: I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge the code from from svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix into trunk. This will change the bootstrap.py script on which some people depend via svn externals. Also, existing bootstrap and buildout invocations that do not specify a particular buildout release will automatically get the new release, as usual. In light of the troubles with the 1.5.0b1 release, particularly for users of virtualenv, I have done four things. 1. I added an automated test for the virtualenv interaction problem encountered in the last release, and fixed it. If you are using virtualenv, it now uses the old behavior for generating scripts. I and others have had success manually testing virtualenv + buildout with with these changes. 2. Today I released zc.buildout 1.4.4 (http://pypi.python.org/pypi/zc.buildout/1.4.4). The 1.4.4 release is a release for people who encounter trouble with the 1.5 line. By switching to the associated bootstrap script (http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py) you can stay on 1.4.4 until you are ready to migrate. 3. I have a new help document describing the changes in the 1.5 line. This will be part of the documentation shown on pypi. You can see it here: http://svn.zope.org/zc.buildout/branches/gary-betafix/SYSTEM_PYTHON_HELP.txt?view=markup 4. For the future, the new (1.5) versions of Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions. This can be changed by using the --accept-buildout-test-releases flag (or -t for short) when calling bootstrap. This will hopefully allow beta releases of these items to be more easily and safely made in the future. The change log is here: http://svn.zope.org/zc.buildout/branches/gary-betafix/CHANGES.txt?view=markup Gary From attilaolah at gmail.com Fri Aug 20 17:38:07 2010 From: attilaolah at gmail.com (=?UTF-8?B?QXR0aWxhIE9sw6Fo?=) Date: Fri, 20 Aug 2010 17:38:07 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: Hi, On Fri, Aug 20, 2010 at 17:22, Gary Poster wrote: > I intend to release zc.buildout 1.5.0 Monday, August 23. ?I will merge the code from from svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix into trunk. Is this the branch that Jim mentioned here: https://bugs.launchpad.net/zc.buildout/+bug/429383/comments/2 ? I still couldn't find a way to generate buildout scripts with excluding the system site-packages. Also, I had troubles with the following bug a while ago, that seems to be fixed in 1.5.0b1 now: - Installing a namespace package using a Python that already has a package in the same namespace (e.g., in the Python's site-packages) failed in some cases. It is now handled correctly. Is there a bootstrap.py file that can be used for early-testing of the release that you're going to make in three days? Attila From barry at python.org Fri Aug 20 17:49:15 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 20 Aug 2010 11:49:15 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: <20100820114915.4d3e39e2@heresy> On Aug 20, 2010, at 11:22 AM, Gary Poster wrote: >I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge >the code from from >svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix >into trunk. Cool. Once you do the merge (or maybe before), what's the best way to test the code on our own projects? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From pje at telecommunity.com Fri Aug 20 18:07:43 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 20 Aug 2010 12:07:43 -0400 Subject: [Distutils] how can I egt the packages and modules in a distribution? In-Reply-To: <4C6E8EFA.3070703@simplistix.co.uk> References: <4C6D69E2.90606@simplistix.co.uk> <20100819184838.7708C3A4100@sparrow.telecommunity.com> <4C6DB0FA.3040409@simplistix.co.uk> <20100819231503.A7D343A4100@sparrow.telecommunity.com> <4C6DBC27.1060006@simplistix.co.uk> <20100819234105.8DABA3A4100@sparrow.telecommunity.com> <4C6E5DC3.9070003@simplistix.co.uk> <20100820141724.BC5DD3A4119@sparrow.telecommunity.com> <4C6E8EFA.3070703@simplistix.co.uk> Message-ID: <20100820160754.CDE163A406B@sparrow.telecommunity.com> At 03:19 PM 8/20/2010 +0100, Chris Withers wrote: >P.J. Eby wrote: >>>Which template do I get if I do: >>> >>>pkg_resources.resource_string(tuple(pkg_resources.parse_requirements('XA'))[0],'templates/master') >>> >>> >>>...in the context of pip or another flat-structure install? >>You get whatever was installed last. > >Is pkg_resources.resource_string(,..) *ever* useful or sane then? Good point. The installation of data in that way was for backward-compatibility with packages using install_data; but such packages by definition are unlikely to be using the resource APIs in the first place. >It doesn't sound like it, in which case, could it be deprecated and >removed please? I'll give it some serious thought. >>(Alternatively, they can put their stuff under the project's >>egg-info directory, the way EggTranslations does.) > >What's EggTranslations and where can I see its code? http://pypi.python.org/pypi/EggTranslations From gary.poster at canonical.com Fri Aug 20 22:17:21 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 20 Aug 2010 16:17:21 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: <0A0B0C79-10BD-4DCD-8567-B48E024B35FB@canonical.com> On Aug 20, 2010, at 11:38 AM, Attila Ol?h wrote: > Hi, > > On Fri, Aug 20, 2010 at 17:22, Gary Poster wrote: >> I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge the code from from svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix into trunk. > > Is this the branch that Jim mentioned here: > https://bugs.launchpad.net/zc.buildout/+bug/429383/comments/2 ? Yes. > I > still couldn't find a way to generate buildout scripts with excluding > the system site-packages. See the documentation I mentioned: http://svn.zope.org/zc.buildout/branches/gary-betafix/SYSTEM_PYTHON_HELP.txt?view=markup . > > Also, I had troubles with the following bug a while ago, that seems to > be fixed in 1.5.0b1 now: > > - Installing a namespace package using a Python that already has a package > in the same namespace (e.g., in the Python's site-packages) failed in > some cases. It is now handled correctly. Do you mean that you had trouble with this behavior in 1.5.0b1? If so, more details would be helpful, particularly after you try following the instructions in the help file. As you might expect, Debian/Ubuntu compatibility has been the standard I've used for my own experiments and those of my beta testers. Note that, as noted elsewhere, using a system Python is inherently fragile. I've caught about everything I can in the Debian/Ubuntu world, but there are still bugs like https://bugs.edge.launchpad.net/ubuntu/+source/python2.6/+bug/621348 that make things less smooth than they would be otherwise. > Is there a bootstrap.py file that can be used for early-testing of the > release that you're going to make in three days? The bootstrap to use for now is http://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix/bootstrap/bootstrap.py Next, as I said, read the SYSTEM_PYTHON_HELP.txt I mentioned (though, of course, the gary-betafix bootstrap I mention above is the one to use for now, not the one in trunk). Follow the pertinent instructions, and set your expectations on the basis of its warnings (in particular, the only script-generating recipe that uses the new functionality so far is z3c.recipe.scripts). Now, because things are not yet released, you'll have to do a number of extra steps. These are the ones I follow. Maybe someone else has a better approach, but this tests both buildout *and* bootstrap (other approaches only test buildout AFAIK, and you need both). 1) If you don't have one yet, set up a download-cache per the usual buildout instructions. Generally, the docs are http://pypi.python.org/pypi/zc.buildout#using-a-download-cache . More specifically, I do what is described here: https://dev.launchpad.net/HackingLazrLibraries#Global%20Cache . 2) In your download-cache's dist's directory (~/.buildout/download-cache/dist/ if you followed the Global Cache instructions) put the following four items in: - the setuptools egg for the Python version you are using (from http://pypi.python.org/pypi/setuptools) - temporary releases of the three pertinent releases, available from http://people.canonical.com/~gary/zc.buildout-1.5.0-early-releases/ 3) When you are ready to call the new bootstrap in a fresh checkout of your project, you'll need to specify the download-cache-dist directory as an argument. Here's an example of my call: python bootstrap.py --download-base=/home/gary/.buildout/download-cache/dist/ Now you should be testing the releases as I intend to make them. Note that, if I do have to make changes before that release on the basis of feedback, you will need to remove all versions of these (in download-cache and in any cached eggs) before you get the new changes. Gary From gary.poster at canonical.com Fri Aug 20 22:19:51 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 20 Aug 2010 16:19:51 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <20100820114915.4d3e39e2@heresy> References: <20100820114915.4d3e39e2@heresy> Message-ID: On Aug 20, 2010, at 11:49 AM, Barry Warsaw wrote: > On Aug 20, 2010, at 11:22 AM, Gary Poster wrote: > >> I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge >> the code from from >> svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix >> into trunk. > > Cool. Once you do the merge (or maybe before), what's the best way to test > the code on our own projects? The email I sent to Attila should have the instructions, painful as they are, for "before". After the release, the http://svn.zope.org/zc.buildout/branches/gary-betafix/SYSTEM_PYTHON_HELP.txt?view=markup doc should be what you need, alone. (As I said, that will be on the PyPI page as well.) Gary From viraptor at gmail.com Sat Aug 21 14:05:54 2010 From: viraptor at gmail.com (=?UTF-8?B?U3RhbmlzxYJhdyBQaXR1Y2hh?=) Date: Sat, 21 Aug 2010 13:05:54 +0100 Subject: [Distutils] Including tests in source package Message-ID: <4C6FC122.4030308@gmail.com> Hi all, I'm trying to package tests file in the sdist only and it doesn't really work :/ If I do: setup( packages = find_packages(exclude=['tests']), test_suite = "tests", ... this doesn't package tests anywhere. Using either general find_packages includes tests in both sdist and bdist. How can I resolve this? Essentially I want either tests.py, or tests/__init__.py to be distributed in sdist, but not in bdist. Thanks, Stan From ben+python at benfinney.id.au Sat Aug 21 14:50:29 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 21 Aug 2010 22:50:29 +1000 Subject: [Distutils] Including tests in source package References: <4C6FC122.4030308@gmail.com> Message-ID: <871v9sruy2.fsf@benfinney.id.au> Stanis?aw Pitucha writes: > If I do: > > setup( > packages = find_packages(exclude=['tests']), > test_suite = "tests", > ... > > this doesn't package tests anywhere. [?] > Essentially I want either tests.py, or tests/__init__.py to be > distributed in sdist, but not in bdist. I am using effectively the same as you describe above, but I get the result you are seeking: the sdist contains the test suite, and the binary dist does not. Have a look at Gracie for an example that works for me. Perhaps it can help you find what's different that causes the problem you're seeing. -- \ ?Few things are harder to put up with than the annoyance of a | `\ good example.? ?Mark Twain, _Pudd'n'head Wilson_ | _o__) | Ben Finney From viraptor at gmail.com Sat Aug 21 17:59:46 2010 From: viraptor at gmail.com (=?UTF-8?B?U3RhbmlzxYJhdyBQaXR1Y2hh?=) Date: Sat, 21 Aug 2010 16:59:46 +0100 Subject: [Distutils] Including tests in source package In-Reply-To: <871v9sruy2.fsf@benfinney.id.au> References: <4C6FC122.4030308@gmail.com> <871v9sruy2.fsf@benfinney.id.au> Message-ID: <4C6FF7F2.2020401@gmail.com> On 21.08.2010 13:50, Ben Finney wrote: > I am using effectively the same as you describe above, but I get the > result you are seeking: the sdist contains the test suite, and the > binary dist does not. > > Have a look at Gracie for an > example that works for me. Perhaps it can help you find what's different > that causes the problem you're seeing. Thanks a lot. I looked through your module, but I really cannot find what's wrong with mine. I moved my tests to 'tests.suite' to match your layout, but that didn't work. I see that when your egg_info/SOURCES.txt is generated, it includes the test module. My package doesn't do that - tests are never included in the SOURCES.txt unless I specifically remove "excludes = ['tests']". Does anyone have other ideas for fixing this? Thanks, Stan From viraptor at gmail.com Sat Aug 21 18:54:38 2010 From: viraptor at gmail.com (=?UTF-8?B?U3RhbmlzxYJhdyBQaXR1Y2hh?=) Date: Sat, 21 Aug 2010 17:54:38 +0100 Subject: [Distutils] Including tests in source package In-Reply-To: <20100821164341.0FD843A406B@sparrow.telecommunity.com> References: <4C6FC122.4030308@gmail.com> <20100821164341.0FD843A406B@sparrow.telecommunity.com> Message-ID: <4C7004CE.3010506@gmail.com> On 21.08.2010 17:43, P.J. Eby wrote: > You need either a revision control system, or a MANIFEST.in file (see > distutils docs regarding MANIFEST.in). And that was the key... I didn't have the MANIFEST.in file. Adding 'recursive-include test *.py' solved the problem. Packages reuploaded and I'm happy :) Thanks, Stan From gary.poster at canonical.com Mon Aug 23 17:43:30 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 23 Aug 2010 11:43:30 -0400 Subject: [Distutils] Buildout 1.5.0 and friends released (was Re: Buildout release news) In-Reply-To: References: Message-ID: I have merged the betafix branch to trunk and made the releases of zc.buildout 1.5.0, zc.recipe.egg 1.3.0, and z3c.recipe.scripts 1.0.0. Gary On Aug 20, 2010, at 11:22 AM, Gary Poster wrote: > I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge the code from from svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix into trunk. > > This will change the bootstrap.py script on which some people depend via svn externals. Also, existing bootstrap and buildout invocations that do not specify a particular buildout release will automatically get the new release, as usual. > > In light of the troubles with the 1.5.0b1 release, particularly for users of virtualenv, I have done four things. > > 1. I added an automated test for the virtualenv interaction problem encountered in the last release, and fixed it. If you are using virtualenv, it now uses the old behavior for generating scripts. I and others have had success manually testing virtualenv + buildout with with these changes. > > 2. Today I released zc.buildout 1.4.4 (http://pypi.python.org/pypi/zc.buildout/1.4.4). The 1.4.4 release is a release for people who encounter trouble with the 1.5 line. By switching to the associated bootstrap script (http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py) you can stay on 1.4.4 until you are ready to migrate. > > 3. I have a new help document describing the changes in the 1.5 line. This will be part of the documentation shown on pypi. You can see it here: http://svn.zope.org/zc.buildout/branches/gary-betafix/SYSTEM_PYTHON_HELP.txt?view=markup > > 4. For the future, the new (1.5) versions of Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions. This can be changed by using the --accept-buildout-test-releases flag (or -t for short) when calling bootstrap. This will hopefully allow beta releases of these items to be more easily and safely made in the future. > > The change log is here: http://svn.zope.org/zc.buildout/branches/gary-betafix/CHANGES.txt?view=markup > > Gary > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From agroszer at gmail.com Mon Aug 23 18:46:14 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Mon, 23 Aug 2010 18:46:14 +0200 Subject: [Distutils] why this version spec does not work? Message-ID: <71118391.20100823184614@gmail.com> Hello, I have here this version spec in setup.py: 'zope.testing >=3.6.0,<3.10.0', but zc.buildout does not get it right. Source is here: svn://svn.zope.org/repos/main/zc.recipe.testrunner/branches/1.2.1 Output is here: http://paste.lisp.org/+2FSQ Why on earth does zc.buildout pick zope.testing = 3.10.0 with the above spec?? -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: There is no true and abiding morality that is not founded in religion. - Henry Ward Beecher From sridharr at activestate.com Mon Aug 23 19:44:44 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Mon, 23 Aug 2010 10:44:44 -0700 Subject: [Distutils] 'setup.py develop' w/ extras? Message-ID: I use the 'develop' command instead of 'install' as it does not copy the source files to site-packages, and I can continue developing on the working copy .. thanks to .egg-link. My project also has 'extras' dependencies. Is there a way to get these dependencies installing automatically with the develop command? None of the options that 'setup.py develop' takes seem to be applicable to the extra dependencies. -srid From pje at telecommunity.com Mon Aug 23 19:44:42 2010 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 23 Aug 2010 13:44:42 -0400 Subject: [Distutils] why this version spec does not work? In-Reply-To: <71118391.20100823184614@gmail.com> References: <71118391.20100823184614@gmail.com> Message-ID: <20100823174450.60AD93A4108@sparrow.telecommunity.com> At 06:46 PM 8/23/2010 +0200, Adam GROSZER wrote: >Hello, > >I have here this version spec in setup.py: >'zope.testing >=3.6.0,<3.10.0', > >but zc.buildout does not get it right. >Source is here: >svn://svn.zope.org/repos/main/zc.recipe.testrunner/branches/1.2.1 >Output is here: >http://paste.lisp.org/+2FSQ > >Why on earth does zc.buildout pick zope.testing = 3.10.0 with the >above spec?? That's really odd; it must be something in buildout, as setuptools' Requirement objects seem to do the right thing: >>> from pkg_resources import Requirement, parse_version >>> r=Requirement.parse('zope.testing >=3.6.0,<3.10.0') >>> '3.6' in r True >>> '3.10' in r False >>> '3.10.0' in r False From pje at telecommunity.com Mon Aug 23 20:29:02 2010 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 23 Aug 2010 14:29:02 -0400 Subject: [Distutils] 'setup.py develop' w/ extras? In-Reply-To: References: Message-ID: <20100823182913.962963A4108@sparrow.telecommunity.com> At 10:44 AM 8/23/2010 -0700, Sridhar Ratnakumar wrote: >I use the 'develop' command instead of 'install' as it does not copy >the source files to site-packages, and I can continue developing on >the working copy .. thanks to .egg-link. My project also has >'extras' dependencies. Is there a way to get these dependencies >installing automatically with the develop command? You could add 'MyProject[extra1,...]' to your install_requires, but you'd need to remember to take them back *out* later. ;-) >None of the options that 'setup.py develop' takes seem to be >applicable to the extra dependencies. I suppose I could add an --extras option in 0.7. Patches welcome. ;-) From jim at zope.com Mon Aug 23 20:48:16 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 23 Aug 2010 14:48:16 -0400 Subject: [Distutils] why this version spec does not work? In-Reply-To: <71118391.20100823184614@gmail.com> References: <71118391.20100823184614@gmail.com> Message-ID: On Mon, Aug 23, 2010 at 12:46 PM, Adam GROSZER wrote: > Hello, > > I have here this version spec in setup.py: > 'zope.testing >=3.6.0,<3.10.0', > > but zc.buildout does not get it right. > Source is here: > svn://svn.zope.org/repos/main/zc.recipe.testrunner/branches/1.2.1 > Output is here: > http://paste.lisp.org/+2FSQ > > Why on earth does zc.buildout pick zope.testing = 3.10.0 with the > above spec?? If you look earlier in the output you provide a link to, buildout does get it right. It picks 3.9.5. Then, when zc.recipe.testrunner runs, it tries to load zope.testing explicitly: eggs, ws = self.egg.working_set(('zope.testing', )) It doesn't restrict the version here. This didn't cause a problem until you added the restriction in in setup.py. Anyone not interested in the implementation of zc.recipe.testrunner should stop reading here, if you haven't already. :) self.egg implies zc.recipe.testrunner, so the low-level buildout APIs are asked to create a working set for zc.recipe.testrunner and zope.testing. It happens to load packages in a breadth first order and gets the newest version of zope.testing before it tries to get the restricted dependency of zc.recipe.testrunner. The easiest way to fix this is to remove the redundant and now incorrect reference to zope.testing in the call to self.egg.working_set: eggs, ws = self.egg.working_set() Jim -- Jim Fulton From attilaolah at gmail.com Tue Aug 24 00:38:16 2010 From: attilaolah at gmail.com (=?UTF-8?B?QXR0aWxhIE9sw6Fo?=) Date: Tue, 24 Aug 2010 00:38:16 +0200 Subject: [Distutils] Buildout 1.5.0 and friends released (was Re: Buildout release news) In-Reply-To: References: Message-ID: Hello, On Mon, Aug 23, 2010 at 17:43, Gary Poster wrote: > I have merged the betafix branch to trunk and made the releases of zc.buildout 1.5.0, zc.recipe.egg 1.3.0, and z3c.recipe.scripts 1.0.0. Here are some observations. 1. I had a line like this in my ~/.bashrc: export PYTHONPATH="/home/aatiis/local/lib/python2.6/site-packages/" This was causing a "RuntimeError: maximum recursion depth exceeded" when running "python bootstrap.py", with Python 2.[4567]. So I had to remove this line; however, this issue seems to be related to the new bootstrap.py file only, coming from http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py 2. The recipe called "djangorecipe" has a weird issue. The cause seems to be that it uses ``zc.buildout.easy_install.scripts`` instead of ``zc.buildout.easy_install.sitepackage_safe_scripts``. The generated scripts have the following issue: Say I have PIL installed globally, and the generated script will have the global one's project root directory added to sys.path, instead of the one in the "eggs" directory (there *is* a PIL in the "eggs" dir, though). Now, my "./bin/django" looks like this: #!/usr/bin/python2.6 import sys sys.path[0:0] = [ ... '/usr/lib64/python2.6/site-packages/PIL', ... ] ... Why is the "PIL" added to sys.path? Regards, Attila From sdouche at gmail.com Tue Aug 24 01:13:22 2010 From: sdouche at gmail.com (Sebastien Douche) Date: Tue, 24 Aug 2010 01:13:22 +0200 Subject: [Distutils] Buildout 1.5.0 and friends released (was Re: Buildout release news) In-Reply-To: References: Message-ID: On Mon, Aug 23, 2010 at 17:43, Gary Poster wrote: > I have merged the betafix branch to trunk and made the releases of zc.buildout 1.5.0, zc.recipe.egg 1.3.0, and z3c.recipe.scripts 1.0.0. Awesome! For the next release, can you add a note on each new functionality in the documentation (something like "new in 1.5") ? It's hard to see the changes. Thanks Gary for the release. -- Sebastien Douche Twitter: http://bit.ly/afkrK (agile, lean, python, open source) From gary.poster at canonical.com Tue Aug 24 03:01:35 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 23 Aug 2010 21:01:35 -0400 Subject: [Distutils] Buildout 1.5.0 and friends released (was Re: Buildout release news) In-Reply-To: References: Message-ID: On Aug 23, 2010, at 6:38 PM, Attila Ol?h wrote: > Hello, > > On Mon, Aug 23, 2010 at 17:43, Gary Poster wrote: >> I have merged the betafix branch to trunk and made the releases of zc.buildout 1.5.0, zc.recipe.egg 1.3.0, and z3c.recipe.scripts 1.0.0. > > Here are some observations. > > 1. I had a line like this in my ~/.bashrc: > > export PYTHONPATH="/home/aatiis/local/lib/python2.6/site-packages/" > > This was causing a "RuntimeError: maximum recursion depth exceeded" > when running "python bootstrap.py", with Python 2.[4567]. So I had to > remove this line; however, this issue seems to be related to the new > bootstrap.py file only, coming from > http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py Huh, that's interesting. I was not able to dupe (I tried ``env PYTHONPATH=/home/gary/py/lib/python2.6/site-packages 26python bootstrap.py``, where 26python is a local Python). Could you give me a traceback? (Off-list, perhaps.) > > 2. The recipe called "djangorecipe" has a weird issue. The cause seems > to be that it uses ``zc.buildout.easy_install.scripts`` instead of > ``zc.buildout.easy_install.sitepackage_safe_scripts``. Yeah. That's one we use too, and I intend to upgrade it eventually if no-one beats me to it. > he generated > scripts have the following issue: Say I have PIL installed globally, > and the generated script will have the global one's project root > directory added to sys.path, instead of the one in the "eggs" > directory (there *is* a PIL in the "eggs" dir, though). Now, my > "./bin/django" looks like this: > > #!/usr/bin/python2.6 > > import sys > sys.path[0:0] = [ > ... > '/usr/lib64/python2.6/site-packages/PIL', > ... > ] > ... > > Why is the "PIL" added to sys.path? As you and I said, it's the old code path, so all bets are off with a system Python. Wanna upgrade djangorecipe for me? :-) Anyway, in this case, I believe that particular spelling of the path has to do with how PIL is often installed. Gary From reinout at vanrees.org Tue Aug 24 13:33:55 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Tue, 24 Aug 2010 13:33:55 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: On 08/20/2010 05:22 PM, Gary Poster wrote: > I intend to release zc.buildout 1.5.0 Monday, August 23. I used http://pypi.python.org/pypi/osc.recipe.sysegg/ to reliably grab specific system python packages (PIL, matplotlib, etc). That one is broken with 1.5.0, as it cannot find PIL and friends anymore. Using the http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py helps getting everything working (at least temporary). osc.recipe.sysegg's problem is that a pkg_resources.require(egg) call doesn't find any of the system eggs anymore. Am I correct that that's one of buildout's changes? That the recipes are really totally isolated? One thing I don't understand is why all my buildouts suddenly started failing: I've pinned zc.buildout in all my buildouts. Did the couple-months-old bootstrap.py (so: no svn:externals!) somehow manage to pull in a new buildout despite the version pin? I had to go through a couple of projects to grab the 1.4 bootstrap. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From gary.poster at canonical.com Tue Aug 24 16:01:32 2010 From: gary.poster at canonical.com (Gary Poster) Date: Tue, 24 Aug 2010 10:01:32 -0400 Subject: [Distutils] Buildout 1.5.0 and friends released (was Re: Buildout release news) In-Reply-To: References: Message-ID: On Aug 23, 2010, at 7:13 PM, Sebastien Douche wrote: > On Mon, Aug 23, 2010 at 17:43, Gary Poster wrote: >> I have merged the betafix branch to trunk and made the releases of zc.buildout 1.5.0, zc.recipe.egg 1.3.0, and z3c.recipe.scripts 1.0.0. > > Awesome! For the next release, can you add a note on each new > functionality in the documentation (something like "new in 1.5") ? > It's hard to see the changes. ...maybe... :-) What's the goal? I think that the main things end users ought to know about are these. (1) How does the system Python support work? See http://pypi.python.org/pypi/zc.buildout/1.5.0#system-python-and-zc-buildout-1-5 . It links out to http://pypi.python.org/pypi/z3c.recipe.scripts . (If you really want to get into the nitty gritty you can see http://pypi.python.org/pypi/zc.buildout/1.5.0#the-sitepackage-safe-scripts-function and the following two sections, but I'm not sure it's worth highlighting. That's maintainer-y information and in my perfect world would not be on the main end-user PyPI page. Sorry, I didn't work on improving the docs. :-) ) (2) How do you get alpha/beta releases of zc.buildout and friends in the future? You can use python bootstrap.py -t or specify a specific version, as mentioned in the CHANGES (http://pypi.python.org/pypi/zc.buildout/1.5.0#id9). I figure that's something that will be emphasized the next time someone wants help testing a beta release, and it doesn't need too much discussion now. (3) Can you use Python 2.7? Yes, see CHANGES (http://pypi.python.org/pypi/zc.buildout/1.5.0#id9). I did other stuff, but it was mostly internal bits related to those things. Well...maybe mentioning the new bootstrap features would be nice (they are mentioned in CHANGES), and mentioning the new output when running buildout with -v that shows you what packages came from site-packages, if any (I forgot to talk about that, though it is tested). Anyway... > Thanks Gary for the release. Thanks for the thanks. Gary From m.van.rees at zestsoftware.nl Tue Aug 24 16:29:28 2010 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Tue, 24 Aug 2010 16:29:28 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: Op 24-08-10 13:33, Reinout van Rees schreef: > On 08/20/2010 05:22 PM, Gary Poster wrote: >> I intend to release zc.buildout 1.5.0 Monday, August 23. > > I used http://pypi.python.org/pypi/osc.recipe.sysegg/ to reliably grab > specific system python packages (PIL, matplotlib, etc). That one is > broken with 1.5.0, as it cannot find PIL and friends anymore. > > Using the > http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py > helps getting everything working (at least temporary). > > osc.recipe.sysegg's problem is that a pkg_resources.require(egg) call > doesn't find any of the system eggs anymore. Am I correct that that's > one of buildout's changes? That the recipes are really totally isolated? > > > One thing I don't understand is why all my buildouts suddenly started > failing: I've pinned zc.buildout in all my buildouts. Did the > couple-months-old bootstrap.py (so: no svn:externals!) somehow manage to > pull in a new buildout despite the version pin? > > I had to go through a couple of projects to grab the 1.4 bootstrap. If I understand correctly, one of the new things that the new bootstrap.py does, is to actually look at the version pinning for zc.buildout (and probably setuptools/distribute) in buildout.cfg and use that for creating the initial bin/buildout script. So if you are using an old bootstrap.py, I would indeed expect that you get the latest zc.buildout version. Errr... no, I have now tried it with a few different bootstrap.py versions, and after 'python2.4 bootstrap.py' I always end up with the zc.buildout that is pinned in the versions. That is not how I remember it, though this suits me fine. :-) Maybe there is a difference here between using distribute (like I do) or setuptools.? What I *do* see is that when you start with zc.buildout 1.5.0, then change the pin to 1.4.3 and rerun the buildout, this fails when you are using a buildout extension. The other way around (first 1.4.3, then change it to 1.5.0) works fine. I tried it with buildout-versions and with mr.developer as extensions, both having the same result. For example, start with this, and any bootstrap.py: =================================================== [buildout] extensions = buildout-versions buildout_versions_file = buildout.cfg parts = pep8 versions = versions [pep8] recipe = zc.recipe.egg eggs = pep8 [versions] buildout-versions = 1.3 distribute = 0.6.14 pep8 = 0.5.0 zc.buildout = 1.5.0 zc.recipe.egg = 1.2.2 =================================================== Note that I pinned zc.recipe.egg to 1.2.2 as 1.3.2 conflicts with zc.buildout 1.4.3, which I want to try next. Run python2.4 bootstrap.py (probably same effect with python2.anything) and then bin/buildout. Works fine. Now change the zc.buildout pin to 1.4.3 and rerun bin/buildout: $ bin/buildout While: Installing. Loading extensions. Error: There is a version conflict. We already have: zc.buildout 1.5.0 With more debugging: $ bin/buildout -vvvvvvvvvvvvvvvvvvvvD Installing 'buildout-versions'. We have the distribution that satisfies 'buildout-versions==1.3'. While: Installing. Loading extensions. Traceback (most recent call last): File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 1784, in main File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 410, in install File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 960, in _load_extensions File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 1085, in install File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 942, in install VersionConflict: There is a version conflict. We already have: zc.buildout 1.5.0 Starting pdb: > .../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py(942)install() (Pdb) a self = specs = ['buildout-versions'] working_set = The workaround is of course to remove bin/buildout, do python2.4 bootstrap.py and run bin/buildout again. So probably no big deal, but a bit unexpected anyway. BTW, this is on MacOSX 10.6.4, with a python compiled using this buildout: http://svn.plone.org/svn/collective/buildout/python I am normally using virtualenv created by this python, with a few more modules installed, but I tried with a fresh virtualenv and with using the python executable directly as well, with the same results. Anyway, I have not seen things go really wrong yet today, and my buildout-using colleagues have not screamed either, so the troubles of the previous beta seem to have been avoided. Thanks for your hard work, Gary! -- Maurits van Rees Programmer, Zest Software From agroszer at gmail.com Tue Aug 24 16:31:35 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Tue, 24 Aug 2010 16:31:35 +0200 Subject: [Distutils] why this version spec does not work? In-Reply-To: References: <71118391.20100823184614@gmail.com> Message-ID: <413207015.20100824163135@gmail.com> Hello Jim, Monday, August 23, 2010, 8:48:16 PM, you wrote: JF> On Mon, Aug 23, 2010 at 12:46 PM, Adam GROSZER wrote: >> Hello, >> >> I have here this version spec in setup.py: >> 'zope.testing >=3.6.0,<3.10.0', >> >> but zc.buildout does not get it right. >> Source is here: >> svn://svn.zope.org/repos/main/zc.recipe.testrunner/branches/1.2.1 >> Output is here: >> http://paste.lisp.org/+2FSQ >> >> Why on earth does zc.buildout pick zope.testing = 3.10.0 with the >> above spec?? JF> If you look earlier in the output you provide a link to, buildout does JF> get it right. It picks 3.9.5. JF> Then, when zc.recipe.testrunner runs, it tries to load zope.testing JF> explicitly: JF> eggs, ws = self.egg.working_set(('zope.testing', )) JF> It doesn't restrict the version here. This didn't cause a problem JF> until you added the restriction in in setup.py. JF> Anyone not interested in the implementation of zc.recipe.testrunner should JF> stop reading here, if you haven't already. :) JF> self.egg implies zc.recipe.testrunner, so the low-level buildout APIs JF> are asked to create a working set for zc.recipe.testrunner and JF> zope.testing. It happens to load packages in a breadth first order JF> and gets the newest version of zope.testing before it tries to get the JF> restricted dependency of zc.recipe.testrunner. JF> The easiest way to fix this is to remove the redundant and now JF> incorrect reference to zope.testing in the call to JF> self.egg.working_set: JF> eggs, ws = self.egg.working_set() Thanks for the pointer, but in my case it seems to work only with the version spec added: eggs, ws = self.egg.working_set(('zope.testing <3.10.0', )) If I do eggs, ws = self.egg.working_set() zope.testing seems to be missed from the test script entirely. -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: God hears no sweeter music than the cracked chimes of the courageous human spirit ringing in imperfect acknowledgement of His perfect love. - Joshua Loth Liebman From jim at zope.com Tue Aug 24 17:29:56 2010 From: jim at zope.com (Jim Fulton) Date: Tue, 24 Aug 2010 11:29:56 -0400 Subject: [Distutils] why this version spec does not work? In-Reply-To: <413207015.20100824163135@gmail.com> References: <71118391.20100823184614@gmail.com> <413207015.20100824163135@gmail.com> Message-ID: On Tue, Aug 24, 2010 at 10:31 AM, Adam GROSZER wrote: > Hello Jim, > > Monday, August 23, 2010, 8:48:16 PM, you wrote: > > JF> On Mon, Aug 23, 2010 at 12:46 PM, Adam GROSZER wrote: >>> Hello, >>> >>> I have here this version spec in setup.py: >>> 'zope.testing >=3.6.0,<3.10.0', >>> >>> but zc.buildout does not get it right. >>> Source is here: >>> svn://svn.zope.org/repos/main/zc.recipe.testrunner/branches/1.2.1 >>> Output is here: >>> http://paste.lisp.org/+2FSQ >>> >>> Why on earth does zc.buildout pick zope.testing = 3.10.0 with the >>> above spec?? > > JF> If you look earlier in the output you provide a link to, buildout does > JF> get it right. ?It picks 3.9.5. > > JF> Then, when zc.recipe.testrunner runs, it tries to load zope.testing > JF> explicitly: > > JF> ? eggs, ws = self.egg.working_set(('zope.testing', )) > > JF> It doesn't restrict the version here. ?This didn't cause a problem > JF> until you added the restriction in in setup.py. > > JF> Anyone not interested in the implementation of zc.recipe.testrunner should > JF> stop reading here, if you haven't already. :) > > JF> self.egg implies zc.recipe.testrunner, so the low-level buildout APIs > JF> are asked to create a working set for zc.recipe.testrunner and > JF> zope.testing. ?It happens to load packages in a breadth first order > JF> and gets the newest version of zope.testing before it tries to get the > JF> restricted dependency of zc.recipe.testrunner. > > JF> The easiest way to fix this is to remove the redundant and now > JF> incorrect reference to zope.testing in the call to > JF> self.egg.working_set: > > JF> ? eggs, ws = self.egg.working_set() > > Thanks for the pointer, but in my case it seems to work only with the > version spec added: > > eggs, ws = self.egg.working_set(('zope.testing <3.10.0', )) > > If I do > > eggs, ws = self.egg.working_set() > > zope.testing seems to be missed from the test script entirely. Ah, that makes sense when you're testing something other than zc.recipe.testrunner itself. Jim -- Jim Fulton From gary.poster at canonical.com Tue Aug 24 17:47:36 2010 From: gary.poster at canonical.com (Gary Poster) Date: Tue, 24 Aug 2010 11:47:36 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: On Aug 24, 2010, at 10:29 AM, Maurits van Rees wrote: > Op 24-08-10 13:33, Reinout van Rees schreef: >> On 08/20/2010 05:22 PM, Gary Poster wrote: >>> I intend to release zc.buildout 1.5.0 Monday, August 23. >> >> I used http://pypi.python.org/pypi/osc.recipe.sysegg/ to reliably grab >> specific system python packages (PIL, matplotlib, etc). That one is >> broken with 1.5.0, as it cannot find PIL and friends anymore. >> >> Using the >> http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py >> helps getting everything working (at least temporary). >> >> osc.recipe.sysegg's problem is that a pkg_resources.require(egg) call >> doesn't find any of the system eggs anymore. Am I correct that that's >> one of buildout's changes? That the recipes are really totally isolated? Yes. Not isolating the recipes makes the process with a system Python very unreliable in my experience. ...maybe we could make it so that recipes could explicitly say that they wanted access to site-packages. I haven't really thought about it, and if we did that, I certainly would not want to support those sort of recipes further (with system Pythons). It would be an "at your own risk" sort of thing. I looked briefly at the recipe. AFAICT, it would not work with ``virtualenv --no-site-packages`` except for eggs installed in the virtualenv itself (and then it ought to work fine for those, because the new buildout stuff just tries to get out of the way when it encounters the broken -S behavior of ``virtualenv --no-site-packages``). Therefore, the new buildout functionality should make this recipe unnecessary, and should be more robust in the face of system Python oddities, with automated tests, IIUC (see include-site-packages and allowed-eggs-from-site-packages from http://pypi.python.org/pypi/z3c.recipe.scripts). How would you like to proceed? 1) We talk about some way of making it so that this kind of recipe can have a "get out of my way" flag to access site-packages. 2) We talk about a way to no longer use this recipe. 3) We do something else you propose. >> One thing I don't understand is why all my buildouts suddenly started >> failing: I've pinned zc.buildout in all my buildouts. Did the >> couple-months-old bootstrap.py (so: no svn:externals!) somehow manage to >> pull in a new buildout despite the version pin? In part. Before the 1.5.0 version, bootstrap would always get the newest buildout, no matter what (unless you used a --version argument to bootstrap). Then that version of buildout would be responsible for changing to the desired version. Now, by default, it will get the newest buildout that is final. >> >> I had to go through a couple of projects to grab the 1.4 bootstrap. > > If I understand correctly, one of the new things that the new bootstrap.py does, is to actually look at the version pinning for zc.buildout (and probably setuptools/distribute) in buildout.cfg and use that for creating the initial bin/buildout script. Close but not quite. That might be desirable--I considered it--but would have required making yet more of buildout's functionality go into bootstrap. Bootstrap would have needed to duplicate just enough of the extended-config-parsing behavior of buildout to do the right thing. That duplication of policy-type code did not seem like a good choice to me. As above, instead, by default, the new bootstrap gets the newest buildout that is final. > So if you are using an old bootstrap.py, I would indeed expect that you get the latest zc.buildout version. > > Errr... no, I have now tried it with a few different bootstrap.py versions, and after 'python2.4 bootstrap.py' I always end up with the zc.buildout that is pinned in the versions. That is not how I remember it, though this suits me fine. :-) Maybe there is a difference here between using distribute (like I do) or setuptools.? Right, pre- and post-1.5.0 you are supposed to *end up* with the desired buildout, irrespective of distribute vs. setuptools. (The mechanism, via bootstrap, might have been a bit indirect. In particular, writing bin/buildout is currently the responsibility of the *original* buildout downloaded by bootstrap. Changing that behavior, and the similar problem behavior when upgrading, was something I punted on.) > What I *do* see is that when you start with zc.buildout 1.5.0, then change the pin to 1.4.3 and rerun the buildout, this fails when you are using a buildout extension. The other way around (first 1.4.3, then change it to 1.5.0) works fine. I tried it with buildout-versions and with mr.developer as extensions, both having the same result. For example, start with this, and any bootstrap.py: > > =================================================== > [buildout] > extensions = buildout-versions > buildout_versions_file = buildout.cfg > parts = pep8 > versions = versions > > [pep8] > recipe = zc.recipe.egg > eggs = pep8 > > [versions] > buildout-versions = 1.3 > distribute = 0.6.14 > pep8 = 0.5.0 > zc.buildout = 1.5.0 > zc.recipe.egg = 1.2.2 (I didn't really expect 1.2.2 + 1.5.0 to work, FWIW...zc.buildout 1.5.0 wants zc.recipe.egg 1.3.0 or higher. There's no way for setup.py to spell "use this version or higher if you you use it at all" AFAIK though. I guess you report it kinda works though...) > =================================================== > > Note that I pinned zc.recipe.egg to 1.2.2 as 1.3.2 conflicts with zc.buildout 1.4.3, which I want to try next. (FWIW, zc.buildout 1.4.4 will automatically do the (over-ridable) pinning of some basic recipes including zc.recipe.egg.) > Run python2.4 bootstrap.py (probably same effect with python2.anything) and then bin/buildout. Works fine. Now change the zc.buildout pin to 1.4.3 and rerun bin/buildout: > > $ bin/buildout > While: > Installing. > Loading extensions. > Error: There is a version conflict. > We already have: zc.buildout 1.5.0 > > With more debugging: > > $ bin/buildout -vvvvvvvvvvvvvvvvvvvvD > Installing 'buildout-versions'. > We have the distribution that satisfies 'buildout-versions==1.3'. > While: > Installing. > Loading extensions. > Traceback (most recent call last): > File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 1784, in main > File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 410, in install > File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 960, in _load_extensions > File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 1085, in install > File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 942, in install > VersionConflict: There is a version conflict. > We already have: zc.buildout 1.5.0 > Starting pdb: > > .../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py(942)install() > (Pdb) a > self = > specs = ['buildout-versions'] > working_set = > > > The workaround is of course to remove bin/buildout, do python2.4 bootstrap.py and run bin/buildout again. So probably no big deal, but a bit unexpected anyway. OK, fair enough. Thanks for the report. I'm inclined to agree with you that it is not a big deal. It sounds like you need a resolution to the osc.recipe.sysegg problem more? > BTW, this is on MacOSX 10.6.4, with a python compiled using this buildout: http://svn.plone.org/svn/collective/buildout/python > I am normally using virtualenv created by this python, with a few more modules installed, but I tried with a fresh virtualenv and with using the python executable directly as well, with the same results. > > Anyway, I have not seen things go really wrong yet today, and my buildout-using colleagues have not screamed either, so the troubles of the previous beta seem to have been avoided. Cool. > Thanks for your hard work, Gary! Welcome. Gary From agroszer at gmail.com Tue Aug 24 18:19:32 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Tue, 24 Aug 2010 18:19:32 +0200 Subject: [Distutils] the spec problem continues Message-ID: <1371268878.20100824181932@gmail.com> Hello, I got zc.recipe.testrunner tests pass. Now after I added zc.recipe.testrunner 1.2.1 to ZTK trunk, I have the following problems after running "bin\buildout -t 2 -c development.cfg": (This is MS windows!) could't load zc.buildout entry point default from z3c.recipe.compattest: Picked: zope.testing = 3.9.5. While: Installing. Getting section test-ztk. Initializing section test-ztk. Installing recipe z3c.recipe.compattest. Getting distribution for 'zope.testing>=3.6.0,<3.10.0'. Error: Picked: zope.testing = 3.9.5 Complete output is here: http://pastebin.org/760371 Source is here: svn://svn.zope.org/repos/main/zopetoolkit/trunk What's going on? buildout has "allow-picked-versions = false" and "zope.testing = 3.9.5" so why "Error: Picked: zope.testing = 3.9.5" ??? -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: We find freedom when we find God; we lose it when we lose Him. - Paul E. Scherer From setuptools at bugs.python.org Tue Aug 24 21:54:19 2010 From: setuptools at bugs.python.org (AndiDog) Date: Tue, 24 Aug 2010 19:54:19 +0000 Subject: [Distutils] [issue112] Shebang for gui_scripts on Windows should be #!pythonw.exe for MSI installers, too In-Reply-To: <1282679659.2.0.442419409714.issue112@psf.upfronthosting.co.za> Message-ID: <1282679659.2.0.442419409714.issue112@psf.upfronthosting.co.za> New submission from AndiDog : This is a follow up to the following problem: http://stackoverflow.com/questions/3542119/create-launchable-gui-script-from-python-setuptools-without-console-window/3542410#3542410 If I create a MSI installer from a Python package with a computer that has its Python installation in 'C:\Program Files\Python26', and install it on another computer that has it installed in 'C:\Python26', for example, the shebang (for a script in gui_scripts) will still be #!"C:\Program Files\Python26\pythonw.exe" With bdist_wininst installers, this problem does not occur because the code makes a distinction for them: if wininst: executable = "python.exe" else: executable = nt_quote_arg(executable) In case of a MSI installer, wininst is False and the full absolute path of pythonw.exe is inserted. I can't understand why an absolute path should be necessary. Using a #!pythonw.exe shebang line works even if the matching Python installation is not in the PATH or an incompatible Python version is in the PATH. So I propose to change that first line to if wininst or sys.platform == "win32": ---------- messages: 535 nosy: AndiDog priority: bug status: unread title: Shebang for gui_scripts on Windows should be #!pythonw.exe for MSI installers, too _______________________________________________ Setuptools tracker _______________________________________________ From attilaolah at gmail.com Wed Aug 25 06:26:13 2010 From: attilaolah at gmail.com (=?UTF-8?B?QXR0aWxhIE9sw6Fo?=) Date: Wed, 25 Aug 2010 06:26:13 +0200 Subject: [Distutils] bootstrap.py/ez_setup doesn't understand username:passwd in URLs Message-ID: Hello, Here's my traceback: $ python bootstrap.py --distribute --download-base=https://username:password at pypi.example.com/ Downloading https://username:password at pypi.example.com/distribute-0.6.14.tar.gz Traceback (most recent call last): File "bootstrap.py", line 171, in ez['use_setuptools'](**setup_args) File "", line 145, in use_setuptools File "", line 124, in _do_download File "", line 193, in download_setuptools File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib64/python2.6/urllib2.py", line 391, in open response = self._open(req, data) File "/usr/lib64/python2.6/urllib2.py", line 409, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py", line 1178, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib64/python2.6/urllib2.py", line 1116, in do_open h = http_class(host, timeout=req.timeout) # will parse host:port File "/usr/lib64/python2.6/httplib.py", line 1105, in __init__ HTTPConnection.__init__(self, host, port, strict, timeout) File "/usr/lib64/python2.6/httplib.py", line 661, in __init__ self._set_hostport(host, port) File "/usr/lib64/python2.6/httplib.py", line 686, in _set_hostport raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) httplib.InvalidURL: nonnumeric port: 'password at pypi.example.com' Since URLs like "https://username:password at pypi.example.com/distribute-0.6.14.tar.gz" seem to be handled well by *urllib*, but not by *urllib2*, I wonder why are they handled by urllib2 instead of urllib. I know it doesn't make sense to put a password on the site that hosts distribute/setuptools, but we have a private PyPI and we'd like to use that as --download-base too. Attila From attilaolah at gmail.com Wed Aug 25 06:29:22 2010 From: attilaolah at gmail.com (=?UTF-8?B?QXR0aWxhIE9sw6Fo?=) Date: Wed, 25 Aug 2010 06:29:22 +0200 Subject: [Distutils] bootstrap.py/ez_setup doesn't understand username:passwd in URLs In-Reply-To: References: Message-ID: On Wed, Aug 25, 2010 at 06:26, Attila Ol?h wrote: > Since URLs like > "https://username:password at pypi.example.com/distribute-0.6.14.tar.gz" > seem to be handled well by *urllib*, but not by *urllib2*, I wonder > why are they handled by urllib2 instead of urllib. Oops, I should have checked better before making the above statement. Actually neither of urllib or urllib2 can handle these types of URLs. However, it would be handy if bootstrap.py or ez_setup.py would be smart enough to handle these edge-cases. Attila From attilaolah at gmail.com Wed Aug 25 06:42:03 2010 From: attilaolah at gmail.com (=?UTF-8?B?QXR0aWxhIE9sw6Fo?=) Date: Wed, 25 Aug 2010 06:42:03 +0200 Subject: [Distutils] How to force bootstrap.py to download setuptools/distribute? Message-ID: Hello, Is it possible to force bootstrap.py to download the setuptools or distribute egg even though it can be found on the system? Not downloading a fresh copy will cause issues in some edge cases, for example take the following situation: Say my site-packages are located in /usr/lib/python2.6/site-packages/. Then, I set my PYTHONPATH to contain /usr/local/lib/python2.6/site-packages/ (note *local*). Then, I install distribute or setuptools into /usr/local/lib/python2.6/site-packages/. Then, I install a third party package, say lxml, also to /usr/local/lib/python2.6/site-packages/. Then, in my project, I run python bootstrap.py (with or without the --distribute flag, depending on which package I have in /usr/local/lib/python2.6/site-packages/). Then, bootstrap.py will not download a fresh copy, instead it will add /usr/local/lib/python2.6/site-packages/ to sys.path so that it can use the existing one. Then, say I have the following buildout.cfg: buildout] develop = . parts = eggs [eggs] zc.recipe.egg include-site-packages = false allowed-eggs-from-site-packages = eggs = lxml I would now expect buildout to not include lxml from my PYTHONPATH, but build it instead and put it in my eggs-dir. However, it's not the case: adding /usr/local/lib/python2.6/site-packages/ to sys.path will make lxml available so buildout will not build it. My question is, how can I force buildout to build its own lxml, or better yet, force it to download a fresh setuptools or distribute egg, avoiding adding extra stuff to sys.path, thus making unnecessary system packages available and, thus, not putting them in the eggs-dir. Note, this is zc.buildout 1.5.0. I hope I gave enough details. Attila From agroszer at gmail.com Wed Aug 25 08:54:28 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Wed, 25 Aug 2010 08:54:28 +0200 Subject: [Distutils] the spec problem continues In-Reply-To: <1371268878.20100824181932@gmail.com> References: <1371268878.20100824181932@gmail.com> Message-ID: <524738539.20100825085428@gmail.com> Hello, Tuesday, August 24, 2010, 6:19:32 PM, you wrote: AG> Hello, AG> I got zc.recipe.testrunner tests pass. AG> Now after I added zc.recipe.testrunner 1.2.1 to ZTK trunk, I have the AG> following problems after running AG> "bin\buildout -t 2 -c development.cfg": AG> (This is MS windows!) AG> could't load zc.buildout entry point default AG> from z3c.recipe.compattest: AG> Picked: zope.testing = 3.9.5. AG> While: AG> Installing. AG> Getting section test-ztk. AG> Initializing section test-ztk. AG> Installing recipe z3c.recipe.compattest. AG> Getting distribution for 'zope.testing>=3.6.0,<3.10.0'. AG> Error: Picked: zope.testing = 3.9.5 AG> Complete output is here: AG> http://pastebin.org/760371 AG> Source is here: AG> svn://svn.zope.org/repos/main/zopetoolkit/trunk AG> What's going on? AG> buildout has "allow-picked-versions = false" and "zope.testing = AG> 3.9.5" so why "Error: Picked: zope.testing = 3.9.5" ??? Weird is that the same seems to work on linux: http://buildbot.afpy.org/ztk1.0/builders/Python2.6.5%20Linux%2064bit/builds/11/steps/buildout/logs/stdio -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: The world we inhabit must have had an origin; that origin must have consisted in a cause; that cause must have been intelligent; that intelligence must have been supreme; and that supreme, which always was and is supreme, we know by the name of God. - Anonymous From m.van.rees at zestsoftware.nl Wed Aug 25 11:21:18 2010 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Wed, 25 Aug 2010 11:21:18 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: Op 24-08-10 17:47, Gary Poster schreef: > On Aug 24, 2010, at 10:29 AM, Maurits van Rees wrote: >> Op 24-08-10 13:33, Reinout van Rees schreef: >>> On 08/20/2010 05:22 PM, Gary Poster wrote: >>>> I intend to release zc.buildout 1.5.0 Monday, August 23. >>> >>> I used http://pypi.python.org/pypi/osc.recipe.sysegg/ to reliably grab >>> specific system python packages (PIL, matplotlib, etc). That one is >>> broken with 1.5.0, as it cannot find PIL and friends anymore. >>> >>> Using the >>> http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py >>> helps getting everything working (at least temporary). >>> >>> osc.recipe.sysegg's problem is that a pkg_resources.require(egg) call >>> doesn't find any of the system eggs anymore. Am I correct that that's >>> one of buildout's changes? That the recipes are really totally isolated? (...) >> $ bin/buildout >> While: >> Installing. >> Loading extensions. >> Error: There is a version conflict. >> We already have: zc.buildout 1.5.0 >> >> With more debugging: >> >> $ bin/buildout -vvvvvvvvvvvvvvvvvvvvD >> Installing 'buildout-versions'. >> We have the distribution that satisfies 'buildout-versions==1.3'. >> While: >> Installing. >> Loading extensions. >> Traceback (most recent call last): >> File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 1784, in main >> File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 410, in install >> File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/buildout.py", line 960, in _load_extensions >> File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 1085, in install >> File ".../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py", line 942, in install >> VersionConflict: There is a version conflict. >> We already have: zc.buildout 1.5.0 >> Starting pdb: >>> .../zc.buildout-1.5.0-py2.4.egg/zc/buildout/easy_install.py(942)install() >> (Pdb) a >> self = >> specs = ['buildout-versions'] >> working_set = >> >> >> The workaround is of course to remove bin/buildout, do python2.4 bootstrap.py and run bin/buildout again. So probably no big deal, but a bit unexpected anyway. > > OK, fair enough. Thanks for the report. > > I'm inclined to agree with you that it is not a big deal. It sounds like you need a resolution to the osc.recipe.sysegg problem more? Ah, you confuse me with my evil twin brother Reinout. :-) Well, okay, he is not evil and no twin, but he is my brother. The first part of the mail (about osc.recipe.sysegg) was from him, and I reacted to that in the second half with my own little problems. To me it sounds like osc.recipe.sysegg should not be necessary anymore when using zc.buildout 1.5.0 and zc.recipe.egg 1.3.2. -- Maurits van Rees Programmer, Zest Software From reinout at vanrees.org Wed Aug 25 11:50:38 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 25 Aug 2010 11:50:38 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: On 08/24/2010 05:47 PM, Gary Poster wrote: > How would you like to proceed? > > 1) We talk about some way of making it so that this kind of recipe can have a "get out of my way" flag to access site-packages. > 2) We talk about a way to no longer use this recipe. > 3) We do something else you propose. Buildout itself supporting system eggs reliably is of course the handiest option, so that would be option 2. It is perhaps handiest if I explain my usecase. I use buildout for my Django projects (with the django recipe being a big piece of the pie). Lots of geographical processing, so mapnik, matplotlib, numpy and a couple of other not-pure-python packages. Some of those are notoriously difficult/impossible to install with a simple "easy_install" or generic buildout dependency install. "aptitude install python-matplotlib" and friends (or the windows equivalent) is the only practical way to get it running. a) I *do* need the system python. b) I prefer listing all dependencies (so also the shitty-to-install ones) in my setup.py. So I need to be able to tell buildout to grab them from the system packages instead of shooting itself in the foot by trying to grab them from pypi. c) I prefer having buildout search explicitly for these specific dependencies and complain when it doesn't find them. Helps with missing libraries. Now, I looked at the new buildout 1.5 system packages options, but I wondered whether it would work fully. Only the script recipe seems to support it at the moment and it didn't look from the documentation that such a globally-found dependency would be valid for the entire buildout (and all the other parts). If I use that osc.recipe.sysegg and list that part first, it makes sure all other parts use the specified system eggs. Just a minute ago, I tried experimenting with a z3c.recipe.egg part mentioned as the first one, but I got an "ImportError: Entry point ('zc.buildout', 'default') not found". I haven't updated the bootstrap yet, so that might be the cause (though that'd be a bug). Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Wed Aug 25 11:52:53 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 25 Aug 2010 11:52:53 +0200 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: On 08/25/2010 11:21 AM, Maurits van Rees wrote: > > Ah, you confuse me with my evil twin brother Reinout. :-) Well, okay, he > is not evil and no twin, but he is my brother. The first part of the > mail (about osc.recipe.sysegg) was from him, and I reacted to that in > the second half with my own little problems. He he :-) Long live confusion :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" From pje at telecommunity.com Wed Aug 25 21:59:01 2010 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 25 Aug 2010 15:59:01 -0400 Subject: [Distutils] bootstrap.py/ez_setup doesn't understand username:passwd in URLs Message-ID: <20100825195908.BD47C3A40A4@sparrow.telecommunity.com> At 06:26 AM 8/25/2010 +0200, Attila Ol??h wrote: >I know it doesn't make sense to put a password on the site that hosts >distribute/setuptools, but we have a private PyPI and we'd like to use >that as --download-base too. The goal of ez_setup is to be small, so every feature added is a tradeoff. In this particular case, it seems like the simplest fix is to pass the URL of a non-password-protected setuptools to ez_setup, and then use your password-protected PyPI afterwards. From alex at foogod.com Thu Aug 26 20:59:11 2010 From: alex at foogod.com (Alex Stewart) Date: Thu, 26 Aug 2010 11:59:11 -0700 Subject: [Distutils] use2to3 and non-package files Message-ID: <20100826185911.GO17834@foogod.com> Hi there, I have a project I want to use with the 'use2to3' option of distribute to maintain a common codebase for both 2.x and 3.x.. (First of all, let me say thank you so much for adding this option to distribute.. it makes this sort of thing so much easier :) ) Anyway, everything works fine for the code in the package directory. The problem, however, is that I also have some directories containing unit tests, etc, which are not under the source dir (since I don't want them to be distributed), but do still need to be converted with 2to3 before they can be used (i.e. when doing 'python3 setup.py test') I poked around a bit and didn't see an obvious way to do this. Can anybody suggest the best way to go about implementing it? Thanks, -alex From chris at simplistix.co.uk Thu Aug 26 22:30:28 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 26 Aug 2010 21:30:28 +0100 Subject: [Distutils] any way to stop buildout/setuptools compiling things that aren't python? Message-ID: <4C76CEE4.9080407@simplistix.co.uk> Hi All, Some packages include things which aren't python, they may even have .py extensions though. Installing these using easy_install/buildout/pip/etc results in this kind of (harmless) ugliness: SyntaxError: ("'return' outside function", ('build/bdist.macosx-10.3-fat/egg/Products/ZSyncer/skins/zsyncer_skins/get_zsyncer_status_for_objects.py', 21, None, 'return statuses\n')) How can I make that not happen? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Thu Aug 26 23:33:42 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 26 Aug 2010 14:33:42 -0700 Subject: [Distutils] any way to stop buildout/setuptools compiling things that aren't python? In-Reply-To: <4C76CEE4.9080407@simplistix.co.uk> References: <4C76CEE4.9080407@simplistix.co.uk> Message-ID: On Thu, Aug 26, 2010 at 1:30 PM, Chris Withers wrote: > Hi All, > > Some packages include things which aren't python, they may even have .py > extensions though. > > Installing these using easy_install/buildout/pip/etc results in this kind of > (harmless) ugliness: > > SyntaxError: ("'return' outside function", > ('build/bdist.macosx-10.3-fat/egg/Products/ZSyncer/skins/zsyncer_skins/get_zsyncer_status_for_objects.py', > 21, None, 'return statuses\n')) > > How can I make that not happen? By preventing Distutils to do the bytecode compiling when it builds/install the package. (this was discussed lately as a possible change) The other option is to have the ZSyncer project treat these files as data files, so they don't get compiled > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > ? ? ? ? ? - http://www.simplistix.co.uk > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org From chris at simplistix.co.uk Thu Aug 26 23:35:34 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 26 Aug 2010 22:35:34 +0100 Subject: [Distutils] any way to stop buildout/setuptools compiling things that aren't python? In-Reply-To: References: <4C76CEE4.9080407@simplistix.co.uk> Message-ID: <4C76DE26.9070101@simplistix.co.uk> Tarek Ziad? wrote: > The other option is to have the ZSyncer project treat these files as > data files, so they > don't get compiled How do I do that? They're only being found by setuptools' include_package_data=True... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From martin at v.loewis.de Thu Aug 26 23:41:46 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Aug 2010 23:41:46 +0200 Subject: [Distutils] use2to3 and non-package files In-Reply-To: <20100826185911.GO17834@foogod.com> References: <20100826185911.GO17834@foogod.com> Message-ID: <4C76DF9A.1070405@v.loewis.de> > I poked around a bit and didn't see an obvious way to do this. Can > anybody suggest the best way to go about implementing it? distutils has the distutils.util.run_2to3 function, which makes it fairly straight-forward to invoke 2to3 on a set of files. On top of these, you have copydir_run_2to3, and Mixin2to3 (which allows writing commands that have 2to3 as a step). I recommend to copy the test suite into the build directory (next to the real build output), and run 2to3 on that copy (e.g. using the copydir version, which will check modification times). Changing the files in-place is typically inappropriate, as that would mess with your version control. See http://bitbucket.org/loewis/django-3k/src/tip/setup.py for an example, in particular build_tests. HTH, Martin From ziade.tarek at gmail.com Fri Aug 27 00:31:14 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 26 Aug 2010 15:31:14 -0700 Subject: [Distutils] any way to stop buildout/setuptools compiling things that aren't python? In-Reply-To: <4C76DE26.9070101@simplistix.co.uk> References: <4C76CEE4.9080407@simplistix.co.uk> <4C76DE26.9070101@simplistix.co.uk> Message-ID: On Thu, Aug 26, 2010 at 2:35 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> The other option is to have the ZSyncer project treat these files as >> data files, so they >> don't get compiled > > How do I do that? > > They're only being found by setuptools' include_package_data=True... http://docs.python.org/distutils/setupscript.html#installing-package-data > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > ? ? ? ? ? - http://www.simplistix.co.uk > -- Tarek Ziad? | http://ziade.org From gary.poster at canonical.com Fri Aug 27 02:31:12 2010 From: gary.poster at canonical.com (Gary Poster) Date: Thu, 26 Aug 2010 20:31:12 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> On Aug 25, 2010, at 5:50 AM, Reinout van Rees wrote: > On 08/24/2010 05:47 PM, Gary Poster wrote: >> How would you like to proceed? >> >> 1) We talk about some way of making it so that this kind of recipe can have a "get out of my way" flag to access site-packages. >> 2) We talk about a way to no longer use this recipe. >> 3) We do something else you propose. Sorry for not replying sooner. FWIW, including me directly in the distribution list will get faster replies. > > Buildout itself supporting system eggs reliably is of course the handiest option, so that would be option 2. > > It is perhaps handiest if I explain my usecase. I use buildout for my Django projects (with the django recipe being a big piece of the pie). Lots of geographical processing, so mapnik, matplotlib, numpy and a couple of other not-pure-python packages. > > Some of those are notoriously difficult/impossible to install with a simple "easy_install" or generic buildout dependency install. "aptitude install python-matplotlib" and friends (or the windows equivalent) is the only practical way to get it running. We have similar requirements. > > a) I *do* need the system python. > > b) I prefer listing all dependencies (so also the shitty-to-install ones) in my setup.py. So I need to be able to tell buildout to grab them from the system packages instead of shooting itself in the foot by trying to grab them from pypi. > > c) I prefer having buildout search explicitly for these specific dependencies and complain when it doesn't find them. Helps with missing libraries. > > > > Now, I looked at the new buildout 1.5 system packages options, but I wondered whether it would work fully. As I said, the requirements sound the same to me. > Only the script recipe seems to support it at the moment and it didn't look from the documentation that such a globally-found dependency would be valid for the entire buildout (and all the other parts). If I use that osc.recipe.sysegg and list that part first, it makes sure all other parts use the specified system eggs. But then you are susceptible to many problems that occur in system Pythons that I've been debugging for this release. Which django recipe do you use, by the way? I intend to update djangorecipe, using some changes that Attila made. > > > Just a minute ago, I tried experimenting with a z3c.recipe.egg part mentioned as the first one, but I got an "ImportError: Entry point ('zc.buildout', 'default') not found". If you can help me dupe, I'm happy to look at it. > I haven't updated the bootstrap yet, so that might be the cause (though that'd be a bug). If it is that you didn't update the bootstrap, then I stated it was one of the requirements. :-) Not inclined to call it a bug. Gary From reinout at vanrees.org Fri Aug 27 10:23:32 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Fri, 27 Aug 2010 10:23:32 +0200 Subject: [Distutils] Buildout release news In-Reply-To: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> Message-ID: <4C777604.6050509@vanrees.org> On 08/27/2010 02:31 AM, Gary Poster wrote: >> Only the script recipe seems to support it at the moment and it didn't look from the documentation that such a globally-found dependency would be valid for the entire buildout (and all the other parts). If I use that osc.recipe.sysegg and list that part first, it makes sure all other parts use the specified system eggs. > > But then you are susceptible to many problems that occur in system Pythons that I've been debugging for this release. Ok, you're trying to tell me not to do something, only I don't know what that "something" is :-) Do you mean it is a bad idea to use "matplotlib from the system packages" for the whole buildout? That it is better just to use it in the two or three parts where it is needed? It's not like I want my system matplotlib in one part and a locally failing-horribly buildout-installed matplotlib in another part. The current implementation makes it look like every single part that needs the system matplotlib needs to have a configuration for "look in the system for matplotlib". Ah! Did I perhaps misunderstand the 1.5.x changes? I'm used to have osc.recipe.sysegg pick two or three specific system eggs and add just those to the sys.path. Does the 1.5.x functionality only have a generic switch to switch on ALL system eggs? I'm getting lost :-) > Which django recipe do you use, by the way? I intend to update djangorecipe, using some changes that Attila made. http://pypi.python.org/pypi/djangorecipe >> Just a minute ago, I tried experimenting with a z3c.recipe.egg part mentioned as the first one, but I got an "ImportError: Entry point ('zc.buildout', 'default') not found". > > If you can help me dupe, I'm happy to look at it. I'll look in to it after next week. Two sites have a deadline then. So I'm happy with the 1.4.x-only bootstrap.py :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Collega's gezocht! Django/python vacature in Utrecht: http://tinyurl.com/35v34f9 From agroszer at gmail.com Fri Aug 27 11:13:20 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Fri, 27 Aug 2010 11:13:20 +0200 Subject: [Distutils] buildbot for buildout (again) Message-ID: <3910053380.20100827111320@gmail.com> Hello, After the 1.5.0 I tried again to setup a buildbot (and on top of that on *windows*) for buildout. Good news is that the tests now work. Bad news is that there are a hell lot of failures. More bad news is that there are some failures even on linux (a local ubuntu). Does anyone care about having a buildbot for buildout and fixing those failures? -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: Kiss your keyboard goodbye! From jim at zope.com Fri Aug 27 12:54:12 2010 From: jim at zope.com (Jim Fulton) Date: Fri, 27 Aug 2010 06:54:12 -0400 Subject: [Distutils] buildbot for buildout (again) In-Reply-To: <3910053380.20100827111320@gmail.com> References: <3910053380.20100827111320@gmail.com> Message-ID: On Fri, Aug 27, 2010 at 5:13 AM, Adam GROSZER wrote: > Hello, > > After the 1.5.0 I tried again to setup a buildbot (and on top of that > on *windows*) for buildout. > Good news is that the tests now work. > Bad news is that there are a hell lot of failures. > More bad news is that there are some failures even on linux (a local > ubuntu). > > Does anyone care about having a buildbot for buildout and fixing those > failures? Obviously, I want the tests to pass. I'll wade in, if necessary after the dust settles from 1.5. I suspect that part of the problem may be that the tests tend to depend on how the python used to run them is configured. I always ran them with a clean Python. Other people tended to run them with a Python that had setuptools or distribute installed, and changed the tests to make them pass in their environment -- and so on. I'm going to take a fresh look at the way the tests are written and run to try to address this, among other things. Jim -- Jim Fulton From agroszer at gmail.com Fri Aug 27 13:05:29 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Fri, 27 Aug 2010 13:05:29 +0200 Subject: [Distutils] buildbot for buildout (again) In-Reply-To: References: <3910053380.20100827111320@gmail.com> Message-ID: <103369955.20100827130529@gmail.com> Hello Jim, Friday, August 27, 2010, 12:54:12 PM, you wrote: JF> On Fri, Aug 27, 2010 at 5:13 AM, Adam GROSZER wrote: >> Hello, >> >> After the 1.5.0 I tried again to setup a buildbot (and on top of that >> on *windows*) for buildout. >> Good news is that the tests now work. >> Bad news is that there are a hell lot of failures. >> More bad news is that there are some failures even on linux (a local >> ubuntu). >> >> Does anyone care about having a buildbot for buildout and fixing those >> failures? JF> Obviously, I want the tests to pass. I'll wade in, if necessary after the JF> dust settles from 1.5. JF> I suspect that part of the problem may be that the tests tend to JF> depend on how the python used to run them is configured. I always ran JF> them with a clean Python. Other people tended to run them with a Python JF> that had setuptools or distribute installed, and changed the tests to make them JF> pass in their environment -- and so on. I'm going to take a fresh JF> look at the way JF> the tests are written and run to try to address this, among other things. I think it would help a lot having the exact requirements for tests written down. Or even maybe checking those in the tests's setUp? There are some presumptions how it would work, but it's rather hard to guess. -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: What happens when you cut back the jungle? It recedes. From gary.poster at canonical.com Fri Aug 27 14:56:20 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 27 Aug 2010 08:56:20 -0400 Subject: [Distutils] buildbot for buildout (again) In-Reply-To: References: <3910053380.20100827111320@gmail.com> Message-ID: On Aug 27, 2010, at 6:54 AM, Jim Fulton wrote: > On Fri, Aug 27, 2010 at 5:13 AM, Adam GROSZER wrote: >> Hello, >> >> After the 1.5.0 I tried again to setup a buildbot (and on top of that >> on *windows*) for buildout. >> Good news is that the tests now work. >> Bad news is that there are a hell lot of failures. >> More bad news is that there are some failures even on linux (a local >> ubuntu). FWIW, tests passed for me on Ubuntu (Lucid) and Windows when I made a release, with Distribute and Setuptools, with a variety of Python versions. >> >> Does anyone care about having a buildbot for buildout and fixing those >> failures? > > Obviously, I want the tests to pass. I'll wade in, if necessary after the > dust settles from 1.5. > > I suspect that part of the problem may be that the tests tend to > depend on how the python used to run them is configured. I always ran > them with a clean Python. Definitely. I contemplated trying to make it possible to develop buildout with a system Python, but that needs zc.recipe.testrunner upgraded to the new approach of generating scripts, maybe among other changes. And the initial setup necessary is to call dev.py, if that makes a difference. And there are some pre-existing tests that fail intermittently on Linux. I didn't address these. Gary > Other people tended to run them with a Python > that had setuptools or distribute installed, and changed the tests to make them > pass in their environment -- and so on. I'm going to take a fresh > look at the way > the tests are written and run to try to address this, among other things. > > Jim > > -- > Jim Fulton > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From marius at pov.lt Fri Aug 27 15:00:08 2010 From: marius at pov.lt (Marius Gedminas) Date: Fri, 27 Aug 2010 16:00:08 +0300 Subject: [Distutils] buildbot for buildout (again) In-Reply-To: <103369955.20100827130529@gmail.com> References: <3910053380.20100827111320@gmail.com> <103369955.20100827130529@gmail.com> Message-ID: <20100827130008.GB22679@fridge.pov.lt> On Fri, Aug 27, 2010 at 01:05:29PM +0200, Adam GROSZER wrote: > Hello Jim, > > Friday, August 27, 2010, 12:54:12 PM, you wrote: > JF> I suspect that part of the problem may be that the tests tend to > JF> depend on how the python used to run them is configured. I always ran > JF> them with a clean Python. Other people tended to run them with a Python > JF> that had setuptools or distribute installed, and changed the tests to make them > JF> pass in their environment -- and so on. I'm going to take a fresh > JF> look at the way > JF> the tests are written and run to try to address this, among other things. > > I think it would help a lot having the exact requirements for tests written > down. I believe they are: http://zope3.pov.lt/trac/browser/zc.buildout/trunk/DEVELOPERS.txt > Or even maybe checking those in the tests's setUp? That would be even better, because people generally don't read documentation. Best of all would be if the tests could be made to work with a Python that already has setuptools/distribute installed. Virtualenv is so much easier than compiling your own clean Python from scratch. Marius Gedminas -- Remember, drive defensively! And of course, the best defense is a good offense! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: Digital signature URL: From gary.poster at canonical.com Fri Aug 27 16:12:09 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 27 Aug 2010 10:12:09 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <4C777604.6050509@vanrees.org> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> Message-ID: <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> On Aug 27, 2010, at 4:23 AM, Reinout van Rees wrote: > On 08/27/2010 02:31 AM, Gary Poster wrote: >>> Only the script recipe seems to support it at the moment and it didn't look from the documentation that such a globally-found dependency would be valid for the entire buildout (and all the other parts). If I use that osc.recipe.sysegg and list that part first, it makes sure all other parts use the specified system eggs. >> >> But then you are susceptible to many problems that occur in system Pythons that I've been debugging for this release. > > Ok, you're trying to tell me not to do something, only I don't know what that "something" is :-) :-) Fair enough. Here's my message. I realize that a system Python happened to work for you before. That's a bit awkward, because with pre-1.5.0 zc.buildout, using a system Python was not supported. Therefore, your recipe was not supported. Happily, zc.buildout does support a system Python. Moreover, it is designed to support your use case. Unhappily, your previously unsupported recipe does not work with it. Also unhappily, you can't switch to the 1.5 line until enough recipes have changed to make it possible for you to use it. For now, then, yay Buildout 1.4.4! Eventually, yay, you will be able to fulfill your recipe's use cases in a way that is much more robust against various things that can happen in a system Python! > Do you mean it is a bad idea to use "matplotlib from the system packages" for the whole buildout? That it is better just to use it in the two or three parts where it is needed? It needs to be possible to do this. > It's not like I want my system matplotlib in one part and a locally failing-horribly buildout-installed matplotlib in another part. No, but single buildouts are used to install different sections with entirely different requirements, including different Python versions. > The current implementation makes it look like every single part that needs the system matplotlib needs to have a configuration for "look in the system for matplotlib". Yes, parts are individually installed, per my statement above. That said, I generally think it is a good idea for recipes to look in the buildout section for defaults to the pertinent options. That's what z3c.recipe.scripts does, and that's what I intend to do for the other recipes I upgrade. That means that, effectively, you would be able to make a single gesture in your buildout section and other parts will honor it. > > Ah! Did I perhaps misunderstand the 1.5.x changes? I'm used to have osc.recipe.sysegg pick two or three specific system eggs and add just those to the sys.path. Does the 1.5.x functionality only have a generic switch to switch on ALL system eggs? No. As mentioned before, the documentation for http://pypi.python.org/pypi/z3c.recipe.scripts should give you an idea on the features. Please see the options include-site-packages and allowed-eggs-from-site-packages in particular. > > I'm getting lost :-) > >> Which django recipe do you use, by the way? I intend to update djangorecipe, using some changes that Attila made. > > http://pypi.python.org/pypi/djangorecipe Cool, that's the one. > >>> Just a minute ago, I tried experimenting with a z3c.recipe.egg part mentioned as the first one, but I got an "ImportError: Entry point ('zc.buildout', 'default') not found". >> >> If you can help me dupe, I'm happy to look at it. > > I'll look in to it after next week. Two sites have a deadline then. So I'm happy with the 1.4.x-only bootstrap.py :-) Cool. :-) Gary From setuptools at bugs.python.org Fri Aug 27 19:57:18 2010 From: setuptools at bugs.python.org (Carl Meyer) Date: Fri, 27 Aug 2010 17:57:18 +0000 Subject: [Distutils] [issue113] namespace packages installed with --single-version-externally-managed break other packages in that namespace In-Reply-To: <1282931838.72.0.475399061658.issue113@psf.upfronthosting.co.za> Message-ID: <1282931838.72.0.475399061658.issue113@psf.upfronthosting.co.za> New submission from Carl Meyer : When a namespace package is installed using the --single-version-externally-managed flag, setuptools generates a special nspkg .pth file for that package to make it work as a namespace package (instead of calling pkg_resources.declare_namespace in the init.py file). When a namespace package using this .pth file is present in site-packages, any other packages installed in that same namespace that use the declare_namespace method will not be available for import. The magic code in the nspkg .pth file should be fixed to make it work with packages in the same namespace using declare_namespace. To reproduce this issue in a virtualenv, try the following: $ virtualenv --no-site-packages testenv # this installs zope.interface as an egg, with declare_namespace in zope/__init__.py $ testenv/bin/easy_install zope.interface $ testenv/bin/python -c "import zope.interface" # this installs zope.event using --single-version-externally-managed - pip uses this flag for all installations $ testenv/bin/pip install zope.event $ testenv/bin/python -c "import zope.interface" Traceback (most recent call last): File "", line 1, in ImportError: No module named interface The use of easy_install and pip above is for convenience and not necessary to reproduce the bug. Downloading the sdists for zope.interface and zope.event, and installing them with "python setup.py install" (and --single-version-externally-managed for zope.event) will show the same symptom. I can also reproduce the same issue without a virtualenv on an Ubuntu Karmic system that has zope.interface installed as a system package (which uses declare_namespace in its init.py). If I install zope.event using --single-version-externally-managed, the system zope.interface is no longer importable. ---------- messages: 537 nosy: carljm priority: bug status: unread title: namespace packages installed with --single-version-externally-managed break other packages in that namespace _______________________________________________ Setuptools tracker _______________________________________________ From reinout at vanrees.org Fri Aug 27 21:59:13 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Fri, 27 Aug 2010 21:59:13 +0200 Subject: [Distutils] Buildout release news In-Reply-To: <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: On 08/27/2010 04:12 PM, Gary Poster wrote: > > I realize that a system Python happened to work for you before. > > That's a bit awkward, because with pre-1.5.0 zc.buildout, using a system Python was not supported. > > Therefore, your recipe was not supported. "my recipe"? osc.recipe.sysegg is not something that *I* made, I'm just using it. "Is not supported" sounds like a helpdesk telling me "linux is not supported". And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. > No, but single buildouts are used to install different sections with entirely different requirements, including different Python versions. > >> The current implementation makes it look like every single part that needs the system matplotlib needs to have a configuration for "look in the system for matplotlib". > > Yes, parts are individually installed, per my statement above. What? I've never ever seen a buildout that has to work with different python versions in different parts. A plone buildout just uses one python. A django buildout, too. So it sounds to me like some impressive over-engineering? Note, I don't intend to offend with that statement :-) It's just that I'm thinking "I'll stick to 1.4.4 for the next half year and I'll re-check whether it all works again afterwards". And I don't like that kind of thought, being that I'm using buildout a lot. I introduced it in my current company. I (co-)wrote recipes. So it feels a bit dangerous if I'm not happily jumping on the newest version ;-) Anway... I'll delve deeper into it in one or two weeks :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Collega's gezocht! Django/python vacature in Utrecht: http://tinyurl.com/35v34f9 From gary.poster at canonical.com Fri Aug 27 22:23:14 2010 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 27 Aug 2010 16:23:14 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: On Aug 27, 2010, at 3:59 PM, Reinout van Rees wrote: > On 08/27/2010 04:12 PM, Gary Poster wrote: >> >> I realize that a system Python happened to work for you before. >> >> That's a bit awkward, because with pre-1.5.0 zc.buildout, using a system Python was not supported. >> >> Therefore, your recipe was not supported. > > "my recipe"? osc.recipe.sysegg is not something that *I* made, I'm just using it. "Is not supported" sounds like a helpdesk telling me "linux is not supported". > > And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. ...Not sure what to tell you there. Jim has been clear from the beginning that system Pythons were not supported with buildout. I added support. Anything before then happened to work. >> No, but single buildouts are used to install different sections with entirely different requirements, including different Python versions. >> >>> The current implementation makes it look like every single part that needs the system matplotlib needs to have a configuration for "look in the system for matplotlib". >> >> Yes, parts are individually installed, per my statement above. > > What? I've never ever seen a buildout that has to work with different python versions in different parts. A plone buildout just uses one python. A django buildout, too. So it sounds to me like some impressive over-engineering? This is something that has been with buildout from the beginning, and has nothing to do with my changes. I've also found it useful, FWIW, both for applications and libraries. > Note, I don't intend to offend with that statement :-) It's just that I'm thinking "I'll stick to 1.4.4 for the next half year and I'll re-check whether it all works again afterwards". And I don't like that kind of thought, being that I'm using buildout a lot. I introduced it in my current company. I (co-)wrote recipes. So it feels a bit dangerous if I'm not happily jumping on the newest version ;-) > > > Anway... I'll delve deeper into it in one or two weeks :-) OK. Gary From agroszer at gmail.com Sat Aug 28 14:48:19 2010 From: agroszer at gmail.com (Adam GROSZER) Date: Sat, 28 Aug 2010 14:48:19 +0200 Subject: [Distutils] buildbot for buildout (again) In-Reply-To: References: <3910053380.20100827111320@gmail.com> Message-ID: <8510123642.20100828144819@gmail.com> Hello, Yeah! Having a clean system python helps a lot. zc_buildout_dev py_254_win32 zc_buildout_dev py_265_win32 zc_buildout_dev py_270_win32 are all GREEN zc_buildout_dev py_244_win32 zc_buildout_dev py_265_win64 zc_buildout_dev py_270_win64 have 1-1 failures. I'll let the winbot now send mails about status to zope-test at zope.org Friday, August 27, 2010, 2:56:20 PM, you wrote: GP> On Aug 27, 2010, at 6:54 AM, Jim Fulton wrote: >> On Fri, Aug 27, 2010 at 5:13 AM, Adam GROSZER wrote: >>> Hello, >>> >>> After the 1.5.0 I tried again to setup a buildbot (and on top of that >>> on *windows*) for buildout. >>> Good news is that the tests now work. >>> Bad news is that there are a hell lot of failures. >>> More bad news is that there are some failures even on linux (a local >>> ubuntu). GP> FWIW, tests passed for me on Ubuntu (Lucid) and Windows when I GP> made a release, with Distribute and Setuptools, with a variety of Python versions. >>> >>> Does anyone care about having a buildbot for buildout and fixing those >>> failures? >> >> Obviously, I want the tests to pass. I'll wade in, if necessary after the >> dust settles from 1.5. >> >> I suspect that part of the problem may be that the tests tend to >> depend on how the python used to run them is configured. I always ran >> them with a clean Python. GP> Definitely. I contemplated trying to make it possible to develop GP> buildout with a system Python, but that needs zc.recipe.testrunner GP> upgraded to the new approach of generating scripts, maybe among other changes. GP> And the initial setup necessary is to call dev.py, if that makes a difference. GP> And there are some pre-existing tests that fail intermittently on Linux. I didn't address these. GP> Gary >> Other people tended to run them with a Python >> that had setuptools or distribute installed, and changed the tests to make them >> pass in their environment -- and so on. I'm going to take a fresh >> look at the way >> the tests are written and run to try to address this, among other things. >> >> Jim >> >> -- >> Jim Fulton >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig GP> _______________________________________________ GP> Distutils-SIG maillist - Distutils-SIG at python.org GP> http://mail.python.org/mailman/listinfo/distutils-sig -- Best regards, Adam GROSZER mailto:agroszer at gmail.com -- Quote of the day: BOO! We changed Coke again! BLEAH! BLEAH! From adso.lists at gmail.com Sat Aug 28 15:50:47 2010 From: adso.lists at gmail.com (adso.lists) Date: Sat, 28 Aug 2010 15:50:47 +0200 Subject: [Distutils] zc.buildout complains about site-dirs option in distutils.cfg ? Message-ID: <4C791437.6010300@gmail.com> Hi all, I am compiling python 2.4.6 from sources to build my custom python environment in my home directory and i set the next files in the enunciate locations to make sure that the sys.path is modified ( even i am using the standard site-package directory by now ) Regards, /home/user/opt/lib/python2.4/distutils/distutils.cfg , /home/user/opt/lib/python2.4/site-packages/sitecustomize.py /home/user/opt/lib/python2.4/site-packages/altinstall.pth /home/user/opt/lib/python2.4/site-packages/myapp.pth The point is that, when i run buildout, i get an error from zc.buildout about site-dir is not in sys.path , but if i execute the python interpreter that i have just build i can see that "/home/user/opt/lib/python2.4/site-packages" it is in sys.path even more than once . Commenting site-dir solves the issue. Any clue ? Thanks! __________________________error_____________________________________________ Getting distribution for 'zc.buildout==1.1.1'. error: /home/user/opt/lib/python2.4/site-packages (in --site-dirs) is not on sys.path An error occurred when trying to install zc.buildout 1.1.1. Look above this message for any errors that were output by easy_install. While: Bootstrapping. Getting distribution for 'zc.buildout==1.1.1'. Error: Couldn't install: zc.buildout 1.1.1 user at host:~/utilities/zish-0.2/build$ python Python 2.4.6 (#4, Aug 28 2010, 14:47:55) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/home/user/opt/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/ZopeSkel-2.17-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/Cheetah-2.2.1-py2.4-linux-i686.egg', '/home/user/opt/lib/python2.4/site-packages/PasteScript-1.7.3-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/PasteDeploy-1.3.3-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/Paste-1.7.4-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/virtualenv-1.4.9-py2.4.egg', '/home/user/opt/lib/python24.zip', '/home/user/opt/lib/python2.4', '/home/user/opt/lib/python2.4/plat-linux2', '/home/user/opt/lib/python2.4/lib-tk', '/home/user/opt/lib/python2.4/lib-dynload', '/home/user/opt/lib/python2.4/site-packages', '/home/user/opt/lib/python2.4/site-packages/PIL', '/home/user/opt/lib/python2.4/site-packages'] >>> #--------------------------distutils.cfg ------------------------------------- [build] compiler = unix [install] [easy_install] install-dir = /home/user/opt/lib/python2.4/site-packages script-dir = /home/user/opt/bin build-directory = /home/user/opt/tmp # uncommented raise an error: #error: /home/user/opt/lib/python2.4/site-packages (in --site-dirs) is not on sys.path #site-dirs = /home/user/opt/lib/python2.4/site-packages #--------------------------sitecustomize.py ------------------------------------- #site.addsitedir(sitedir, known_paths=None) #Adds a directory to sys.path and processes its pth files. try: import os import sys import site except ImportError: pass else: sys.setdefaultencoding("utf-8") site.addsitedir(os.path.abspath("/home/user/opt/lib/python2.4/site-packages")) #-------------------------- altinstall.pth ------------------------------------- import os, site site.addsitedir(os.path.abspath("/home/user/opt/lib/python2.4/site-packages")) #-------------------------- myapp.pth ------------------------------------- # quite redundant righ ? /home/user/opt/lib/python2.4 /home/user/opt/lib/python2.4/site-packages From barry at python.org Mon Aug 30 13:09:42 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 30 Aug 2010 07:09:42 -0400 Subject: [Distutils] zope.testrunner behavior change Message-ID: <20100830070942.599bcea3@heresy> I'm just back from a mini-vacation. Before I left, I upgrade the Mailman 3 branch to use buildout 1.5, but I experienced a build failure [1] and a bunch of test failures. Gary fixed all the build problems (thanks!) and I think released a new zope.testrunner package that got my branch back into working state. Mostly. I had one test failure that was fairly easy to fix, but it pointed to a change in zope.testrunner. This may be a question better suited to the TIP list, but well, I'll send it here as a follow up to my buildout issues. ;) The diff should illustrate the behavior change in z.t that broke the test: % bzr diff -c 6934 === modified file 'src/mailman/app/docs/hooks.txt' --- src/mailman/app/docs/hooks.txt 2010-01-12 13:27:38 +0000 +++ src/mailman/app/docs/hooks.txt 2010-08-30 01:00:26 +0000 @@ -51,7 +51,8 @@ >>> def call(): ... proc = subprocess.Popen( ... 'bin/mailman lists --domain ignore -q'.split(), - ... cwd='../..', # testrunner runs from ./parts/test + ... # testrunner runs from ./parts/test/working-directory + ... cwd='../../..', ... env=dict(MAILMAN_CONFIG_FILE=config_path, ... PYTHONPATH=config_directory), ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) The test is trying to run a script that gets built into the bin directory of the built-out tree. I had to set cwd in order to find the bin directory. I probably could have used other landmarks to more dynamically find the bin directory, but this seemed expedient... until z.t changed the cwd in the new version. So my question is: would it make sense to be able to ask zope.testrunner where it's running from, so that I don't have to hard code cwd? Better still would be a function in z.t that I could call that would get me back to root directory. E.g. something like: ... cwd=zope.testrunner.root_directory() Thoughts? It's not that big of a deal, but I don't like that my test relies on undocumented zope.testrunner behavior. Cheers, -Barry [1] https://bugs.edge.launchpad.net/zc.buildout/+bug/623590 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From gary.poster at canonical.com Mon Aug 30 16:08:58 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 10:08:58 -0400 Subject: [Distutils] zope.testrunner behavior change In-Reply-To: <20100830070942.599bcea3@heresy> References: <20100830070942.599bcea3@heresy> Message-ID: On Aug 30, 2010, at 7:09 AM, Barry Warsaw wrote: > I'm just back from a mini-vacation. Before I left, I upgrade the Mailman 3 > branch to use buildout 1.5, but I experienced a build failure [1] and a bunch > of test failures. Gary fixed all the build problems (thanks!) and I think > released a new zope.testrunner package that got my branch back into working > state. > > Mostly. I had one test failure that was fairly easy to fix, but it pointed to > a change in zope.testrunner. This may be a question better suited to the TIP > list, but well, I'll send it here as a follow up to my buildout issues. ;) > > The diff should illustrate the behavior change in z.t that broke the test: > > % bzr diff -c 6934 > === modified file 'src/mailman/app/docs/hooks.txt' > --- src/mailman/app/docs/hooks.txt 2010-01-12 13:27:38 +0000 > +++ src/mailman/app/docs/hooks.txt 2010-08-30 01:00:26 +0000 > @@ -51,7 +51,8 @@ >>>> def call(): > ... proc = subprocess.Popen( > ... 'bin/mailman lists --domain ignore -q'.split(), > - ... cwd='../..', # testrunner runs from ./parts/test > + ... # testrunner runs from ./parts/test/working-directory > + ... cwd='../../..', > ... env=dict(MAILMAN_CONFIG_FILE=config_path, > ... PYTHONPATH=config_directory), > ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) > > > The test is trying to run a script that gets built into the bin directory of > the built-out tree. I had to set cwd in order to find the bin directory. I > probably could have used other landmarks to more dynamically find the bin > directory, but this seemed expedient... until z.t changed the cwd in the new > version. That was a (my) change in zc.recipe.testrunner, not in zope.testrunner. > > So my question is: would it make sense to be able to ask zope.testrunner where > it's running from, so that I don't have to hard code cwd? Better still would > be a function in z.t that I could call that would get me back to root > directory. E.g. something like: > > ... cwd=zope.testrunner.root_directory() > > Thoughts? It's not that big of a deal, but I don't like that my test relies > on undocumented zope.testrunner behavior. I think it is more a buildout question, and buildout does offer a way of getting that information when you are writing scripts. ${buildout: directory} is available in the .cfg files for script initialization (stuff it in an environ variable, maybe?), and of course can be also used in templating solutions like z3c.recipe.filetemplate. Is that sufficient for a solution? I don't know if the zope.testrunner would actually be able to give you the answer you want, but I don't know that package very well. Gary From barry at python.org Mon Aug 30 17:02:51 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 30 Aug 2010 11:02:51 -0400 Subject: [Distutils] zope.testrunner behavior change In-Reply-To: References: <20100830070942.599bcea3@heresy> Message-ID: <20100830110251.0e66e445@heresy> On Aug 30, 2010, at 10:08 AM, Gary Poster wrote: >> So my question is: would it make sense to be able to ask >> zope.testrunner where it's running from, so that I don't have to >> hard code cwd? Better still would be a function in z.t that I could >> call that would get me back to root directory. E.g. something like: >> >> ... cwd=zope.testrunner.root_directory() >> >> Thoughts? It's not that big of a deal, but I don't like that my >> test relies on undocumented zope.testrunner behavior. > >I think it is more a buildout question, and buildout does offer a way >of getting that information when you are writing scripts. ${buildout: >directory} is available in the .cfg files for script initialization >(stuff it in an environ variable, maybe?), and of course can be also >used in templating solutions like z3c.recipe.filetemplate. > >Is that sufficient for a solution? It is! I've already hooked in an initialization key in the [test] section's use of zc.recipe.testrunner in my buildout.cfg, so it was fairly easy to put this together. Below is the relevant code, for the archives. Thanks Gary! -Barry -----buildout.cfg [test] recipe = zc.recipe.testrunner eggs = mailman defaults = '--tests-pattern ^tests --exit-with-status'.split() # Hack in extra arguments to zope.testrunner. initialization = from mailman.testing.layers import ConfigLayer; ConfigLayer.enable_stderr(); ConfigLayer.set_root_directory('${buildout:directory}') -----src/mailman/testing/layers.py class ConfigLayer: # ... @classmethod def set_root_directory(cls, directory): """Set the directory at the root of our source tree. zc.recipe.testrunner runs from parts/test/working-directory, but that's actually changed over the life of the package. Some tests care, e.g. because they need to find our built-out bin directory. Fortunately, buildout can give us this information. See the `buildout.cfg` file for where this method is called. """ cls.root_directory = directory -----src/mailman/app/docs/hooks.txt >>> def call(): ... proc = subprocess.Popen( ... 'bin/mailman lists --domain ignore -q'.split(), ... cwd=ConfigLayer.root_directory, ... env=dict(MAILMAN_CONFIG_FILE=config_path, ... PYTHONPATH=config_directory), ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From chris at simplistix.co.uk Mon Aug 30 19:06:13 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 30 Aug 2010 18:06:13 +0100 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: <4C7BE505.5020208@simplistix.co.uk> Gary Poster wrote: >> It's not like I want my system matplotlib in one part and a locally failing-horribly buildout-installed matplotlib in another part. > > No, but single buildouts are used to install different sections with entirely different requirements, including different Python versions. This feels like needless complexity to me. If a different python is needed, it should be in its own buildout. If you need to bundle a bunch of buildouts together because of this, use a recipe that runs "sub buildouts" in a separate process... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Mon Aug 30 19:07:28 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 30 Aug 2010 18:07:28 +0100 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: <4C7BE550.4080009@simplistix.co.uk> Gary Poster wrote: >> And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. > > ...Not sure what to tell you there. Jim has been clear from the beginning that system Pythons were not supported with buildout. This is somewhat puzzling. What makes a "system python" different from any other sort of python? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Mon Aug 30 19:09:29 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 30 Aug 2010 18:09:29 +0100 Subject: [Distutils] Buildout release news In-Reply-To: References: Message-ID: <4C7BE5C9.4010904@simplistix.co.uk> Maurits van Rees wrote: > What I *do* see is that when you start with zc.buildout 1.5.0, then > change the pin to 1.4.3 and rerun the buildout, this fails when you are > using a buildout extension. The other way around (first 1.4.3, then > change it to 1.5.0) works fine. I tried it with buildout-versions and > with mr.developer as extensions, both having the same result. For > example, start with this, and any bootstrap.py: If it turns out that buildout-versions is doing anything "wrong", please do let me know and I'll see what I can do to fix it! cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From gary.poster at canonical.com Mon Aug 30 19:17:08 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 13:17:08 -0400 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: <4C7BE505.5020208@simplistix.co.uk> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> Message-ID: On Aug 30, 2010, at 1:06 PM, Chris Withers wrote: > Gary Poster wrote: >>> It's not like I want my system matplotlib in one part and a locally failing-horribly buildout-installed matplotlib in another part. >> No, but single buildouts are used to install different sections with entirely different requirements, including different Python versions. > > This feels like needless complexity to me. > > If a different python is needed, it should be in its own buildout. > If you need to bundle a bunch of buildouts together because of this, use a recipe that runs "sub buildouts" in a separate process... - In my experience, that would have been more annoying than the feature that exists. - What you describe does not exist. - It's from before my time; you can argue it with Jim if you like. I've liked it and used it. Gary From chris at simplistix.co.uk Mon Aug 30 19:17:45 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 30 Aug 2010 18:17:45 +0100 Subject: [Distutils] any way to stop buildout/setuptools compiling things that aren't python? In-Reply-To: References: <4C76CEE4.9080407@simplistix.co.uk> <4C76DE26.9070101@simplistix.co.uk> Message-ID: <4C7BE7B9.60309@simplistix.co.uk> Tarek Ziad? wrote: > On Thu, Aug 26, 2010 at 2:35 PM, Chris Withers wrote: >> Tarek Ziad? wrote: >>> The other option is to have the ZSyncer project treat these files as >>> data files, so they >>> don't get compiled >> How do I do that? >> >> They're only being found by setuptools' include_package_data=True... > > http://docs.python.org/distutils/setupscript.html#installing-package-data I'm not sure what you're trying to communicate by pasting this link in reply... I'm using setuptools include_package_data=True, if you're advocating another approach: - will it work on Python 2.6 and 2.5? - what should I replace include_package_data=True with to get the effect you're recommending? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From gary.poster at canonical.com Mon Aug 30 19:17:57 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 13:17:57 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <4C7BE550.4080009@simplistix.co.uk> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE550.4080009@simplistix.co.uk> Message-ID: <3AEA5A06-185B-4299-871C-14CF211A657A@canonical.com> On Aug 30, 2010, at 1:07 PM, Chris Withers wrote: > Gary Poster wrote: >>> And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. >> ...Not sure what to tell you there. Jim has been clear from the beginning that system Pythons were not supported with buildout. > > This is somewhat puzzling. What makes a "system python" different from any other sort of python? Distributions install packages there. Gary From chris at simplistix.co.uk Mon Aug 30 19:22:45 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 30 Aug 2010 18:22:45 +0100 Subject: [Distutils] Buildout release news In-Reply-To: <3AEA5A06-185B-4299-871C-14CF211A657A@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE550.4080009@simplistix.co.uk> <3AEA5A06-185B-4299-871C-14CF211A657A@canonical.com> Message-ID: <4C7BE8E5.10603@simplistix.co.uk> Gary Poster wrote: > On Aug 30, 2010, at 1:07 PM, Chris Withers wrote: > >> Gary Poster wrote: >>>> And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. >>> ...Not sure what to tell you there. Jim has been clear from the beginning that system Pythons were not supported with buildout. >> This is somewhat puzzling. What makes a "system python" different from any other sort of python? > > Distributions install packages there. Okay, but how is that any different from another source-compiled python in which someone has easy_installed packages? "system python" sounds wrong here, it feels like what's meant is "any python in which anything has been installed in site-packages", right? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From jim at zope.com Mon Aug 30 19:27:13 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 30 Aug 2010 13:27:13 -0400 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: <4C7BE505.5020208@simplistix.co.uk> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> Message-ID: On Mon, Aug 30, 2010 at 1:06 PM, Chris Withers wrote: > Gary Poster wrote: >>> >>> It's not like I want my system matplotlib in one part and a locally >>> failing-horribly buildout-installed matplotlib in another part. >> >> No, but single buildouts are used to install different sections with >> entirely different requirements, including different Python versions. > > This feels like needless complexity to me. This is a requirement we had here at ZC. It wasn't needless. > If a different python is needed, it should be in its own buildout. > If you need to bundle a bunch of buildouts together because of this, use a > recipe that runs "sub buildouts" in a separate process... It's possible that this would be a better approach. It's true that supporting multiple Python interpreters is a pain. I don't have this need atm, so I'm not motivated to try your solution. :) I wonder what other people think. Does anyone else have current need to deal with multiple Python versions in the same buildout? Jim -- Jim Fulton From benji at benjiyork.com Mon Aug 30 19:42:34 2010 From: benji at benjiyork.com (Benji York) Date: Mon, 30 Aug 2010 13:42:34 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <4C7BE8E5.10603@simplistix.co.uk> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE550.4080009@simplistix.co.uk> <3AEA5A06-185B-4299-871C-14CF211A657A@canonical.com> <4C7BE8E5.10603@simplistix.co.uk> Message-ID: On Mon, Aug 30, 2010 at 1:22 PM, Chris Withers wrote: > Gary Poster wrote: >> >> On Aug 30, 2010, at 1:07 PM, Chris Withers wrote: >> >>> Gary Poster wrote: >>>>> >>>>> And "not supported"? ?osc.recipe.sysegg installs the specific eggs I >>>>> tell it to find (in the system python) as develop eggs. ?A nice trick that >>>>> works pretty well. >>>> >>>> ...Not sure what to tell you there. ?Jim has been clear from the >>>> beginning that system Pythons were not supported ?with buildout. >>> >>> This is somewhat puzzling. What makes a "system python" different from >>> any other sort of python? >> >> Distributions install packages there. > > Okay, but how is that any different from another source-compiled python in > which someone has easy_installed packages? > > "system python" sounds wrong here, it feels like what's meant is "any python > in which anything has been installed in site-packages", right? It's not that "system python" is wrong, it just doesn't encompass all the things that are unsupported. You are correct in that a system Python is a "dirty" Python and dirty Python's may break buildout. -- Benji York From jim at zope.com Mon Aug 30 19:50:34 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 30 Aug 2010 13:50:34 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: On Fri, Aug 27, 2010 at 4:23 PM, Gary Poster wrote: > > On Aug 27, 2010, at 3:59 PM, Reinout van Rees wrote: > >> On 08/27/2010 04:12 PM, Gary Poster wrote: >>> >>> I realize that a system Python happened to work for you before. >>> >>> That's a bit awkward, because with pre-1.5.0 zc.buildout, using a system Python was not supported. >>> >>> Therefore, your recipe was not supported. >> >> "my recipe"? ?osc.recipe.sysegg is not something that *I* made, I'm just using it. ?"Is not supported" sounds like a helpdesk telling me "linux is not supported". >> >> And "not supported"? ?osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. ?A nice trick that works pretty well. > > ...Not sure what to tell you there. ?Jim has been clear from the beginning that system Pythons were not supported ?with buildout. ?I added support. ?Anything before then happened to work. I'm not sure I agree with this. :) Technically, you're right, in that I didn't offer any sympathy or support for people who had trouble using unclean Pythons (defined as Python installations that differ from a build from source). Still, I expected buildout to use anything found in the Python it was bootstrapped with, with all of the advantages and disadvantages that implied. :) This seemed the most conservative approach at the time and didn't cost me anything because I've never found it burdonsome to provide clean Python installes separate from the system Python. While, I think my decision was raesonable, I came to regret it. I prefer the approach taken by virtualenv to provide isolation by default and provide a way to relax it. My preferance is that buildout provide isolated and non-isolated modes and that the isolation apply equally to recipes, extensions, and eggs installed by parts. I'd be inclined to have buildout be non-islated by default, for backwards compatibility, and switch the default in buildout 2. I say all of this having done none of the recent work. :) Jim -- Jim Fulton From gary.poster at canonical.com Mon Aug 30 20:10:28 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 14:10:28 -0400 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> Message-ID: <6773CA91-6C05-461D-98B2-5E0E10068C08@canonical.com> On Aug 30, 2010, at 1:27 PM, Jim Fulton wrote: > On Mon, Aug 30, 2010 at 1:06 PM, Chris Withers wrote: >> Gary Poster wrote: >>>> >>>> It's not like I want my system matplotlib in one part and a locally >>>> failing-horribly buildout-installed matplotlib in another part. >>> >>> No, but single buildouts are used to install different sections with >>> entirely different requirements, including different Python versions. >> >> This feels like needless complexity to me. > > This is a requirement we had here at ZC. It wasn't > needless. > >> If a different python is needed, it should be in its own buildout. >> If you need to bundle a bunch of buildouts together because of this, use a >> recipe that runs "sub buildouts" in a separate process... > > It's possible that this would be a better approach. It's true that > supporting multiple Python interpreters is a pain. I don't have this > need atm, so I'm not motivated to try your solution. :) > > I wonder what other people think. Does anyone else have current > need to deal with multiple Python versions in the same buildout? Launchpad has needed to when we needed to generate scripts that were only supported by Python version X but Launchpad only supported version Y. This was typically in transition periods, so the feature was very nice--we would be switching back to a uniform Python soon, but meanwhile, the buildout and its components still worked. That kind of thing is generally my most compelling usage. I've also used it (or seen it used? I forget) to set up multiple test runners for different Python versions, for library packages. That's convenient sometimes. I could imagine these being done in a way as Chris described it, but to be convenient I'd want to share configuration across the buildouts--and meanwhile, the feature that does exist has suited nicely. Gary From jim at zope.com Mon Aug 30 20:19:36 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 30 Aug 2010 14:19:36 -0400 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: <6773CA91-6C05-461D-98B2-5E0E10068C08@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> <6773CA91-6C05-461D-98B2-5E0E10068C08@canonical.com> Message-ID: On Mon, Aug 30, 2010 at 2:10 PM, Gary Poster wrote: > > On Aug 30, 2010, at 1:27 PM, Jim Fulton wrote: > >> On Mon, Aug 30, 2010 at 1:06 PM, Chris Withers wrote: >>> Gary Poster wrote: >>>>> >>>>> It's not like I want my system matplotlib in one part and a locally >>>>> failing-horribly buildout-installed matplotlib in another part. >>>> >>>> No, but single buildouts are used to install different sections with >>>> entirely different requirements, including different Python versions. >>> >>> This feels like needless complexity to me. >> >> This is a requirement we had here at ZC. ?It wasn't >> needless. >> >>> If a different python is needed, it should be in its own buildout. >>> If you need to bundle a bunch of buildouts together because of this, use a >>> recipe that runs "sub buildouts" in a separate process... >> >> It's possible that this would be a better approach. ?It's true that >> supporting multiple Python interpreters is a pain. ?I don't have this >> need atm, so I'm not motivated to try your solution. :) >> >> I wonder what other people think. ?Does anyone else have current >> need to deal with multiple Python versions in the same buildout? > > Launchpad has needed to when we needed to generate scripts that were only supported by Python version X but Launchpad only supported version Y. ?This was typically in transition periods, so the feature was very nice--we would be switching back to a uniform Python soon, but meanwhile, the buildout and its components still worked. > > That kind of thing is generally my most compelling usage. > > I've also used it (or seen it used? ?I forget) to set up multiple test runners for different Python versions, for library packages. ?That's convenient sometimes. Now that you mention it, I've used it this way too recently, however recent turmoil with the Zope test runner make this use case break for me. Some subtleties arise from the fact that you can only use one Python for the recipes and the recipes will sometimes depend in significant ways on the Python used. > > I could imagine these being done in a way as Chris described it, but to be convenient I'd want to share configuration across the buildouts This could be done using a files that were extended. It might even be practical to share a single config and pass overrides to the sub-buildout. It's hard to know without trying it. :) >--and meanwhile, the feature that does exist has suited nicely. True, however, It has provided faitly painful to maintain, especially wrt tests. Jim -- Jim Fulton From gary.poster at canonical.com Mon Aug 30 21:06:52 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 15:06:52 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> Message-ID: <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> On Aug 30, 2010, at 1:50 PM, Jim Fulton wrote: > On Fri, Aug 27, 2010 at 4:23 PM, Gary Poster wrote: >> >> On Aug 27, 2010, at 3:59 PM, Reinout van Rees wrote: >> >>> On 08/27/2010 04:12 PM, Gary Poster wrote: >>>> >>>> I realize that a system Python happened to work for you before. >>>> >>>> That's a bit awkward, because with pre-1.5.0 zc.buildout, using a system Python was not supported. >>>> >>>> Therefore, your recipe was not supported. >>> >>> "my recipe"? osc.recipe.sysegg is not something that *I* made, I'm just using it. "Is not supported" sounds like a helpdesk telling me "linux is not supported". >>> >>> And "not supported"? osc.recipe.sysegg installs the specific eggs I tell it to find (in the system python) as develop eggs. A nice trick that works pretty well. >> >> ...Not sure what to tell you there. Jim has been clear from the beginning that system Pythons were not supported with buildout. I added support. Anything before then happened to work. > > I'm not sure I agree with this. :) LOL. Sorry to misrepresent. > Technically, you're right, in that > I didn't offer any sympathy or support for people who had trouble > using unclean Pythons (defined as Python installations that differ > from a build from source). Right, this is what I was referring to. > Still, I expected buildout to use anything found in the Python it was > bootstrapped with, with all of the advantages and disadvantages that > implied. :) OK. > This seemed the most conservative approach at the time > and didn't cost me anything because I've never found it burdonsome to > provide clean Python installes separate from the system Python. > > While, I think my decision was raesonable, I came to regret it. I > prefer the approach taken by virtualenv to provide isolation by > default and provide a way to relax it. > > My preferance is that buildout provide isolated and non-isolated modes > and that the isolation apply equally to recipes, extensions, and eggs > installed by parts. Having fought through all this, I really think having separate isolation controls for the buildout (buildout, recipes, extensions) versus the things that are built by parts is the right way to go. It's two very different domains, and the people writing the two kinds of code are often completely separate. > I'd be inclined to have buildout be non-islated > by default, for backwards compatibility, and switch the default in > buildout 2. As a meta comment, from the peanut gallery , I'm concerned about the buildout 2 plans (where "buildout 2" means "package name change") because they remind me of a couple of other similar historical changes that have permanently (or, at least, so far, often for years) divided communities. The forking is entirely your prerogative, of course. As to the practical issue of what to do now, here are options, working up to your stated preference. 1) Leave 1.5 and the trunk as it is, try to support osc.recipe.sysegg if someone directs me to the code, and move on, without a buildout 2. Unsurprisingly, that's my preference. (Post-weekend, I think I can see how to adapt the recipe Reinout listed to work with an isolated buildout, given the current way 1.5 works. I'd do the searching for eggs in a subprocess with the desired Python, and I suspect everything else would work out fine, with the added robustness of 1.5.0's general isolation. Since the recipe has no tests in the tarball and Google doesn't show me the sourcecode, I'm not yet encouraged to give it a whirl, but could be.) 2) Built eggs are non-isolated by default if you use zc.recipe.egg. Switching to z3c.recipe.scripts is an explicit gesture. However, I'm adjusting *other* recipes to use code from z3c.recipe.scripts to generate their scripts, and these (zc.recipe.testrunner and z3c.recipe.tag for now, z3c.recipe.i18n locally on my machine, djangorecipe intended to come possibly among others) are defaulting to isolation. I could make these other recipes default to not isolating. I'd regard that as a shame, because I think it is really nice for buildout to be able to be used with a system Python easily without having to know about special switches or anything else, but practically it would be pretty easy for me. 3) It would be more complex but doable to make all buildout bits (recipes, extensions, and buildout itself) default to not being isolated. Given the current architecture, which I have no desire to change myself, I think it would have to involve one bootstrap for an isolated environment and one bootstrap for a non-isolated environment, and then various changes within Buildout itself. I don't like this, but it is what I see would be needed to support the current version of osc.recipe.sysegg. 4) We can have unified isolation controls for buildout (+ recipes and extensions) and the built eggs. FWIW, I'm happy with option 1, and helping out with osc.recipe.sysegg if given some direction. I'm OK with option 2, and doing the work. I'm unhappy with option 3, but would try to do the work. I'd have to be further convinced about option 4. Gary From reinout at vanrees.org Mon Aug 30 22:16:05 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 30 Aug 2010 22:16:05 +0200 Subject: [Distutils] Buildout release news In-Reply-To: <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> Message-ID: On 08/30/2010 09:06 PM, Gary Poster wrote: > > (Post-weekend, I think I can see how to adapt the recipe Reinout listed to work with an isolated buildout, given the current way 1.5 works. I'd do the searching for eggs in a subprocess with the desired Python, and I suspect everything else would work out fine, with the added robustness of 1.5.0's general isolation. Since the recipe has no tests in the tarball and Google doesn't show me the sourcecode, I'm not yet encouraged to give it a whirl, but could be.) I mailed the author for something unrelated to this discussion about two weeks ago. I didn't hear anything back, yet. - Author doesn't reply quickly (ok, it could be vacation time :-) ) - No findable source code. => Don't put (too much) effort into this. Rewriting the recipe by doing the thing you suggest (subprocess with the python buildout was started with) sounds doable. Just install a couple of explicitly searched-for packages from that system python as development eggs or so. That would give me what I want: "just use the installed debian package for matplotlib and mapnik" for all parts in the buildout. The subprocess hack sounds a bit dirty, though. It makes me feel like I'm working around buildout. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Collega's gezocht! Django/python vacature in Utrecht: http://tinyurl.com/35v34f9 From reinout at vanrees.org Mon Aug 30 22:18:35 2010 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 30 Aug 2010 22:18:35 +0200 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> Message-ID: On 08/30/2010 07:27 PM, Jim Fulton wrote: > > I wonder what other people think. Does anyone else have current > need to deal with multiple Python versions in the same buildout? I was surprised it was even possible :-) Never heard of the option till a couple of days ago. I've used buildout for 4 years or so and I've never had the need to use multiple pythons. Everything was just a single plone/grok/django project with some scripts and recipes. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Collega's gezocht! Django/python vacature in Utrecht: http://tinyurl.com/35v34f9 From gary.poster at canonical.com Mon Aug 30 22:47:38 2010 From: gary.poster at canonical.com (Gary Poster) Date: Mon, 30 Aug 2010 16:47:38 -0400 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> Message-ID: <59B5AA0D-915C-485D-A0C0-565A0602A37C@canonical.com> On Aug 30, 2010, at 4:16 PM, Reinout van Rees wrote: > On 08/30/2010 09:06 PM, Gary Poster wrote: >> >> (Post-weekend, I think I can see how to adapt the recipe Reinout listed to work with an isolated buildout, given the current way 1.5 works. I'd do the searching for eggs in a subprocess with the desired Python, and I suspect everything else would work out fine, with the added robustness of 1.5.0's general isolation. Since the recipe has no tests in the tarball and Google doesn't show me the sourcecode, I'm not yet encouraged to give it a whirl, but could be.) > > I mailed the author for something unrelated to this discussion about two weeks ago. I didn't hear anything back, yet. > > - Author doesn't reply quickly (ok, it could be vacation time :-) ) > > - No findable source code. > > => Don't put (too much) effort into this. OK, thank you for heads up. > > > > Rewriting the recipe by doing the thing you suggest (subprocess with the python buildout was started with) sounds doable. Just install a couple of explicitly searched-for packages from that system python as development eggs or so. > > That would give me what I want: "just use the installed debian package for matplotlib and mapnik" for all parts in the buildout. > > The subprocess hack sounds a bit dirty, though. It makes me feel like I'm working around buildout. FWIW, It's what buildout does for itself all over the place. It's also reasonable to my mind if you want to explicitly break through an isolation barrier like this, but I won't argue taste. Gary From jim at zope.com Mon Aug 30 23:17:41 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 30 Aug 2010 17:17:41 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> Message-ID: On Mon, Aug 30, 2010 at 3:06 PM, Gary Poster wrote: > > On Aug 30, 2010, at 1:50 PM, Jim Fulton wrote: ... >> My preferance is that buildout provide isolated and non-isolated modes >> and that the isolation apply equally to recipes, extensions, and eggs >> installed by parts. > Having fought through all this, I really think having separate isolation controls for the buildout (buildout, recipes, extensions) versus the things that are built by parts is the right way to go. ?It's two very different domains, and the people writing the two kinds of code are often completely separate. I don't want to make work for you, but I don't understand what issues you fought. Are there any links you can give that might allow me to get this for myself? > As a meta comment, from the peanut gallery , I'm concerned about the buildout 2 plans (where "buildout 2" means "package name change") because they remind me of a couple of other similar historical changes that have permanently (or, at least, so far, often for years) divided communities. ?The forking is entirely your prerogative, of course. I wasn't planning on a fork, a name change, or to do anything disruptive. My main desire would be to try to see if I can simplify the implementation, *especially* wrt tests. The reason for the "2" would be mainly to change some defaults in non-backward-compatable ways, including: - Isolated by default - Prefer final by default - unzip by default > As to the practical issue of what to do now, here are options, working up to your stated preference. ... > 2) Built eggs are non-isolated by default if you use zc.recipe.egg. I don't know what you mean by "built eggs". ... Jim -- Jim Fulton From jim at zope.com Mon Aug 30 23:20:24 2010 From: jim at zope.com (Jim Fulton) Date: Mon, 30 Aug 2010 17:20:24 -0400 Subject: [Distutils] Buildout release news In-Reply-To: <59B5AA0D-915C-485D-A0C0-565A0602A37C@canonical.com> References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> <59B5AA0D-915C-485D-A0C0-565A0602A37C@canonical.com> Message-ID: On Mon, Aug 30, 2010 at 4:47 PM, Gary Poster wrote: > > On Aug 30, 2010, at 4:16 PM, Reinout van Rees wrote: ... >> The subprocess hack sounds a bit dirty, though. It makes me feel like I'm working around buildout. > > FWIW, It's what buildout does for itself all over the place. ?It's also reasonable to my mind if you want to explicitly break through an isolation barrier like this, but I won't argue taste. Buildout uses subprocess to run setup scripts. That's pretty much all. It runs setups scripts in a subprocess because they are written expecting to be the main program and could do all sorts of things that could break buildout if run in process. Jim -- Jim Fulton From marius at pov.lt Tue Aug 31 00:13:47 2010 From: marius at pov.lt (Marius Gedminas) Date: Tue, 31 Aug 2010 01:13:47 +0300 Subject: [Distutils] Buildout release news In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <0F06B2EA-7DD5-4280-825F-83F01AA2BA86@canonical.com> Message-ID: <20100830221347.GA29560@fridge.pov.lt> On Mon, Aug 30, 2010 at 05:17:41PM -0400, Jim Fulton wrote: > I wasn't planning on a fork, a name change, or to do anything > disruptive. > > My main desire would be to try to see if I can simplify the > implementation, *especially* wrt tests. The reason for the "2" would > be mainly to change some defaults in non-backward-compatable ways, > including: ... > - unzip by default YEEEEEEEEES!!!! Sorry, got carried away there for a bit. Marius Gedminas -- MSDOS didn't get as bad as it is overnight -- it took over ten years of careful development. -- dmeggins at aix1.uottawa.ca -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 190 bytes Desc: Digital signature URL: From sdouche at gmail.com Tue Aug 31 01:09:43 2010 From: sdouche at gmail.com (Sebastien Douche) Date: Tue, 31 Aug 2010 01:09:43 +0200 Subject: [Distutils] different python versions in one buildout?! In-Reply-To: References: <19BF88C4-9AE3-46B0-BBCE-6115F4A4C625@canonical.com> <4C777604.6050509@vanrees.org> <94B2F054-FA29-430B-94B1-AAB2B04F748F@canonical.com> <4C7BE505.5020208@simplistix.co.uk> Message-ID: On Mon, Aug 30, 2010 at 22:18, Reinout van Rees wrote: > I've used buildout for 4 years or so and I've never had the need to use > multiple pythons. ?Everything was just a single plone/grok/django project > with some scripts and recipes. "me too", and I'm a heavy user of buildout (we use it for dev and prod). -- Sebastien Douche Twitter: http://bit.ly/afkrK (agile, lean, python, open source)