From skip at pobox.com Wed Mar 4 07:11:52 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 4 Mar 2009 00:11:52 -0600 (CST) Subject: [Distutils] Can you collapse easy-install.pth somehow? Message-ID: <20090304061152.31AC3E2E5AF@montanaro.dyndns.org> So I have rather casually installed a bunch of packages from PyPI using easy_install. It now contains 23 eggs, all of which get used a bit, but nowhere near as often as the standard lib or our internal stuff. Unsuccessfully stat()ing in all those egg directories was really starting to hammer our NFS server (we often start up hundreds of Python processes at a time), so I commented out the second import line at the end which serves to leave the eggs at the end of sys.path. Now every time I install a package I have to quickly edit easy-install.path to remove the import line the install process adds. I wonder though... Is there some alternate way to install stuff from PyPI which reduces the number of directories on sys.path? Maybe some post-processing step which takes the current set of eggs, creates a single directory into which they are all unfolded? That single directory could then be added to sys.path. Am I making sense? It's late. I might well not be. If so, ask me for a more complete explanation of the idea floating around my brain. -- Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ From pje at telecommunity.com Wed Mar 4 17:29:51 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 04 Mar 2009 11:29:51 -0500 Subject: [Distutils] Can you collapse easy-install.pth somehow? In-Reply-To: <20090304061152.31AC3E2E5AF@montanaro.dyndns.org> References: <20090304061152.31AC3E2E5AF@montanaro.dyndns.org> Message-ID: <20090304162736.CF6AB3A4097@sparrow.telecommunity.com> At 12:11 AM 3/4/2009 -0600, skip at pobox.com wrote: >So I have rather casually installed a bunch of packages from PyPI using >easy_install. It now contains 23 eggs, all of which get used a bit, but >nowhere near as often as the standard lib or our internal stuff. >Unsuccessfully stat()ing in all those egg directories was really starting to >hammer our NFS server (we often start up hundreds of Python processes at a >time), so I commented out the second import line at the end which serves to >leave the eggs at the end of sys.path. Now every time I install a package I >have to quickly edit easy-install.path to remove the import line the install >process adds. > >I wonder though... Is there some alternate way to install stuff from PyPI >which reduces the number of directories on sys.path? Maybe some >post-processing step which takes the current set of eggs, creates a single >directory into which they are all unfolded? That single directory could >then be added to sys.path. You could use Ian Bicking's 'pip' installer. It can't install from eggs, but it does do source installs this way. There is a very straightforward mapping from zipped eggs to "flat" files, btw. For a given .egg file or directory, the mapping is simply to unzip it, renaming its EGG-INFO subdirectory to the original egg's filename with '-info' appended. E.g. "foobar-1.2-py2.5.egg/EGG-INFO/" becomes "foobar-1.2-py2.5.egg-info/". Everything else remains exactly the same. It wouldn't be hard to make a tool that does this; in fact, the setuptools.archive_util module has an 'extract_archive' function that works on directories as well as zipfiles, and can be passed a filter function that can change the extraction location for any given file. Something like: def flatten_egg(egg_filename, extract_dir): eggbase = os.path.filename(egg_filename)+'-info' def file_filter(src, dst): if src.startswith('EGG-INFO/'): src = eggbase+s[8:] dst = os.path.join(extract_dir, *src.split('/')) return dst return extract_archive(egg_filename, extract_dir, file_filter) Of course, this will only work if 'egg_filename' is a valid .egg file or directory; the results are otherwise pretty unpredictable. This also doesn't check for overwriting other files, deal with permissions, etc. Ergo, making this production-ready is left as an exercise for the implementer. ;-) From ianb at colorstudy.com Wed Mar 4 21:12:12 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 4 Mar 2009 14:12:12 -0600 Subject: [Distutils] PEP 376 for Distutils In-Reply-To: <20090225221440.59D323A4075@sparrow.telecommunity.com> References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com> <20090223231310.GA5711@laurie.devork> <94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com> <49A3B236.4050001@online.de> <94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com> <20090224151824.A79383A4075@sparrow.telecommunity.com> <49A44EFD.50201@astraw.com> <20090225220310.GA4665@laurie.devork> <20090225221440.59D323A4075@sparrow.telecommunity.com> Message-ID: On Wed, Feb 25, 2009 at 4:16 PM, P.J. Eby wrote: > I wonder why they don't just use the sitewide distutils.cfg file, which > would let them configure user-installed packages to go to somewhere else, > e.g.: > > [install] > prefix = /usr/local > > (And of course the build tools could specify different options.) Some systems do this. I've found this problematic with virtualenv, as people get weird results because they don't realize this configuration file exists. Also there's no way currently in distutils to unset a value, so if something like prefix is given explicitly there's no way to indicate it should be calculated (you just have to give an explicit value for --prefix). -- Ian Bicking | http://blog.ianbicking.org From tseaver at palladion.com Wed Mar 4 22:29:26 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 04 Mar 2009 16:29:26 -0500 Subject: [Distutils] Running tests for zc.recipe.cmmi? Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I was trying to update the package to get rid of a deprecation warning under Python 2.6. $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk $ cd cmmi-trunk $ /path/to/bin/python2.6 bootstrap/bootstrap.py ... $ bin/buildout ... $ bin/test ... *Lots* of import errors and test failures What's up with that? It blows up with python2.4 and python2.5, too. While we're at it, can we get a release of zc.buildout? The trunk has a similar deprecation fix under Python 2.6. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJrvK2+gerLs4ltQ4RAlgPAKDDbWgoh8WRlenPLV6CsKhXwdwgaQCfbmuV OsMW7reliBe+M2V3K1g0bSs= =pyP2 -----END PGP SIGNATURE----- From jim at zope.com Wed Mar 4 23:14:02 2009 From: jim at zope.com (Jim Fulton) Date: Wed, 4 Mar 2009 17:14:02 -0500 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: Message-ID: <4FBD207C-48B0-4FC2-9988-74F8B0487F2C@zope.com> On Mar 4, 2009, at 4:29 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I was trying to update the package to get rid of a deprecation warning > under Python 2.6. > > $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk > $ cd cmmi-trunk > $ /path/to/bin/python2.6 bootstrap/bootstrap.py > ... > $ bin/buildout > ... > $ bin/test > ... > *Lots* of import errors and test failures > > What's up with that? It blows up with python2.4 and python2.5, too. I don't know. I'll look into it. (Unless someone else beats me to it. :) > While we're at it, can we get a release of zc.buildout? The trunk > has a > similar deprecation fix under Python 2.6. Sure. I'll try to fit this in soon. Jim -- Jim Fulton Zope Corporation From rupert.thurner at gmail.com Thu Mar 5 04:25:52 2009 From: rupert.thurner at gmail.com (rupert.thurner) Date: Wed, 4 Mar 2009 19:25:52 -0800 (PST) Subject: [Distutils] easy_install --upgrade-all ? Message-ID: <4c2f1fd6-efc2-4947-9ac1-1cfa7f28d762@r16g2000vbp.googlegroups.com> is there a possibility to upgrade all installed eggs with easy_install resp setuptools? From nad at visi.com Thu Mar 5 06:15:25 2009 From: nad at visi.com (Ned Deily) Date: Wed, 04 Mar 2009 21:15:25 -0800 Subject: [Distutils] easy_install --upgrade-all ? References: <4c2f1fd6-efc2-4947-9ac1-1cfa7f28d762@r16g2000vbp.googlegroups.com> Message-ID: In article <4c2f1fd6-efc2-4947-9ac1-1cfa7f28d762 at r16g2000vbp.googlegroups.com>, "rupert.thurner" wrote: > is there a possibility to upgrade all installed eggs with easy_install > resp setuptools? yolk -U queries PyPI and reports which installed packages are out-of-date. A little bit of hackery could turn the output into easy_install commands, maybe something like: yolk -U | cut -d ' ' -f 2 | xargs easy_install -- Ned Deily, nad at visi.com -- [] From fadhley.salim at uk.calyon.com Thu Mar 5 18:27:46 2009 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Thu, 5 Mar 2009 17:27:46 -0000 Subject: [Distutils] How to find the default values for --find-links References: <4c2f1fd6-efc2-4947-9ac1-1cfa7f28d762@r16g2000vbp.googlegroups.com> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483FA1@MXCU10MX1.MSX.CIB> In an automated build environment I need to be able to make eggs which depend on non-released "testing" eggs. These are all published to a web-server operated by my team. I know that it's possible to globally change the default URLs of the "--find-links" easy_install option by editing the distutils.cfg file, however I want a fully automated process which will validate a system's configuration and determine if the test-egg URL have been added to the system. If not, I want my build process to fail with an explicit error emssage. Other than by directly inspecting and parsing this file, is there a programatic way to find which hosts have been configured in the distutils.cfg file? Ideally I'd like to access this information via pkg_resources. Any suggestions? Thanks -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit? des Etablissements de Cr?dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From pje at telecommunity.com Thu Mar 5 18:55:50 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 05 Mar 2009 12:55:50 -0500 Subject: [Distutils] How to find the default values for --find-links In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483FA1@MXCU10MX1.MSX.CIB > References: <4c2f1fd6-efc2-4947-9ac1-1cfa7f28d762@r16g2000vbp.googlegroups.com> <7F347D91614EBC48AA7540A2A76D3BB204483FA1@MXCU10MX1.MSX.CIB> Message-ID: <20090305175335.152593A4097@sparrow.telecommunity.com> At 05:27 PM 3/5/2009 +0000, Fadhley Salim wrote: >In an automated build environment I need to be able to make eggs which >depend on non-released "testing" eggs. These are all published to a >web-server operated by my team. > >I know that it's possible to globally change the default URLs of the >"--find-links" easy_install option by editing the distutils.cfg file, >however I want a fully automated process which will validate a system's >configuration and determine if the test-egg URL have been added to the >system. If not, I want my build process to fail with an explicit error >emssage. > >Other than by directly inspecting and parsing this file, is there a >programatic way to find which hosts have been configured in the >distutils.cfg file? Ideally I'd like to access this information via >pkg_resources. It's not available there. You would need to create an easy_install command instance and inspect its find_links attribute. Something like: ei = makeSetup().get_command_obj('easy_install') ei.ensure_finalized() print ei.find_links See the 'makeSetup()' function in setuptools/tests/__init__.py for how to do the equivalent. (Now, if what you're really asking is, can you tell whether a specific version of an egg is available on sys.path, you can certainly do that by checking pkg_resources.require('projectname')[0].version to see if it's the version you want.) From philippe at fluendo.com Fri Mar 6 11:06:08 2009 From: philippe at fluendo.com (Philippe Normand) Date: Fri, 06 Mar 2009 11:06:08 +0100 Subject: [Distutils] unicode and pkg_resources Message-ID: <1236333968.16024.7.camel@localhost.localdomain> Hi, I'm having issues with pkg_resources.resource_filename(). If the file is located in a directory containing accentuated characters (like ?), the path is wrong, it's a byte string unproperly encoded whereas it should be a unicode... I searched on the Python bugtracker about that issue but found nothing... Phillip, and others, what do you think? Regards, Philippe From ziade.tarek at gmail.com Mon Mar 9 06:42:46 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 9 Mar 2009 06:42:46 +0100 Subject: [Distutils] Python Packaging Survey Message-ID: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> Hello, Thanks to all the people that helped building the survey, and a special thanks to Massimo Di Pierro who created the application that runs the Survey and helped me set it up. Please, take it ! http://tinyurl.com/package-survey Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From greno at verizon.net Mon Mar 9 19:39:58 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 09 Mar 2009 14:39:58 -0400 Subject: [Distutils] Python Packaging Survey In-Reply-To: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> Message-ID: <49B5627E.8060802@verizon.net> Tarek Ziad? wrote: > Hello, > > Thanks to all the people that helped building the survey, and a > special thanks to Massimo Di Pierro who created the application that > runs the Survey and helped me set it up. > > Please, take it ! > > http://tinyurl.com/package-survey > > Regards > Tarek > > Tarek, I took the survey but I forgot to add this: Need to add 'bdist_deb' patches into Distutils. Also, can 'bdist_deb' be added as an extension to existing Distutils installations? That would be great. Regards, Gerry From p.f.moore at gmail.com Mon Mar 9 20:35:59 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Mar 2009 19:35:59 +0000 Subject: [Distutils] Python Packaging Survey In-Reply-To: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> Message-ID: <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> 2009/3/9 Tarek Ziad? : > Thanks to all the people that helped building the survey, and a > special thanks to Massimo Di Pierro who created the application that > runs the Survey and helped me set it up. > > Please, take it ! > > http://tinyurl.com/package-survey I get a page load error - something about an expired self-signed certificate (this with Firefox on Windows). Paul From greno at verizon.net Mon Mar 9 20:44:51 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 09 Mar 2009 15:44:51 -0400 Subject: [Distutils] Python Packaging Survey In-Reply-To: <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> Message-ID: <49B571B3.7000500@verizon.net> Paul Moore wrote: > 2009/3/9 Tarek Ziad? : > >> Thanks to all the people that helped building the survey, and a >> special thanks to Massimo Di Pierro who created the application that >> runs the Survey and helped me set it up. >> >> Please, take it ! >> >> http://tinyurl.com/package-survey >> > > I get a page load error - something about an expired self-signed > certificate (this with Firefox on Windows). > > Paul > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > Paul, I got that too but I just clicked by all the warnings and the survey works fine after that. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Mon Mar 9 20:47:37 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 9 Mar 2009 20:47:37 +0100 Subject: [Distutils] Python Packaging Survey In-Reply-To: <49B571B3.7000500@verizon.net> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> Message-ID: <94bdd2610903091247y35d41cc3v991a2276ad9e96d3@mail.gmail.com> Yes I am aware of thsi. I told Massimo. Thx for the feedback Tarek On Mon, Mar 9, 2009 at 8:44 PM, Gerry Reno wrote: > Paul Moore wrote: > > 2009/3/9 Tarek Ziad? : > > > Thanks to all the people that helped building the survey, and a > special thanks to Massimo Di Pierro who created the application that > runs the Survey and helped me set it up. > > Please, take it ! > > http://tinyurl.com/package-survey > > > I get a page load error - something about an expired self-signed > certificate (this with Firefox on Windows). > > Paul > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > > > Paul, > ? I got that too but I just clicked by all the warnings and the survey works > fine after that. > > Regards, > Gerry > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From bill at celestial.com Mon Mar 9 20:44:02 2009 From: bill at celestial.com (Bill Campbell) Date: Mon, 9 Mar 2009 12:44:02 -0700 Subject: [Distutils] Python Packaging Survey In-Reply-To: <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> Message-ID: <20090309194402.GA11511@ayn.mi.celestial.com> On Mon, Mar 09, 2009, Paul Moore wrote: >2009/3/9 Tarek Ziad? : >> Thanks to all the people that helped building the survey, and a >> special thanks to Massimo Di Pierro who created the application that >> runs the Survey and helped me set it up. >> >> Please, take it ! >> >> http://tinyurl.com/package-survey > >I get a page load error - something about an expired self-signed >certificate (this with Firefox on Windows). The latest versions of Firefox are pretty anal about certs, but can be coerced into accepting them. I got a minor message using Safari on OS X. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 The stamping of paper is an operation so much easier than the laying of taxes, that a government, in the practice of paper emissions, would rarely fail, in any such emergency [such as an election], to indulge itself too far in the employment of that resource... -- Alexander Hamilton From p.f.moore at gmail.com Mon Mar 9 21:19:52 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Mar 2009 20:19:52 +0000 Subject: [Distutils] Python Packaging Survey In-Reply-To: <49B571B3.7000500@verizon.net> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> Message-ID: <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> 2009/3/9 Gerry Reno : >> I get a page load error - something about an expired self-signed >> certificate (this with Firefox on Windows). > ? I got that too but I just clicked by all the warnings and the survey works > fine after that. Ah, I see it now. It's a lot harder to find the "let me through anyway" option on Firefox. Paul From ziade.tarek at gmail.com Mon Mar 9 22:05:05 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 9 Mar 2009 22:05:05 +0100 Subject: [Distutils] Python Packaging Survey In-Reply-To: <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> Message-ID: <94bdd2610903091405r333d09es1e794e2bf206a65e@mail.gmail.com> On Mon, Mar 9, 2009 at 9:19 PM, Paul Moore wrote: > 2009/3/9 Gerry Reno : >>> I get a page load error - something about an expired self-signed >>> certificate (this with Firefox on Windows). > >> ? I got that too but I just clicked by all the warnings and the survey works >> fine after that. > > Ah, I see it now. It's a lot harder to find the "let me through > anyway" option on Firefox. > Massimo has enabled the http version, you can reach it with : http://tinyurl.com/packaging-survey > Paul > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From greno at verizon.net Mon Mar 9 22:15:18 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 09 Mar 2009 17:15:18 -0400 Subject: [Distutils] Python Packaging Survey In-Reply-To: <94bdd2610903091405r333d09es1e794e2bf206a65e@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> <94bdd2610903091405r333d09es1e794e2bf206a65e@mail.gmail.com> Message-ID: <49B586E6.5050701@verizon.net> Tarek Ziad? wrote: > On Mon, Mar 9, 2009 at 9:19 PM, Paul Moore wrote: > > Massimo has enabled the http version, you can reach it with : > http://tinyurl.com/packaging-survey > Tarek, In the survey there were some questions that were all or nothing. For one I answered Pylons but we also have projects using TG2 so I flipped a coin basically. Don't know how many others might have similar circumstance or whether it makes any difference. Regards, Gerry From david.lyon at preisshare.net Tue Mar 10 00:29:32 2009 From: david.lyon at preisshare.net (David Lyon) Date: Mon, 09 Mar 2009 19:29:32 -0400 Subject: [Distutils] Python Packaging Survey In-Reply-To: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> Message-ID: <2e1bd01a4c4505885e04a7cb5e5b2497@preisshare.net> On Mon, 9 Mar 2009 06:42:46 +0100, Tarek Ziad? wrote: > Please, take it ! > > http://tinyurl.com/package-survey mdp.cti.depaul.edu uses an invalid security certificate. The certificate is not trusted because it is self signed. The certificate is only valid for Massimo Di Pierro. The certificate expired on 1/03/2009 4:56 PM. (Error code: sec_error_expired_issuer_certificate) From waterbug at pangalactic.us Tue Mar 10 01:50:29 2009 From: waterbug at pangalactic.us (Stephen Waterbury) Date: Mon, 09 Mar 2009 20:50:29 -0400 Subject: [Distutils] Python Packaging Survey -- question 8 not well-formulated In-Reply-To: <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> Message-ID: <49B5B955.6030909@pangalactic.us> (And now for a comment that's not about the site certificate! >:) Question 8 provides 5 options, none of which apply to me, and no option to comment without choosing one. Question 7 does the right thing by providing an option to give another answer in the comment field. I will wait until that question is fixed before responding to the survey ... Steve From david.lyon at preisshare.net Tue Mar 10 04:19:27 2009 From: david.lyon at preisshare.net (David Lyon) Date: Mon, 09 Mar 2009 23:19:27 -0400 Subject: [Distutils] Packages listed in Easy-Install.pth In-Reply-To: <2e1bd01a4c4505885e04a7cb5e5b2497@preisshare.net> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <2e1bd01a4c4505885e04a7cb5e5b2497@preisshare.net> Message-ID: <251705a76540812a3b2b69fd8faa5879@preisshare.net> Hi All, I am doing some grubby work on the Python Package Manager project. http://sourceforge.net/projects/pythonpkgmgr/ After running and installing some packages.. with EasyInstall, my Easy-Install.pth file gets filled with the package and egg names. What I want to do is use these in a list of installed packages. And then later provide for deinstallation. I'm thinking of parsing this file to read all the package names.. My question is is there any way to do the same thing with setuptools? Forgive my ignorance - I am just hacking the code. :-) ---c:\python25\lib\site-packages\Easy-Install.pth----------------------------- import sys; sys.__plen = len(sys.path) ./setuptools-0.6c9-py2.5.egg ./xmlmp-1.1-py2.5.egg ./xml2ddl-0.3.1-py2.5.egg ./xml_compare-1.0.5-py2.5.egg ./html5lib-0.11.1-py2.5.egg ./html2text-2.35-py2.5.egg import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new) ------------------------------------------------------------------------------- Regards David From pje at telecommunity.com Tue Mar 10 07:17:07 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 10 Mar 2009 02:17:07 -0400 Subject: [Distutils] Packages listed in Easy-Install.pth In-Reply-To: <251705a76540812a3b2b69fd8faa5879@preisshare.net> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <2e1bd01a4c4505885e04a7cb5e5b2497@preisshare.net> <251705a76540812a3b2b69fd8faa5879@preisshare.net> Message-ID: <20090310061450.ECE7D3A410E@sparrow.telecommunity.com> At 11:19 PM 3/9/2009 -0400, David Lyon wrote: >What I want to do is use these in a list of installed packages. And then >later provide for deinstallation. I'm thinking of parsing this file to read >all the package names.. > >My question is is there any way to do the same thing with setuptools? http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects You can simply iterate over pkg_resources.working_set to obtain Distribution objects for all eggs on sys.path -- that will give you all *activated* packages (i.e., ones listed in .pth files or otherwise explicitly on sys.path). If you want all *installed* packages (which may not be on sys.path, but are still physically present in a directory on sys.path), you want an Environment object instead: http://peak.telecommunity.com/DevCenter/PkgResources#environment-objects And to remove a package from the .pth file, you can use "easy_install -mxNd targetdir project==version" -- this will remove the named project from the easy-install.pth file in targetdir. (The -m takes it out, the -x means don't reinstall scripts, the -N means don't install dependencies, and the -d targetdir specifies where the .pth file is.) From ziade.tarek at gmail.com Tue Mar 10 08:29:32 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 10 Mar 2009 08:29:32 +0100 Subject: [Distutils] Python Packaging Survey -- question 8 not well-formulated In-Reply-To: <49B5B955.6030909@pangalactic.us> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <79990c6b0903091235k722d0e9fu1c45e7dc01a1a163@mail.gmail.com> <49B571B3.7000500@verizon.net> <79990c6b0903091319p30e8765eu613cb94f2586ee2@mail.gmail.com> <49B5B955.6030909@pangalactic.us> Message-ID: <94bdd2610903100029v6ef1f6f4m58756ab040add566@mail.gmail.com> On Tue, Mar 10, 2009 at 1:50 AM, Stephen Waterbury wrote: > (And now for a comment that's not about the site certificate! ?>:) > > Question 8 provides 5 options, none of which apply to me, and no > option to comment without choosing one. > > Question 7 does the right thing by providing an option to give > another answer in the comment field. > > I will wait until that question is fixed before responding to > the survey ... added ! Tarek > > Steve > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From zooko at zooko.com Tue Mar 10 15:56:22 2009 From: zooko at zooko.com (zooko) Date: Tue, 10 Mar 2009 08:56:22 -0600 Subject: [Distutils] [zc.buildout] buildout to .deb package In-Reply-To: References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com> <94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com> Message-ID: By the way, stdeb builds .deb files from distutils-packaged Python source distributions as well as from setuptools-packaged Python source distributions. Regards, Zooko From zooko at zooko.com Tue Mar 10 16:09:42 2009 From: zooko at zooko.com (zooko) Date: Tue, 10 Mar 2009 09:09:42 -0600 Subject: [Distutils] patch: solving the two major things that people dislike about setuptools, part 1 In-Reply-To: <7.0.1.0.0.20090123230523.0278fec8@telecommunity.com> References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com> <17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com> <20090121002856.405BA3A4065@sparrow.telecommunity.com> <4978D070.9010709@stsci.edu> <87iqo72e0z.fsf@benfinney.id.au> <497A481F.4000703@stsci.edu> <7.0.1.0.0.20090123230523.0278fec8@telecommunity.com> Message-ID: <06FD2157-C082-43BA-B1B3-CEF2BF55B66C@zooko.com> What's the status of Mark Sienkiewicz's patch to solve the most widely-hated feature of setuptools (that it disrespects the order of your PYTHONPATH) while also satisfying PJE's requirements for precedence of egg importing? http://mail.python.org/pipermail/distutils-sig/2009-January/010768.html Here's the ticket: http://bugs.python.org/setuptools/issue53 # respect the PYTHONPATH Regards, Zooko From jim at zope.com Tue Mar 10 17:30:12 2009 From: jim at zope.com (Jim Fulton) Date: Tue, 10 Mar 2009 12:30:12 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: Message-ID: On Mar 4, 2009, at 4:29 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I was trying to update the package to get rid of a deprecation warning > under Python 2.6. > > $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk > $ cd cmmi-trunk > $ /path/to/bin/python2.6 bootstrap/bootstrap.py > ... > $ bin/buildout > ... > $ bin/test > ... > *Lots* of import errors and test failures > > What's up with that? It blows up with python2.4 and python2.5, too. I don't get test failures with Python 2.4 or 2.5. I do get failures for 2.6. Jim -- Jim Fulton Zope Corporation From sienkiew at stsci.edu Tue Mar 10 21:59:50 2009 From: sienkiew at stsci.edu (Mark Sienkiewicz) Date: Tue, 10 Mar 2009 16:59:50 -0400 Subject: [Distutils] patch: solving the two major things that people dislike about setuptools, part 1 In-Reply-To: <06FD2157-C082-43BA-B1B3-CEF2BF55B66C@zooko.com> References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com> <17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com> <20090121002856.405BA3A4065@sparrow.telecommunity.com> <4978D070.9010709@stsci.edu> <87iqo72e0z.fsf@benfinney.id.au> <497A481F.4000703@stsci.edu> <7.0.1.0.0.20090123230523.0278fec8@telecommunity.com> <06FD2157-C082-43BA-B1B3-CEF2BF55B66C@zooko.com> Message-ID: <49B6D4C6.7090008@stsci.edu> zooko wrote: > What's the status of Mark Sienkiewicz's patch to solve the most > widely-hated feature of setuptools (that it disrespects the order of > your PYTHONPATH) while also satisfying PJE's requirements for > precedence of egg importing? It would make certain things easier if I could install setuptools, so this patch is not entirely outside my job description, but I have a lot of work with a higher priority. I still intend to get to it at some point. If somebody else wants to write it, go right ahead. Just send me an email so I know you're doing it and don't write a conflicting patch. One of the followups to the message you cited contains requirements (or as close as you're likely to get, anyway) for the patch. I also posted a brief outline of a patch some time before that. As far as I can tell, defining that set of requirements is the hard part of the patch. The actual implementation is not rocket science. Mark S. From arne_bab at web.de Tue Mar 10 22:13:11 2009 From: arne_bab at web.de (Arne Babenhauserheide) Date: Tue, 10 Mar 2009 22:13:11 +0100 Subject: [Distutils] data in a source distribution? Message-ID: <200903102213.14925.arne_bab@web.de> Hi, Is it possible to include data in a source distribution? I'd like to avoid eggs, since they don't integrate well with Gentoo packages, but I didn't find a way to include data in a source distribution (python setup.py sdist). Did I miss something? Best wishes, Arne -- -- Ein W?rfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln. -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From david.lyon at preisshare.net Tue Mar 10 23:36:53 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 10 Mar 2009 18:36:53 -0400 Subject: [Distutils] Packages listed in Easy-Install.pth In-Reply-To: <20090310061450.ECE7D3A410E@sparrow.telecommunity.com> References: <94bdd2610903082242s47beeb40nd96f3ac50e0ea904@mail.gmail.com> <2e1bd01a4c4505885e04a7cb5e5b2497@preisshare.net> <251705a76540812a3b2b69fd8faa5879@preisshare.net> <20090310061450.ECE7D3A410E@sparrow.telecommunity.com> Message-ID: <7c83b88a5fcf4d3f4a189f284122c0b0@preisshare.net> Thanks you PJ.... that is most helpful. Seems to be the answer I was looking for. :-) On Tue, 10 Mar 2009 02:17:07 -0400, "P.J. Eby" wrote: > At 11:19 PM 3/9/2009 -0400, David Lyon wrote: > >>What I want to do is use these in a list of installed packages. And then >>later provide for deinstallation. I'm thinking of parsing this file to > read >>all the package names.. >> >>My question is is there any way to do the same thing with setuptools? > > http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects > > You can simply iterate over pkg_resources.working_set to obtain > Distribution objects for all eggs on sys.path -- that will give you > all *activated* packages (i.e., ones listed in .pth files or > otherwise explicitly on sys.path). > > If you want all *installed* packages (which may not be on sys.path, > but are still physically present in a directory on sys.path), you > want an Environment object instead: > > http://peak.telecommunity.com/DevCenter/PkgResources#environment-objects > > And to remove a package from the .pth file, you can use "easy_install > -mxNd targetdir project==version" -- this will remove the named > project from the easy-install.pth file in targetdir. (The -m takes > it out, the -x means don't reinstall scripts, the -N means don't > install dependencies, and the -d targetdir specifies where the .pth file > is.) From tseaver at palladion.com Wed Mar 11 00:22:48 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 10 Mar 2009 19:22:48 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: Message-ID: <49B6F648.8040208@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > On Mar 4, 2009, at 4:29 PM, Tres Seaver wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> I was trying to update the package to get rid of a deprecation warning >> under Python 2.6. >> >> $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk >> $ cd cmmi-trunk >> $ /path/to/bin/python2.6 bootstrap/bootstrap.py >> ... >> $ bin/buildout >> ... >> $ bin/test >> ... >> *Lots* of import errors and test failures >> >> What's up with that? It blows up with python2.4 and python2.5, too. > > > I don't get test failures with Python 2.4 or 2.5. I do get failures > for 2.6. I still see failures when repeating that recipe, with 2.4, 2.5, and 2.6. I thought it might have been due to the fact that Grok jammed a goofy ~/.buildout on my site, but the behavior is the same even after I chopped it back out. The error is as follows: - ----------------------------- 8< ---------------------------------- Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Ran 4 tests with 0 failures and 0 errors in 4.441 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Test-modules with import problems: eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_packageindex eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_resources eggs.zc.buildout-1.1.1-py2.4.egg.zc.buildout.tests eggs.zc.recipe.egg-1.1.0-py2.4.egg.zc.recipe.egg.tests eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.common.tests.test_idatetime eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.common.tests.test_import_interfaces eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_adapter eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_advice eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_declarations eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_document eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_element eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_interface eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_odd_declarations eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_sorting eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_verify eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.tests eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.testrunner.tests - ----------------------------- 8< ---------------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJtvZI+gerLs4ltQ4RAt7+AJ97NOhWNhZhhclM6bUPmR2gi2Zc0ACfVpxl BZtDAYNS+jGm/o2l1qd5yKc= =l6vE -----END PGP SIGNATURE----- From tseaver at palladion.com Wed Mar 11 00:22:48 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 10 Mar 2009 19:22:48 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: Message-ID: <49B6F648.8040208@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > On Mar 4, 2009, at 4:29 PM, Tres Seaver wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> I was trying to update the package to get rid of a deprecation warning >> under Python 2.6. >> >> $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk >> $ cd cmmi-trunk >> $ /path/to/bin/python2.6 bootstrap/bootstrap.py >> ... >> $ bin/buildout >> ... >> $ bin/test >> ... >> *Lots* of import errors and test failures >> >> What's up with that? It blows up with python2.4 and python2.5, too. > > > I don't get test failures with Python 2.4 or 2.5. I do get failures > for 2.6. I still see failures when repeating that recipe, with 2.4, 2.5, and 2.6. I thought it might have been due to the fact that Grok jammed a goofy ~/.buildout on my site, but the behavior is the same even after I chopped it back out. The error is as follows: - ----------------------------- 8< ---------------------------------- Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Ran 4 tests with 0 failures and 0 errors in 4.441 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Test-modules with import problems: eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_packageindex eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_resources eggs.zc.buildout-1.1.1-py2.4.egg.zc.buildout.tests eggs.zc.recipe.egg-1.1.0-py2.4.egg.zc.recipe.egg.tests eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.common.tests.test_idatetime eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.common.tests.test_import_interfaces eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_adapter eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_advice eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_declarations eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_document eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_element eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_interface eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_odd_declarations eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_sorting eggs.zope.interface-3.5.0-py2.4-linux-i686.egg.zope.interface.tests.test_verify eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.tests eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.testrunner.tests - ----------------------------- 8< ---------------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJtvZI+gerLs4ltQ4RAt7+AJ97NOhWNhZhhclM6bUPmR2gi2Zc0ACfVpxl BZtDAYNS+jGm/o2l1qd5yKc= =l6vE -----END PGP SIGNATURE----- From pje at telecommunity.com Wed Mar 11 02:27:20 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 10 Mar 2009 21:27:20 -0400 Subject: [Distutils] data in a source distribution? In-Reply-To: <200903102213.14925.arne_bab@web.de> References: <200903102213.14925.arne_bab@web.de> Message-ID: <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> At 10:13 PM 3/10/2009 +0100, Arne Babenhauserheide wrote: >Hi, > >Is it possible to include data in a source distribution? You need to specify the data files in your MANIFEST.in file -- the distutils don't do that automatically for you. Of course, if you use setuptools to create your sdist, and the data files are in revision control, they'll be included automatically. Note, too, that you don't have to use setuptools in your setup.py to do this; you only need to import setuptools to generate your sdist, e.g.: python -c "import setuptools; execfile('setup.py')" sdist register upload That command line will generate and upload an sdist using setuptools, but persons installing the package won't get a setuptools-based installation. From rudd-o at rudd-o.com Wed Mar 11 04:34:48 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Tue, 10 Mar 2009 22:34:48 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) Message-ID: <200903102234.50856.rudd-o@rudd-o.com> Hello, guys, I have fixed distutils (and setuptools remains working) with the attached patch. Now, RPMs will be built according to the Fedora Package Naming Guidelines: http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non- Numeric_Version_in_Release which I understand to be the most useful reference in terms of naming pre- release packages. This should work correctly in at least: - Fedora - RHEL - SUSE I urge you patch your python 2.4s and 2.5s and 2.6s and push this update to distributions. I have done that myself at my own repository. Now we can enjoy one more reason to build RPMs (and eggs! ... according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick its brains) DIRECTLY from the cheese shop, especially if you're using pip. Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). See attached patch. I will log bugs where it corresponds too. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Aqua - Cartoon heroes Windows 95 is not a virus. Viruses actually do something. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils-bdist_rpm-prereleaseversion.patch Type: text/x-patch Size: 1596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Wed Mar 11 05:32:23 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Tue, 10 Mar 2009 23:32:23 -0500 Subject: [Distutils] Addendum SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) Message-ID: <200903102332.24052.rudd-o@rudd-o.com> Hello, guys, The mail I sent earlier had a patch which seems to depend upon a RHEL- distributed patch which I am now attaching (along with the original patch too). I have logged bugs in the pertinent bug trackers. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Aqua - Cartoon heroes Windows 95 is not a virus. Viruses actually do something. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils-bdist_rpm-prereleaseversion.patch Type: text/x-patch Size: 1596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist-rpm.patch (prerequisite for the patch I wrote which is the prereleaseversion.patch) Type: text/x-patch Size: 2935 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From arne_bab at web.de Wed Mar 11 12:48:06 2009 From: arne_bab at web.de (Arne Babenhauserheide) Date: Wed, 11 Mar 2009 12:48:06 +0100 Subject: [Distutils] data in a source distribution? In-Reply-To: <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> References: <200903102213.14925.arne_bab@web.de> <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> Message-ID: <200903111248.11316.arne_bab@web.de> Am Mittwoch 11 M?rz 2009 02:27:20 schrieb P.J. Eby: > You need to specify the data files in your MANIFEST.in file -- the > distutils don't do that automatically for you. Ah - OK. Thank you! > Of course, if you use setuptools to create your sdist, and the data > files are in revision control, they'll be included > automatically. Is there a chance of setuptools recognizing other systems than SVN and cvs? I use Mercurial for all my own projects, and the setuptools don't seem to recognize that yet. > python -c "import setuptools; execfile('setup.py')" sdist register upload That trick looks neat! Many thanks! Best wishes, Arne -- -- Ein W?rfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln. -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From jannis at leidel.info Wed Mar 11 13:21:36 2009 From: jannis at leidel.info (Jannis Leidel) Date: Wed, 11 Mar 2009 13:21:36 +0100 Subject: [Distutils] data in a source distribution? In-Reply-To: <200903111248.11316.arne_bab@web.de> References: <200903102213.14925.arne_bab@web.de> <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> <200903111248.11316.arne_bab@web.de> Message-ID: Hi, > Is there a chance of setuptools recognizing other systems than SVN > and cvs? Yes, there is [1]. > I use Mercurial for all my own projects, and the setuptools don't > seem to > recognize that yet. I created setuptools_hg [2] for that purpose. The bitbucket wiki [3] has a bit more information, too. 1: http://peak.telecommunity.com/DevCenter/setuptools#adding-support-for-other-revision-control-systems 2: http://pypi.python.org/pypi/setuptools_hg 3: http://bitbucket.org/jezdez/setuptools_hg/wiki/ Best, Jannis From greno at verizon.net Wed Mar 11 13:28:40 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 08:28:40 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903102234.50856.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> Message-ID: <49B7AE78.1060208@verizon.net> Manuel Amador (Rudd-O) wrote: > Hello, guys, > > > I have fixed distutils (and setuptools remains working) with the > attached patch. Now, RPMs will be built according to the Fedora > Package Naming Guidelines: > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Version_in_Release > > > which I understand to be the most useful reference in terms of naming > pre-release packages. This should work correctly in at least: > > > - Fedora > - RHEL > - SUSE > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this > update to distributions. I have done that myself at my own repository. > > > Now we can enjoy one more reason to build RPMs (and eggs! ... > according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel > free to pick its brains) DIRECTLY from the cheese shop, especially if > you're using pip. > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > See attached patch. I will log bugs where it corresponds too. > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, You worked on my problem! Great. So today what we have been doing to deal with the pre-release and lexical ordering problem involving pre-releases is this: We impose a restriction on how the pre-release is identified. So for example if you intend to end up with a final version-release of 5.0.0-1 and you want to first put out some betas or release candidates then we have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that the lexical ordering for RPM will be correct. In other words you must put the pre-release designation into the 'release' part of VERSION-RELEASE. What we had seen developers doing previously was to name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release designation part of the 'version' string) which then did not work for the lexical ordering of the final release of 5.0.0-1 because 5.0.0 (version) was not lexically superior to 5.0.0_rc1. So we were able to solve this problem without any code changes to distutils. But this also presented a challenge for the other distribution targets such as 'sdist' because they were totally unaware of this 'version-release' combination and only knew about 'version'. So as a workaround we were doing this: # WORKAROUND # define both version AND release version='5.0.0' release='1' # combine them for all targets except 'bdist_rpm' if sys.argv[1] != 'bdist_rpm': version = version+'-'+release So this wasn't perfect but it actually worked quite well and we could get 'sdist' to work properly in conjunction with 'bdist_rpm'. So now with your patch all the targets should be able to set and use both 'version' and 'release' and we don't need our workaround and that will be great. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Wed Mar 11 13:33:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 11 Mar 2009 13:33:41 +0100 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B7AE78.1060208@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> Message-ID: <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> Hi, Great work ! FYI it's on my pile in the bug tracker in Python. I'll try to work on these before Pycon Also, note that I am planning to release Distutils as a standalone package before Pycon; since the current trunk targets Python 2.3 to 3.1 Regards Tarek On Wed, Mar 11, 2009 at 1:28 PM, Gerry Reno wrote: > Manuel Amador (Rudd-O) wrote: > > Hello, guys, > > I have fixed distutils (and setuptools remains working) with the attached > patch. Now, RPMs will be built according to the Fedora Package Naming > Guidelines: > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Version_in_Release > > which I understand to be the most useful reference in terms of naming > pre-release packages. This should work correctly in at least: > > - Fedora > - RHEL > - SUSE > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this update to > distributions. I have done that myself at my own repository. > > Now we can enjoy one more reason to build RPMs (and eggs! ... according to > my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick its > brains) DIRECTLY from the cheese shop, especially if you're using pip. > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > See attached patch. I will log bugs where it corresponds too. > -- > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, > ? You worked on my problem!? Great. > ? So today what we have been doing to deal with the pre-release and lexical > ordering problem involving pre-releases is this: > ??? We impose a restriction on how the pre-release is identified.? So for > example if you intend to end up with a final version-release of 5.0.0-1 and > you want to first put out some betas or release candidates then we have to > name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that the lexical > ordering for RPM will be correct.? In other words you must put the > pre-release designation into the 'release' part of VERSION-RELEASE.? What we > had seen developers doing previously was to name these as 5.0.0_beta1 or > 5.0.0_rc1 (making the pre-release designation part of the 'version' string) > which then did not work for the lexical ordering of the final release of > 5.0.0-1 because 5.0.0 (version) was not lexically superior to 5.0.0_rc1.? So > we were able to solve this problem without any code changes to distutils. > But this also presented a challenge for the other distribution targets such > as 'sdist' because they were totally unaware of this 'version-release' > combination and only knew about 'version'.? So as a workaround we were doing > this: > # WORKAROUND > # define both version AND release > version='5.0.0' > release='1' > # combine them for all targets except 'bdist_rpm' > if sys.argv[1] != 'bdist_rpm': > ??? version = version+'-'+release > So this wasn't perfect but it actually worked quite well and we could get > 'sdist' to work properly in conjunction with 'bdist_rpm'. > So now with your patch all the targets should be able to set and use both > 'version' and 'release' and we don't need our workaround and that will be > great. > > Regards, > Gerry > > > > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From greno at verizon.net Wed Mar 11 14:01:35 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 09:01:35 -0400 Subject: [Distutils] Packaging: bdist_deb patches Message-ID: <49B7B62F.40306@verizon.net> Tarek, Speaking of packaging patches. Would there be any chance that you could get the 'bdist_deb' patches integrated in Distutils prior to PyCon? Regards, Gerry From jim at zope.com Wed Mar 11 14:07:39 2009 From: jim at zope.com (Jim Fulton) Date: Wed, 11 Mar 2009 09:07:39 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: <49B6F648.8040208@palladion.com> References: <49B6F648.8040208@palladion.com> Message-ID: It might be good to take this off list. Are you using a Python with anything in site-packages? It seem rather odd that, for you, it seems to be importing test modules that are unrelated to zc.recipe.cmmi. Jim On Mar 10, 2009, at 7:22 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jim Fulton wrote: >> On Mar 4, 2009, at 4:29 PM, Tres Seaver wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> I was trying to update the package to get rid of a deprecation >>> warning >>> under Python 2.6. >>> >>> $ svn co $ZSVN/zc.recipe.cmmi/trunk cmmi-trunk >>> $ cd cmmi-trunk >>> $ /path/to/bin/python2.6 bootstrap/bootstrap.py >>> ... >>> $ bin/buildout >>> ... >>> $ bin/test >>> ... >>> *Lots* of import errors and test failures >>> >>> What's up with that? It blows up with python2.4 and python2.5, too. >> >> >> I don't get test failures with Python 2.4 or 2.5. I do get failures >> for 2.6. > > I still see failures when repeating that recipe, with 2.4, 2.5, and > 2.6. > I thought it might have been due to the fact that Grok jammed a goofy > ~/.buildout on my site, but the behavior is the same even after I > chopped it back out. The error is as follows: > > - ----------------------------- 8< ---------------------------------- > Running zope.testing.testrunner.layer.UnitTests tests: > Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. > Ran 4 tests with 0 failures and 0 errors in 4.441 seconds. > Tearing down left over layers: > Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. > > Test-modules with import problems: > eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_packageindex > eggs.setuptools-0.6c9-py2.4.egg.setuptools.tests.test_resources > eggs.zc.buildout-1.1.1-py2.4.egg.zc.buildout.tests > eggs.zc.recipe.egg-1.1.0-py2.4.egg.zc.recipe.egg.tests > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.common.tests.test_idatetime > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.common.tests.test_import_interfaces > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_adapter > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_advice > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_declarations > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_document > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_element > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_interface > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_odd_declarations > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_sorting > > eggs.zope.interface-3.5.0-py2.4-linux- > i686.egg.zope.interface.tests.test_verify > eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.tests > eggs.zope.testing-3.7.1-py2.4.egg.zope.testing.testrunner.tests > - ----------------------------- 8< ---------------------------------- > > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFJtvZI+gerLs4ltQ4RAt7+AJ97NOhWNhZhhclM6bUPmR2gi2Zc0ACfVpxl > BZtDAYNS+jGm/o2l1qd5yKc= > =l6vE > -----END PGP SIGNATURE----- -- Jim Fulton Zope Corporation From rudd-o at rudd-o.com Wed Mar 11 14:29:21 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 08:29:21 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> Message-ID: <200903110829.24944.rudd-o@rudd-o.com> Great. I just pushed onto the various trackers tracking the issue -- among which is the python tracker -- support for prerelease packages (...dev and ...dev-r77655 versioned packages) so they do not overwrite alphas, betas, or final -1 and above releases. El Mi?rcoles 11 Marzo 2009, Tarek Ziad? escribi?: > Hi, > > Great work ! > > FYI it's on my pile in the bug tracker in Python. I'll try to work on > these before Pycon > > Also, note that I am planning to release Distutils as a standalone > package before Pycon; > since the current trunk targets Python 2.3 to 3.1 > > Regards > Tarek > > On Wed, Mar 11, 2009 at 1:28 PM, Gerry Reno wrote: > > Manuel Amador (Rudd-O) wrote: > > > > Hello, guys, > > > > I have fixed distutils (and setuptools remains working) with the attached > > patch. Now, RPMs will be built according to the Fedora Package Naming > > Guidelines: > > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Vers > >ion_in_Release > > > > which I understand to be the most useful reference in terms of naming > > pre-release packages. This should work correctly in at least: > > > > - Fedora > > - RHEL > > - SUSE > > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this update > > to distributions. I have done that myself at my own repository. > > > > Now we can enjoy one more reason to build RPMs (and eggs! ... according > > to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick > > its brains) DIRECTLY from the cheese shop, especially if you're using > > pip. > > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > > See attached patch. I will log bugs where it corresponds too. > > -- > > > > Manuel Amador (Rudd-O) > > Rudd-O.com - http://rudd-o.com/ > > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > > Windows 95 is not a virus. Viruses actually do something. > > > > Hi Manuel, > > ? You worked on my problem!? Great. > > ? So today what we have been doing to deal with the pre-release and > > lexical ordering problem involving pre-releases is this: > > ??? We impose a restriction on how the pre-release is identified.? So for > > example if you intend to end up with a final version-release of 5.0.0-1 > > and you want to first put out some betas or release candidates then we > > have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that > > the lexical ordering for RPM will be correct.? In other words you must > > put the pre-release designation into the 'release' part of > > VERSION-RELEASE.? What we had seen developers doing previously was to > > name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release > > designation part of the 'version' string) which then did not work for the > > lexical ordering of the final release of 5.0.0-1 because 5.0.0 (version) > > was not lexically superior to 5.0.0_rc1.? So we were able to solve this > > problem without any code changes to distutils. But this also presented a > > challenge for the other distribution targets such as 'sdist' because they > > were totally unaware of this 'version-release' combination and only knew > > about 'version'.? So as a workaround we were doing this: > > # WORKAROUND > > # define both version AND release > > version='5.0.0' > > release='1' > > # combine them for all targets except 'bdist_rpm' > > if sys.argv[1] != 'bdist_rpm': > > ??? version = version+'-'+release > > So this wasn't perfect but it actually worked quite well and we could get > > 'sdist' to work properly in conjunction with 'bdist_rpm'. > > So now with your patch all the targets should be able to set and use both > > 'version' and 'release' and we don't need our workaround and that will be > > great. > > > > Regards, > > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ A copy of the universe is not what is required of art; one of the damned things is ample. -- Rebecca West -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From tseaver at palladion.com Wed Mar 11 15:06:31 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 11 Mar 2009 10:06:31 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: <49B6F648.8040208@palladion.com> Message-ID: <49B7C567.3090906@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > It might be good to take this off list. If you think so, OK. > Are you using a Python with anything in site-packages? Only the setuptools 0.6c9 egg. > It seem rather odd that, for you, it seems to be importing test > modules that are unrelated to zc.recipe.cmmi. My thoughts exactly, Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJt8Vn+gerLs4ltQ4RAoslAKCxvziyuv/rCskuESRcknBpSNmDVQCggHd7 gTZHIbL38m8IN5k07l8H9xI= =kMLj -----END PGP SIGNATURE----- From rudd-o at rudd-o.com Wed Mar 11 15:21:45 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 09:21:45 -0500 Subject: [Distutils] =?iso-8859-15?q?SOLVED=3A_bdist=5Frpm_and_pre-release?= =?iso-8859-15?q?_python_packages_/_eggs_=28was=3A_pre-release_versioning_?= =?iso-8859-15?q?problems_with_sdist=2C_=09bdist=5Frpm_=2C_bdist=5Fdebian?= =?iso-8859-15?q?=29?= In-Reply-To: <49B7AE78.1060208@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> Message-ID: <200903110921.48576.rudd-o@rudd-o.com> My solution tackles it from another direction, using observed egg packaging practices. This is what I have observed in setup.pys, and eggs' requires.txts so far: 1.2dev: refers to unspecified not-even-alpha quality checkout or source drop 1.2dev-r5667: refers to the checkout of revision 5667, not-even-alpha 1.2a1: refers to Alpha 1 of intended version 1.2 1.2a2: refers to Alpha 2 of intended version 1.2 1.2b1: refers to Beta 1 of intended version 1.2 1.2: refers to official 1.2 release Note NO UNDERSCORES. In order to satisfy the fedora package naming guidelines and RPM's lexicographical sort, this is what happens within my patches: 1.2dev: version 1.2 release 0.0.0dev REGARDLESS of user-specified release 1.2dev-r56: version 1.2 release 0.0.56devr REGARDLESS of user-specified rel 1.2a1: version 1.2 release 0..a1 1.2a2: version 1.2 release 0..a2 1.2b1: version 1.2 release 0..b1 (note in the last three it is the responsibility of the builder to bump the manually, there is really nothing we can do about it) 1.2: version 1.2 release In this manner we can accurately map existing practice (what people are actually DOING at least in a significant amount of eggs -- the plone ones) in python egg versioning numbers to RPM lexicographical sort in a manner that lets package managers "do the right thing". Also, we need a standard way of trolling the requires.txt for the egg dependencies and transforming them into RPM dependencies, otherwise your RPMs won't work in concert. Code for that in my workbench yum.rudd-o.com subdir SCRIPTS, it is documented step by step, but this should really be integrated within the setuptools bdist_rpm kludge. Really. I am exhausted. El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Manuel Amador (Rudd-O) wrote: > > Hello, guys, > > > > > > I have fixed distutils (and setuptools remains working) with the > > attached patch. Now, RPMs will be built according to the Fedora > > Package Naming Guidelines: > > > > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Vers > >ion_in_Release > > > > > > which I understand to be the most useful reference in terms of naming > > pre-release packages. This should work correctly in at least: > > > > > > - Fedora > > - RHEL > > - SUSE > > > > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this > > update to distributions. I have done that myself at my own repository. > > > > > > Now we can enjoy one more reason to build RPMs (and eggs! ... > > according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel > > free to pick its brains) DIRECTLY from the cheese shop, especially if > > you're using pip. > > > > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > > > > See attached patch. I will log bugs where it corresponds too. > > -- > > > > > > Manuel Amador (Rudd-O) > > Rudd-O.com - http://rudd-o.com/ > > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, > You worked on my problem! Great. > So today what we have been doing to deal with the pre-release and > lexical ordering problem involving pre-releases is this: > We impose a restriction on how the pre-release is identified. So > for example if you intend to end up with a final version-release of > 5.0.0-1 and you want to first put out some betas or release candidates > then we have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is > so that the lexical ordering for RPM will be correct. In other words > you must put the pre-release designation into the 'release' part of > VERSION-RELEASE. What we had seen developers doing previously was to > name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release > designation part of the 'version' string) which then did not work for > the lexical ordering of the final release of 5.0.0-1 because 5.0.0 > (version) was not lexically superior to 5.0.0_rc1. So we were able to > solve this problem without any code changes to distutils. But this also > presented a challenge for the other distribution targets such as 'sdist' > because they were totally unaware of this 'version-release' combination > and only knew about 'version'. So as a workaround we were doing this: > # WORKAROUND > # define both version AND release > version='5.0.0' > release='1' > # combine them for all targets except 'bdist_rpm' > if sys.argv[1] != 'bdist_rpm': > version = version+'-'+release > So this wasn't perfect but it actually worked quite well and we could > get 'sdist' to work properly in conjunction with 'bdist_rpm'. > So now with your patch all the targets should be able to set and use > both 'version' and 'release' and we don't need our workaround and that > will be great. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Example is not the main thing in influencing others. It is the only thing. -- Albert Schweitzer -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From tarek.ziade at ingeniweb.com Wed Mar 11 15:42:33 2009 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Wed, 11 Mar 2009 15:42:33 +0100 Subject: [Distutils] Packaging: bdist_deb patches In-Reply-To: <49B7B62F.40306@verizon.net> References: <49B7B62F.40306@verizon.net> Message-ID: 2009/3/11 Gerry Reno : > Tarek, > ?Speaking of packaging patches. ?Would there be any chance that you could > get the 'bdist_deb' patches integrated in Distutils prior to PyCon? The work to be done for bdist_deb is quite important and requires some Debian expertise I don't have (yet). My plan was to be helped by some Debian people amongst the people that have interacted on that issue and in this list, and work on the patches. But that won't be possible before Pycon, because I am commited on other issues before this one. But if someone is willing to lead this work, and if a fully tested patch is provided before Pycon, I am willing to review it and commit it. Although I doubt this will be possible in such a short time. Regards Tarek > > Regards, > Gerry > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way From greno at verizon.net Wed Mar 11 15:51:47 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 10:51:47 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903110921.48576.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> <200903110921.48576.rudd-o@rudd-o.com> Message-ID: <49B7D003.80305@verizon.net> Manuel Amador (Rudd-O) wrote: > My solution tackles it from another direction, using observed egg > packaging practices. This is what I have observed in setup.pys, and > eggs' requires.txts so far: > > > 1.2dev: refers to unspecified not-even-alpha quality checkout or > source drop > 1.2dev-r5667: refers to the checkout of revision 5667, not-even-alpha > 1.2a1: refers to Alpha 1 of intended version 1.2 > 1.2a2: refers to Alpha 2 of intended version 1.2 > 1.2b1: refers to Beta 1 of intended version 1.2 > 1.2: refers to official 1.2 release > > > Note NO UNDERSCORES. In order to satisfy the fedora package naming > guidelines and RPM's lexicographical sort, this is what happens within > my patches: > > > 1.2dev: version 1.2 release 0.0.0dev REGARDLESS of user-specified release > 1.2dev-r56: version 1.2 release 0.0.56devr REGARDLESS of > user-specified rel > 1.2a1: version 1.2 release 0..a1 > 1.2a2: version 1.2 release 0..a2 > 1.2b1: version 1.2 release 0..b1 > (note in the last three it is the responsibility of the builder to > bump the manually, there is really nothing we can do about it) > 1.2: version 1.2 release > > > In this manner we can accurately map existing practice (what people > are actually DOING at least in a significant amount of eggs -- the > plone ones) in python egg versioning numbers to RPM lexicographical > sort in a manner that lets package managers "do the right thing". > > > Also, we need a standard way of trolling the requires.txt for the egg > dependencies and transforming them into RPM dependencies, otherwise > your RPMs won't work in concert. Code for that in my workbench > yum.rudd-o.com subdir SCRIPTS, it is documented step by step, but this > should really be integrated within the setuptools bdist_rpm kludge. > Really. > > > I am exhausted. Hi Manuel, Let me share with you what we had implemented and the POLICY that we put in place with regards to VERSION-RELEASE strings. We did not look at this from egg standpoint as we just distribute RPMS. I sent this to Tarek a while ago: What I had implemented seems to have the VERSION-RELEASE string issue solved using a 'workaround' and a manual policy for the moment. It would of course be better if distutils just did this automatically but this is at least working. Tarballs, rpms have the same designations and all the lexical ordering is correct so that even for pre-release versioning the rpms update each other correctly. #WORKAROUND: What I did was in setup.py added: ======================== version = '5.0.0' release = '0_rc3_00001681' if sys.argv[1] != 'bdist_rpm': version = version + '-' + release ======================== In setup.cfg added: ======================== [bdist_rpm] # release must exactly match 'release' as set in setup.py release=0_rc3_00001681 ================== # END WORKAROUND And then I setup the following policy regarding VERSION-RELEASE naming: (this example uses bzr as source code control system but you can substitute svn, rcs, or any other type of sccs) Development code: version="5.0.0" release="0_00012345" (where 00012345 is the revision number from bzr in ZERO-PADDED FIELD OF 8) tarball: foo-5.0.0-0_00012345.tar.gz rpm: foo-5.0.0-0_00012345.noarch.rpm Alpha code: version="5.0.0" release="0_alpha1_00123456" tarball: foo-5.0.0-0_alpha1_00123456.tar.gz rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm Beta code: version="5.0.0" release="0_beta1_00234567" tarball: foo-5.0.0-0_beta1_00234567.tar.gz rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm Release Candidate code: version="5.0.0" release="0_rc1" tarball: foo-5.0.0-0_rc1.tar.gz rpm: foo-5.0.0-0_rc1.noarch.rpm Release Candidate code fix: version="5.0.0" release="0_rc1_00345678" tarball: foo-5.0.0-0_rc1_00345678.tar.gz rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm Final Release code: version="5.0.0" release="1" tarball: foo-5.0.0-1.tar.gz rpm: foo-5.0.0-1.noarch.rpm Notice that lexical ordering is proper in all cases. Even where the alpha, beta, rc releases may be followed by a bzr fix revision. The ensures that both tarballs and packages such as rpm will contain the same VERSION-RELEASE designations. Also, the use of UNDERSCORES in the 'release' string assures that there is no ambiguity with regard to parsing VERSION-RELEASE string combination or full package names which have hyphens in specific positions. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Wed Mar 11 16:03:45 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Mar 2009 11:03:45 -0400 Subject: [Distutils] data in a source distribution? In-Reply-To: <200903111248.11316.arne_bab@web.de> References: <200903102213.14925.arne_bab@web.de> <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> <200903111248.11316.arne_bab@web.de> Message-ID: <20090311150141.687203A4116@sparrow.telecommunity.com> At 12:48 PM 3/11/2009 +0100, Arne Babenhauserheide wrote: >Am Mittwoch 11 M?rz 2009 02:27:20 schrieb P.J. Eby: > > You need to specify the data files in your MANIFEST.in file -- the > > distutils don't do that automatically for you. > >Ah - OK. > >Thank you! > > > Of course, if you use setuptools to create your sdist, and the data > > files are in revision control, they'll be included > > automatically. > >Is there a chance of setuptools recognizing other systems than SVN and cvs? > >I use Mercurial for all my own projects, and the setuptools don't seem to >recognize that yet. The first Google result for "setuptools mercurial" is: http://pypi.python.org/pypi/hg.setuptools So, yes. ;-) From jannis at leidel.info Wed Mar 11 16:17:04 2009 From: jannis at leidel.info (Jannis Leidel) Date: Wed, 11 Mar 2009 16:17:04 +0100 Subject: [Distutils] data in a source distribution? In-Reply-To: <20090311150141.687203A4116@sparrow.telecommunity.com> References: <200903102213.14925.arne_bab@web.de> <20090311012505.53C0C3A40A5@sparrow.telecommunity.com> <200903111248.11316.arne_bab@web.de> <20090311150141.687203A4116@sparrow.telecommunity.com> Message-ID: <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> >> > Of course, if you use setuptools to create your sdist, and the data >> > files are in revision control, they'll be included >> > automatically. >> >> Is there a chance of setuptools recognizing other systems than SVN >> and cvs? >> >> I use Mercurial for all my own projects, and the setuptools don't >> seem to >> recognize that yet. > > The first Google result for "setuptools mercurial" is: > > http://pypi.python.org/pypi/hg.setuptools > > So, yes. ;-) Please note "hg.setuptools" is quite old and won't work with Mercurial > 1.0. Since the author told me he wouldn't update it anymore I created http://pypi.python.org/pypi/setuptools_hg. Cheers, Jannis From jim at zope.com Wed Mar 11 16:39:55 2009 From: jim at zope.com (Jim Fulton) Date: Wed, 11 Mar 2009 11:39:55 -0400 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: <49B7C567.3090906@palladion.com> References: <49B6F648.8040208@palladion.com> <49B7C567.3090906@palladion.com> Message-ID: On Mar 11, 2009, at 10:06 AM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jim Fulton wrote: >> It might be good to take this off list. > > If you think so, OK. > >> Are you using a Python with anything in site-packages? > > Only the setuptools 0.6c9 egg. > >> It seem rather odd that, for you, it seems to be importing test >> modules that are unrelated to zc.recipe.cmmi. > > My thoughts exactly, I should have spotted this sooner. The errors are due to a bug in the test runner that causes spurious failures when a buildout directly contains packages without an intermediate "src" directory and when it has it's own eggs directory rather than using a shared eggs directory. I've added a src directory and the tests pass under Python 2.4 and 2.5. It looks like they'll pass under 2.6 once the deprecation errors are dealt with. I'll raise the priority of fixing the test runner. Jim -- Jim Fulton Zope Corporation From regebro at gmail.com Wed Mar 11 16:50:32 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 11 Mar 2009 16:50:32 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. Message-ID: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> When porting setuptools to Python 3, I recently looked into doing the 3.0 development a bit nicer, by being able to run 2to3 as a part of the installation or test commands. However, this turns out to be impossible, because every part of setuptools assumes that setuptools is available and working. This also means, even under Python 2.x development, that should you break for example the dist module a bit too much, setuptools stops working, and you can't even run the tests. Therefore, one thing that should be done is to make setuptools not depend on setuptools. Building, installoing, testing, should require disutils only. I'm not going to work on this, at least not until after the summit. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From arne_bab at web.de Wed Mar 11 17:45:55 2009 From: arne_bab at web.de (Arne Babenhauserheide) Date: Wed, 11 Mar 2009 17:45:55 +0100 Subject: [Distutils] data in a source distribution? In-Reply-To: <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> References: <200903102213.14925.arne_bab@web.de> <20090311150141.687203A4116@sparrow.telecommunity.com> <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> Message-ID: <200903111746.00014.arne_bab@web.de> Am Mittwoch 11 M?rz 2009 16:17:04 schrieb Jannis Leidel: > > http://pypi.python.org/pypi/hg.setuptools > > > > So, yes. ;-) > > Please note "hg.setuptools" is quite old and won't work with Mercurial > > 1.0. Since the author told me he wouldn't update it anymore I > created http://pypi.python.org/pypi/setuptools_hg. Could you ask the author, if he could add a link into hg.setuptools which points users to setuptools_hg? He said he won't update it anymore, and users should know that. Best wishes, Arne -- -- Ein W?rfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln. -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From pje at telecommunity.com Wed Mar 11 19:07:58 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Mar 2009 14:07:58 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.co m> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> Message-ID: <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> At 04:50 PM 3/11/2009 +0100, Lennart Regebro wrote: >When porting setuptools to Python 3, I recently looked into doing the >3.0 development a bit nicer, by being able to run 2to3 as a part of >the installation or test commands. However, this turns out to be >impossible, because every part of setuptools assumes that setuptools >is available and working. This also means, even under Python 2.x >development, that should you break for example the dist module a bit >too much, setuptools stops working, and you can't even run the tests. > >Therefore, one thing that should be done is to make setuptools not >depend on setuptools. Building, installoing, testing, should require >disutils only. Distutils doesn't have a "test" command... let alone an egg_info command. It might be easier to have a "build_py" option to run 2to3 on stuff in the build directory, and a "test" variant that runs against the build tree instead of the source tree... and runs under Python 3 instead of 2. It's easier to see (at least for me) how something like that could potentially work, than how to make setuptools not depend on itself. From regebro at gmail.com Wed Mar 11 19:16:56 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 11 Mar 2009 19:16:56 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> On Wed, Mar 11, 2009 at 19:07, P.J. Eby wrote: > Distutils doesn't have a "test" command... ?let alone an egg_info command. Well.... does setuptools? It's all command extensions, isn't it? > It might be easier to have a "build_py" option to run 2to3 on stuff in the > build directory, and a "test" variant that runs against the build tree > instead of the source tree... ?and runs under Python 3 instead of 2. Yes, that's the idea, but I can't do that as long as the test option depends on setuptools, as it will fail to import, since it's 2.x code. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From pje at telecommunity.com Wed Mar 11 21:47:40 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Mar 2009 16:47:40 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com > References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> Message-ID: <20090311204528.649E73A40A5@sparrow.telecommunity.com> At 07:16 PM 3/11/2009 +0100, Lennart Regebro wrote: >On Wed, Mar 11, 2009 at 19:07, P.J. Eby wrote: > > Distutils doesn't have a "test" command... let alone an egg_info command. > >Well.... does setuptools? It's all command extensions, isn't it? > > > It might be easier to have a "build_py" option to run 2to3 on stuff in the > > build directory, and a "test" variant that runs against the build tree > > instead of the source tree... and runs under Python 3 instead of 2. > >Yes, that's the idea, but I can't do that as long as the test option >depends on setuptools, as it will fail to import, since it's 2.x code. Why not just invoke: python3 -m unittest somemodule.somesuite Then there are no dependencies on distutils or setuptools. From regebro at gmail.com Wed Mar 11 22:14:07 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 11 Mar 2009 22:14:07 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090311204528.649E73A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> On Wed, Mar 11, 2009 at 21:47, P.J. Eby wrote: > Why not just invoke: > > ?python3 -m unittest somemodule.somesuite > > Then there are no dependencies on distutils or setuptools. Because it needs to run 2to3 on the code first. Also, python3 setup.py install will fail, becuase the installation depends on setuptools, and then I need to do build_py_2to3 on the code, which isn't going to work if the install program depends on setuptools, as it hasn't been converted to Python3 yet. The alternative, which seems less likely, is to rewrite setuptools to run under Python 2.3, 2.4, 2.5, 2.6 and 3.0 without using 2to3. :-D Or of course, continue having a separate source distribution for Python 3 and a bit complicated development process, which is survivable, but I don't think it's a good long term strategy. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From rudd-o at rudd-o.com Wed Mar 11 22:17:50 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 16:17:50 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B7D003.80305@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> Message-ID: <200903111617.50953.rudd-o@rudd-o.com> Gerry, What you have said is interesting. Note that you're using underscores in the release number (against F packaging policy), and the general scheme for devising the release number is also against Fedora packaging policy too (which is highly likely to cause problems in RHEL down the road too). Sure, it works too, but it'd be good if we could stick to the published policies which are also known to work and everybody adheres to them already. > Hi Manuel, > Let me share with you what we had implemented and the POLICY that we > put in place with regards to VERSION-RELEASE strings. We did not look > at this from egg standpoint as we just distribute RPMS. > > I sent this to Tarek a while ago: > What I had implemented seems to have the VERSION-RELEASE string issue > solved using a 'workaround' and a manual policy for the moment. It > would of course be better if distutils just did this automatically but > this is at least working. Tarballs, rpms have the same designations and > all the lexical ordering is correct so that even for pre-release > versioning the rpms update each other correctly. > > #WORKAROUND: > What I did was in setup.py added: > ======================== > version = '5.0.0' > release = '0_rc3_00001681' > > if sys.argv[1] != 'bdist_rpm': > version = version + '-' + release > ======================== > > In setup.cfg added: > ======================== > [bdist_rpm] > # release must exactly match 'release' as set in setup.py > release=0_rc3_00001681 > ================== > # END WORKAROUND > > > > And then I setup the following policy regarding VERSION-RELEASE naming: > (this example uses bzr as source code control system but you can > substitute svn, rcs, or any other type of sccs) > > Development code: > version="5.0.0" > release="0_00012345" (where 00012345 is the revision number from > bzr in ZERO-PADDED FIELD OF 8) > tarball: foo-5.0.0-0_00012345.tar.gz > rpm: foo-5.0.0-0_00012345.noarch.rpm > > Alpha code: > version="5.0.0" > release="0_alpha1_00123456" > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm > > Beta code: > version="5.0.0" > release="0_beta1_00234567" > tarball: foo-5.0.0-0_beta1_00234567.tar.gz > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm > > Release Candidate code: > version="5.0.0" release="0_rc1" > tarball: foo-5.0.0-0_rc1.tar.gz > rpm: foo-5.0.0-0_rc1.noarch.rpm > > Release Candidate code fix: > version="5.0.0" > release="0_rc1_00345678" > tarball: foo-5.0.0-0_rc1_00345678.tar.gz > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm > > Final Release code: > version="5.0.0" > release="1" > tarball: foo-5.0.0-1.tar.gz > rpm: foo-5.0.0-1.noarch.rpm > > Notice that lexical ordering is proper in all cases. Even where the > alpha, beta, rc releases may be followed by a bzr fix revision. > The ensures that both tarballs and packages such as rpm will contain the > same VERSION-RELEASE designations. > > Also, the use of UNDERSCORES in the 'release' string assures that there > is no ambiguity with regard to parsing VERSION-RELEASE string > combination or full package names which have hyphens in specific positions. > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Reality always seems harsher in the early morning. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From pje at telecommunity.com Wed Mar 11 22:56:53 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Mar 2009 17:56:53 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.co m> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> Message-ID: <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> At 10:14 PM 3/11/2009 +0100, Lennart Regebro wrote: >On Wed, Mar 11, 2009 at 21:47, P.J. Eby wrote: > > Why not just invoke: > > > > python3 -m unittest somemodule.somesuite > > > > Then there are no dependencies on distutils or setuptools. > >Because it needs to run 2to3 on the code first. That's what the build_py option (to run 2to3 on the copied .py files) would be for. >Also, python3 setup.py install will fail, becuase the installation >depends on setuptools, and then I need to do build_py_2to3 on the >code, which isn't going to work if the install program depends on >setuptools, as it hasn't been converted to Python3 yet. I think we must be entirely misunderstanding each other here. I don't see where the problem is, exactly. Assuming that 2to3 can convert some version of 2.x-compatible setuptools to a valid 3.x-compatible version, self-dependency would not be an issue. Likewise, using 2.x to manage the process of converting and then invoking tests in 3.x on the converted version, doesn't seem like a problem to me either. You just run separate interpreters for anything that needs 3.x to run, while leaving the primary build operations in 2.x. If you're trying to say that you want a build process that can run without there being a 2.x interpreter present, but starts with the same source code base, I don't see how that's possible without having some sort of alternative setup3.py or other .py scripts present, to do the conversion. >The alternative, which seems less likely, is to rewrite setuptools to >run under Python 2.3, 2.4, 2.5, 2.6 and 3.0 without using 2to3. :-D > >Or of course, continue having a separate source distribution for >Python 3 and a bit complicated development process, which is >survivable, but I don't think it's a good long term strategy. You can have a single source distribution, you'll just need a different way to access the 3.x stuff from the 2.x stuff, like a 'setup3.py' or something. (This might be a useful idea in general, not just for setuptools.) From regebro at gmail.com Wed Mar 11 23:04:13 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 11 Mar 2009 23:04:13 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> On Wed, Mar 11, 2009 at 22:56, P.J. Eby wrote: > If you're trying to say that you want a build process that can run without > there being a 2.x interpreter present, but starts with the same source code > base, I don't see how that's possible without having some sort of > alternative setup3.py or other .py scripts present, to do the conversion. It's not a problem in general, the probleme here is that setuptools setup.py depends on setuptools, causing a catch22 situation, where you need to convert setuptools to Python3, before you can run the installer install it, and therefore you can't run the installer to convert to Python 3. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From greno at verizon.net Thu Mar 12 01:13:33 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 20:13:33 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903111617.50953.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> <200903111617.50953.rudd-o@rudd-o.com> Message-ID: <49B853AD.5090807@verizon.net> Manuel Amador (Rudd-O) wrote: > Gerry, > > > What you have said is interesting. Note that you're using underscores > in the release number (against F packaging policy), and the general > scheme for devising the release number is also against Fedora > packaging policy too (which is highly likely to cause problems in RHEL > down the road too). Sure, it works too, but it'd be good if we could > stick to the published policies which are also known to work and > everybody adheres to them already. > > > > > Hi Manuel, > > Let me share with you what we had implemented and the POLICY that we > > put in place with regards to VERSION-RELEASE strings. We did not look > > at this from egg standpoint as we just distribute RPMS. > > > > I sent this to Tarek a while ago: > > What I had implemented seems to have the VERSION-RELEASE string issue > > solved using a 'workaround' and a manual policy for the moment. It > > would of course be better if distutils just did this automatically but > > this is at least working. Tarballs, rpms have the same designations and > > all the lexical ordering is correct so that even for pre-release > > versioning the rpms update each other correctly. > > > > #WORKAROUND: > > What I did was in setup.py added: > > ======================== > > version = '5.0.0' > > release = '0_rc3_00001681' > > > > if sys.argv[1] != 'bdist_rpm': > > version = version + '-' + release > > ======================== > > > > In setup.cfg added: > > ======================== > > [bdist_rpm] > > # release must exactly match 'release' as set in setup.py > > release=0_rc3_00001681 > > ================== > > # END WORKAROUND > > > > > > > > And then I setup the following policy regarding VERSION-RELEASE naming: > > (this example uses bzr as source code control system but you can > > substitute svn, rcs, or any other type of sccs) > > > > Development code: > > version="5.0.0" > > release="0_00012345" (where 00012345 is the revision number from > > bzr in ZERO-PADDED FIELD OF 8) > > tarball: foo-5.0.0-0_00012345.tar.gz > > rpm: foo-5.0.0-0_00012345.noarch.rpm > > > > Alpha code: > > version="5.0.0" > > release="0_alpha1_00123456" > > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz > > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm > > > > Beta code: > > version="5.0.0" > > release="0_beta1_00234567" > > tarball: foo-5.0.0-0_beta1_00234567.tar.gz > > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm > > > > Release Candidate code: > > version="5.0.0" release="0_rc1" > > tarball: foo-5.0.0-0_rc1.tar.gz > > rpm: foo-5.0.0-0_rc1.noarch.rpm > > > > Release Candidate code fix: > > version="5.0.0" > > release="0_rc1_00345678" > > tarball: foo-5.0.0-0_rc1_00345678.tar.gz > > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm > > > > Final Release code: > > version="5.0.0" > > release="1" > > tarball: foo-5.0.0-1.tar.gz > > rpm: foo-5.0.0-1.noarch.rpm > > > > Notice that lexical ordering is proper in all cases. Even where the > > alpha, beta, rc releases may be followed by a bzr fix revision. > > The ensures that both tarballs and packages such as rpm will contain the > > same VERSION-RELEASE designations. > > > > Also, the use of UNDERSCORES in the 'release' string assures that there > > is no ambiguity with regard to parsing VERSION-RELEASE string > > combination or full package names which have hyphens in specific > positions. > > > > > > Regards, > > Gerry > > > > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > Reality always seems harsher in the early morning. > > Manuel, Are you on the FPP committee? I looked at the fedora packaging policy and it is rather similar except it has periods where we have underscores. It would be nice if fedora could expand the policy to allow the underscore style as well. Both approaches work just as well. And to me the underscore form is a lot more readable. Also, there isn't a lot of examples in the FPP regarding adding the vcs revision number into the release string. I like the way we do it because it's lexically correct always. Maybe this could be added to FPP? Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From marius at pov.lt Thu Mar 12 01:26:31 2009 From: marius at pov.lt (Marius Gedminas) Date: Thu, 12 Mar 2009 02:26:31 +0200 Subject: [Distutils] Running tests for zc.recipe.cmmi? In-Reply-To: References: <49B6F648.8040208@palladion.com> <49B7C567.3090906@palladion.com> Message-ID: <20090312002631.GN19902@fridge.pov.lt> On Wed, Mar 11, 2009 at 11:39:55AM -0400, Jim Fulton wrote: > I should have spotted this sooner. The errors are due to a bug in the > test runner that causes spurious failures when a buildout directly > contains packages without an intermediate "src" directory and when it > has it's own eggs directory rather than using a shared eggs directory. > > I'll raise the priority of fixing the test runner. This is https://bugs.launchpad.net/zope.testing/+bug/251759 Marius Gedminas -- If all else fails, read the documentation. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From greno at verizon.net Thu Mar 12 01:54:26 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 20:54:26 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B853AD.5090807@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> <200903111617.50953.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> Message-ID: <49B85D42.9020501@verizon.net> Gerry Reno wrote: > Manuel Amador (Rudd-O) wrote: >> Gerry, >> >> >> What you have said is interesting. Note that you're using underscores >> in the release number (against F packaging policy), and the general >> scheme for devising the release number is also against Fedora >> packaging policy too (which is highly likely to cause problems in >> RHEL down the road too). Sure, it works too, but it'd be good if we >> could stick to the published policies which are also known to work >> and everybody adheres to them already. >> >> >> >> > Hi Manuel, >> > Let me share with you what we had implemented and the POLICY that we >> > put in place with regards to VERSION-RELEASE strings. We did not look >> > at this from egg standpoint as we just distribute RPMS. >> > >> > I sent this to Tarek a while ago: >> > What I had implemented seems to have the VERSION-RELEASE string issue >> > solved using a 'workaround' and a manual policy for the moment. It >> > would of course be better if distutils just did this automatically but >> > this is at least working. Tarballs, rpms have the same designations and >> > all the lexical ordering is correct so that even for pre-release >> > versioning the rpms update each other correctly. >> > >> > #WORKAROUND: >> > What I did was in setup.py added: >> > ======================== >> > version = '5.0.0' >> > release = '0_rc3_00001681' >> > >> > if sys.argv[1] != 'bdist_rpm': >> > version = version + '-' + release >> > ======================== >> > >> > In setup.cfg added: >> > ======================== >> > [bdist_rpm] >> > # release must exactly match 'release' as set in setup.py >> > release=0_rc3_00001681 >> > ================== >> > # END WORKAROUND >> > >> > >> > >> > And then I setup the following policy regarding VERSION-RELEASE naming: >> > (this example uses bzr as source code control system but you can >> > substitute svn, rcs, or any other type of sccs) >> > >> > Development code: >> > version="5.0.0" >> > release="0_00012345" (where 00012345 is the revision number from >> > bzr in ZERO-PADDED FIELD OF 8) >> > tarball: foo-5.0.0-0_00012345.tar.gz >> > rpm: foo-5.0.0-0_00012345.noarch.rpm >> > >> > Alpha code: >> > version="5.0.0" >> > release="0_alpha1_00123456" >> > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz >> > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm >> > >> > Beta code: >> > version="5.0.0" >> > release="0_beta1_00234567" >> > tarball: foo-5.0.0-0_beta1_00234567.tar.gz >> > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm >> > >> > Release Candidate code: >> > version="5.0.0" release="0_rc1" >> > tarball: foo-5.0.0-0_rc1.tar.gz >> > rpm: foo-5.0.0-0_rc1.noarch.rpm >> > >> > Release Candidate code fix: >> > version="5.0.0" >> > release="0_rc1_00345678" >> > tarball: foo-5.0.0-0_rc1_00345678.tar.gz >> > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm >> > >> > Final Release code: >> > version="5.0.0" >> > release="1" >> > tarball: foo-5.0.0-1.tar.gz >> > rpm: foo-5.0.0-1.noarch.rpm >> > >> > Notice that lexical ordering is proper in all cases. Even where the >> > alpha, beta, rc releases may be followed by a bzr fix revision. >> > The ensures that both tarballs and packages such as rpm will >> contain the >> > same VERSION-RELEASE designations. >> > >> > Also, the use of UNDERSCORES in the 'release' string assures that there >> > is no ambiguity with regard to parsing VERSION-RELEASE string >> > combination or full package names which have hyphens in specific >> positions. >> > >> > >> > Regards, >> > Gerry >> >> >> >> -- >> >> >> Manuel Amador (Rudd-O) >> Rudd-O.com - http://rudd-o.com/ >> GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ >> >> >> Reality always seems harsher in the early morning. >> >> > Manuel, > Are you on the FPP committee? I looked at the fedora packaging > policy and it is rather similar except it has periods where we have > underscores. It would be nice if fedora could expand the policy to > allow the underscore style as well. Both approaches work just as > well. And to me the underscore form is a lot more readable. > Also, there isn't a lot of examples in the FPP regarding adding the > vcs revision number into the release string. I like the way we do it > because it's lexically correct always. Maybe this could be added to FPP? > > Manuel, Also examples in the fedora packaging policy don't even make sense. It's no wonder no one every gets this right. alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the Department of Redundancy Department This would be far better and make a lot more sense like: alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, using numbers like patch 1 is absolutely meanless to most people, they want to know what tagged release (beta1 tag) and then what revision contains the update patches. OR alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Thu Mar 12 02:19:17 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 20:19:17 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B85D42.9020501@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> Message-ID: <200903112019.20391.rudd-o@rudd-o.com> 1) No, I'm not on the committee. 2) The policy is similar. 3) Both approaches cannot work in parallel because the period is lexicographically distinct to the underscore 5) The policy has been tested and takes advantage of the already-deployed RPM code in tens of millions of machines out there. We do not know what sorts of malfunctions using underscores might cause. 4) The policy *does* make sense especially in the place you quoted it. The release begins with 0.1 and increments to 0.2 0.3 0.4 whenever you change the package (add patches, modify specfile) but continue to use the same source package, which is the purpose of the release tag in the first place -- to mark a new release of the SAME source package. Unfortunately, it requires that when a new beta source package (even one that is lexicographically later than the existing package) is compiled, the release must be bumped to upgrade it. My distutils packages explicitly punt this release bump to the packager so that he has control over which beta packages to push out. And the reason that the release tag always begins with 0. in nonstable releases is pretty dead simple: so that the release package with version 0.9.8-1 (the first official stable release) can be lexicographically higher and is automatically updated with package managers like yum. Guys, amirite? > > Manuel, > > Are you on the FPP committee? I looked at the fedora packaging > > policy and it is rather similar except it has periods where we have > > underscores. It would be nice if fedora could expand the policy to > > allow the underscore style as well. Both approaches work just as > > well. And to me the underscore form is a lot more readable. > > Also, there isn't a lot of examples in the FPP regarding adding the > > vcs revision number into the release string. I like the way we do it > > because it's lexically correct always. Maybe this could be added to FPP? > > Manuel, > Also examples in the fedora packaging policy don't even make sense. > It's no wonder no one every gets this right. > > alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) > alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) > alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) > ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the > Department of Redundancy Department > > This would be far better and make a lot more sense like: > alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, > using numbers like patch 1 is absolutely meanless to most people, they > want to know what tagged release (beta1 tag) and then what revision > contains the update patches. > OR > alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ If you are what you eat, does that mean Euell Gibbons really was a nut? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From zooko at zooko.com Thu Mar 12 02:18:38 2009 From: zooko at zooko.com (zooko) Date: Wed, 11 Mar 2009 19:18:38 -0600 Subject: [Distutils] Packaging: bdist_deb patches In-Reply-To: References: <49B7B62F.40306@verizon.net> Message-ID: <3BA4AE13-CCA5-4331-951F-196D7CAC922F@zooko.com> Tarek: Will the bdist_deb code be based on or copied from the existing stdeb tool? Should people who are interested in bdist_deb be using stdeb right now in order to find bugs in it and learn what they like/ dislike about it? Regards, Zooko From greno at verizon.net Thu Mar 12 02:43:58 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 21:43:58 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903112019.20391.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> Message-ID: <49B868DE.9010108@verizon.net> Manuel Amador (Rudd-O) wrote: > 1) No, I'm not on the committee. > 2) The policy is similar. > 3) Both approaches cannot work in parallel because the period is > lexicographically distinct to the underscore No. Within a project you would have to choose, either the underscore form or the dot form within the 'release'. You could not mix them. And then either form would work just fine. > 5) The policy has been tested and takes advantage of the > already-deployed RPM code in tens of millions of machines out there. > We do not know what sorts of malfunctions using underscores might cause. Yes, and there are plenty of examples of packages that don't follow the policy as well. Probably more than are actually following it. > 4) The policy *does* make sense especially in the place you quoted it. > The release begins with 0.1 and increments to 0.2 0.3 0.4 whenever you > change the package (add patches, modify specfile) but continue to use > the same source package, which is the purpose of the release tag in > the first place -- to mark a new release of the SAME source package. > Unfortunately, it requires that when a new beta source package (even > one that is lexicographically later than the existing package) is > compiled, the release must be bumped to upgrade it. My distutils > packages explicitly punt this release bump to the packager so that he > has control over which beta packages to push out. And the reason that > the release tag always begins with 0. in nonstable releases is pretty > dead simple: so that the release package with version 0.9.8-1 (the > first official stable release) can be lexicographically higher and is > automatically updated with package managers like yum. The policy doesn't make sense in the example I quoted if you read the comments. Look where it says upgrade to 0.9.2beta3 yet the example shows beta2. I can see that what is trying to be done is to add extra numerics just to make the numerics significant and then basically you can ignore the beta, alpha, etc. But this is more confusing than helpful and its not necessary to make things lexically ordered. The goal is proper lexical ordering so that rpm/yum is happy and packages will upgrade correctly. And either form accomplishes this. Regards, Gerry > > Guys, amirite? > > > > > Manuel, > > > Are you on the FPP committee? I looked at the fedora packaging > > > policy and it is rather similar except it has periods where we have > > > underscores. It would be nice if fedora could expand the policy to > > > allow the underscore style as well. Both approaches work just as > > > well. And to me the underscore form is a lot more readable. > > > Also, there isn't a lot of examples in the FPP regarding adding the > > > vcs revision number into the release string. I like the way we do it > > > because it's lexically correct always. Maybe this could be added > to FPP? > > > > Manuel, > > Also examples in the fedora packaging policy don't even make sense. > > It's no wonder no one every gets this right. > > > > alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) > > alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) > > alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) > > ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the > > Department of Redundancy Department > > > > This would be far better and make a lot more sense like: > > alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, > > using numbers like patch 1 is absolutely meanless to most people, they > > want to know what tagged release (beta1 tag) and then what revision > > contains the update patches. > > OR > > alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 > > > > > > Regards, > > Gerry > > > > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > If you are what you eat, does that mean Euell Gibbons really was a nut? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greno at verizon.net Thu Mar 12 02:52:34 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 21:52:34 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B868DE.9010108@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> Message-ID: <49B86AE2.4030608@verizon.net> Manuel, Additionally, from a Python Distutils perspective, Distutils should not be enforcing one distro's policy. There are RPM-based distro's that DO NOT follow fedora's packaging policy. The software should only be concerned with providing for the use of both the 'version' and 'release' strings in all distribution targets. It should not concern itself with HOW those fields are formatted. Regards, Gerry From greno at verizon.net Thu Mar 12 03:16:51 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 22:16:51 -0400 Subject: [Distutils] Packaging: bdist_deb patches In-Reply-To: <3BA4AE13-CCA5-4331-951F-196D7CAC922F@zooko.com> References: <49B7B62F.40306@verizon.net> <3BA4AE13-CCA5-4331-951F-196D7CAC922F@zooko.com> Message-ID: <49B87093.8060804@verizon.net> Zooko, While we're waiting for the 'official' answer from Tarek... I don't think the code is being taken from stdeb. There were 'bdist_deb' patches that were developed/tested/contributed a while ago and just never got into distutils. To the best of my knowledge it is that patch set. Of course let's see what Tarek says but I'm pretty sure that's the set. Regards, Gerry From pje at telecommunity.com Thu Mar 12 03:38:03 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 11 Mar 2009 22:38:03 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.co m> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> Message-ID: <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> At 11:04 PM 3/11/2009 +0100, Lennart Regebro wrote: >On Wed, Mar 11, 2009 at 22:56, P.J. Eby wrote: > > If you're trying to say that you want a build process that can run without > > there being a 2.x interpreter present, but starts with the same source code > > base, I don't see how that's possible without having some sort of > > alternative setup3.py or other .py scripts present, to do the conversion. > >It's not a problem in general, the probleme here is that setuptools >setup.py depends on setuptools, causing a catch22 situation, where you >need to convert setuptools to Python3, before you can run the >installer install it, and therefore you can't run the installer to >convert to Python 3. That's not a catch 22. You simply run a 2.x setup.py with options that cause the conversion to take place before running 3.x over the converted result. Now you have a 3.x version. Heck, you could probably just make a '2to3' command plugin, that converts the program and then runs python3 on the converted setup.py, passing in its remaining arguments, such that e.g.: python setup.py 2to3 [options] test ... Would be the equivalent of running 2to3 with the specified options, then running "python3 setup.py test ..." in the target directory. Again, I'm not seeing where any problem exists here. From kevin at bud.ca Thu Mar 12 03:16:45 2009 From: kevin at bud.ca (Kevin Teague) Date: Wed, 11 Mar 2009 19:16:45 -0700 Subject: [Distutils] data in a source distribution? In-Reply-To: <200903111746.00014.arne_bab@web.de> References: <200903102213.14925.arne_bab@web.de> <20090311150141.687203A4116@sparrow.telecommunity.com> <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> <200903111746.00014.arne_bab@web.de> Message-ID: On Mar 11, 2009, at 9:45 AM, Arne Babenhauserheide wrote: > Am Mittwoch 11 M?rz 2009 16:17:04 schrieb Jannis Leidel: >>> http://pypi.python.org/pypi/hg.setuptools >>> >>> So, yes. ;-) >> >> Please note "hg.setuptools" is quite old and won't work with >> Mercurial >>> 1.0. Since the author told me he wouldn't update it anymore I >> created http://pypi.python.org/pypi/setuptools_hg. > > Could you ask the author, if he could add a link into hg.setuptools > which > points users to setuptools_hg? > > He said he won't update it anymore, and users should know that. I stumbled on the "Googling for setuptools plus Mercurial leads to an outdated plugin" problem the other day. I e-mailed the author and asked him to update the page with a link to setuptools_hg, but haven't gotten response yet. Alternatively, people could give a nod to setuptools_hg with a link in their blog, etc. and eventually this should skew the Google ranking :P http://pypi.python.org/pypi/setuptools_hg From rudd-o at rudd-o.com Thu Mar 12 04:18:55 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 22:18:55 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B868DE.9010108@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> Message-ID: <200903112218.58341.rudd-o@rudd-o.com> > The policy doesn't make sense in the example I quoted if you read the > comments. Look where it says upgrade to 0.9.2beta3 yet the example > shows beta2. I already told you that you need to bump the release to if you want beta3 to upgrade beta2. This is for good reason. > I can see that what is trying to be done is to add extra numerics just > to make the numerics significant and then basically you can ignore the > beta, alpha, etc. But this is more confusing than helpful and its not > necessary to make things lexically ordered. Actually not totally true, b comes after a, so an alpha package with the same release integer as a beta package would be upgraded by the beta package. > > The goal is proper lexical ordering so that rpm/yum is happy and > packages will upgrade correctly. And either form accomplishes this. Yet the policy says dots and the policy applies to over 10000 packages already built in the Fedora and RHEL / CentOS repos. >-- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ And we heard him exclaim As he started to roam: "I'm a hologram, kids, please don't try this at home!'" -- Bob Violence -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Thu Mar 12 04:26:42 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 22:26:42 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B86AE2.4030608@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> Message-ID: <200903112226.43210.rudd-o@rudd-o.com> El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Manuel, > Additionally, from a Python Distutils perspective, Distutils should > not be enforcing one distro's policy. There are RPM-based distro's that > DO NOT follow fedora's packaging policy. Well then, give me one packaging policy that is discordant or incompatible with Fedora's packaging policy. I'm sure you can show us that. Besides, we're not enforcing a distro policy here over all platforms. My patches merely reformat the existing information to make it lexicographically compatible with RPM package managers (which are the SAME across all bdist_rpm distro targets), and that's it. You still get to specify your release numbers if you want to, and the version numbers are intelligently determined based on heuristics but in the case of stable releases, they are just as they used to be before. > The software should only be > concerned with providing for the use of both the 'version' and 'release' > strings in all distribution targets. It should not concern itself with > HOW those fields are formatted. If distutils did only that and we were to reject the distutils patches I wrote, then I would simply be technically unable to build Plone for Fedora or RHEL. In fact, with the current non-patched distutils, nobody could build eggs using bdist_rpm, because many packages present in the cheese shop rely heavily on beta packages and dependencies that specify python-style version numbers. The mission of the patches is to provide a reasonable way to build eggs and other python source drops ensuring a maximum of compatibility with RPM package manager upgrade paths. The patches accomplish this. And the next hurdle -- something I still need to write into setuptools, but I have already scripted in my workbench -- is the ability to auto-detect egg dependencies and take those dependencies into account for the purpose of proper dependency management of auto-built eggs. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Every path has its puddle. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From ivazqueznet at gmail.com Thu Mar 12 04:28:09 2009 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Wed, 11 Mar 2009 23:28:09 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B86AE2.4030608@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> Message-ID: <1236828489.31110.108.camel@ignacio.lan> On Wed, 2009-03-11 at 21:52 -0400, Gerry Reno wrote: > Manuel, > Additionally, from a Python Distutils perspective, Distutils should > not be enforcing one distro's policy. There are RPM-based distro's that > DO NOT follow fedora's packaging policy. Actually, Fedora people have been approached by other distros because they want to adopt most or all of the policies due to its comprehensiveness. But you didn't hear this from me. -- Ignacio Vazquez-Abrams -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From greno at verizon.net Thu Mar 12 04:52:21 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 23:52:21 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903112226.43210.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> <200903112226.43210.rudd-o@rudd-o.com> Message-ID: <49B886F5.9090500@verizon.net> Manuel Amador (Rudd-O) wrote: > El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > > Manuel, > > Additionally, from a Python Distutils perspective, Distutils should > > not be enforcing one distro's policy. There are RPM-based distro's that > > DO NOT follow fedora's packaging policy. > > > Well then, give me one packaging policy that is discordant or > incompatible with Fedora's packaging policy. I'm sure you can show us > that. Ok, Mandriva comes to mind. It's policy is different and allows more flexibility than Fedora's. Besides, policies are meant for humans. And policies change. We don't need distro policies hard-coded into Distutils. That makes for brittle code. > > Besides, we're not enforcing a distro policy here over all platforms. > My patches merely reformat the existing information to make it > lexicographically compatible with RPM package managers (which are the > SAME across all bdist_rpm distro targets), and that's it. You still > get to specify your release numbers if you want to, and the version > numbers are intelligently determined based on heuristics but in the > case of stable releases, they are just as they used to be before. > > > > The software should only be > > concerned with providing for the use of both the 'version' and 'release' > > strings in all distribution targets. It should not concern itself with > > HOW those fields are formatted. > > > If distutils did only that and we were to reject the distutils patches > I wrote, then I would simply be technically unable to build Plone for > Fedora or RHEL. Why? A distro's policy is defined for 'human' behavior for users of that distro. Users of distro X should not be hostage to policies of distro Y. 'version' and 'release' should be formatted by humans for whatever distro is the target. There should be no enforcement of any distro's 'packaging policy' over the fields. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From yanegomi at gmail.com Thu Mar 12 05:32:56 2009 From: yanegomi at gmail.com (Garrett Cooper) Date: Wed, 11 Mar 2009 21:32:56 -0700 Subject: [Distutils] Fixing parallel installs with easy_install / setuptools Message-ID: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> Hello distutils folks, My group at Cisco uses easy_install / setuptools extensively for installing packages with our make system and there are 2 issues with using this with parallel make: 1. Duplicate dependencies cause corruption when installing the same package twice. 2. easy-install.pth is never updated according to the installed packages. Thus when we install 8+ packages and they all depend on python being installed 3. Cross-compilation is impossible (at least with what I've seen -- either that or the folks who hacked the sources before I got to it didn't understand that feeding in the correct variables would ensure that things could cross-compile). Then again many things Python don't cross-compile from what I've seen :(... the interpreter itself is a prime example =\. My goal is to fix these issues and contribute back the fixes, but I want to ensure that I use the best technical solution for the problems I mentioned. About problems 1 + 2: I'm short on time so I'm going to implement a simple locking mechanism around easy-install.pth. I also realize that .pth files are chosen because they are flat files and are included simply from within python with setup.py. Would it make more sense to use a backend database like pysqlite to store the package data though? That would require some reworking with setup.py, but considering that sqlite3 is _already_ included with 2.5+ and it's a public domain licensed piece of OSS software, would it make more sense to store packaging data with a stable system like SQLite, especially when it would make removal a trivial task? My knowledge of SQLite is limited, but I assume that it supports the same level of endian neutral code that other technologies like BDB do. Of course I'm going out on a huge limb in making that assumption, but the only document I found that blatantly disproves my claim straight out is http://groups.google.com/group/wview/browse_thread/thread/6fcc993dd548206c/fef9608843533ef7?#fef9608843533ef7 -- please correct me if I'm totally off-base. Also, please let me know whether or not you disagree with my proposal, and if there's a different method that could be employed which could maximize data integrity, assure ATOMicity, and guarantee that the data is endian neutral and thus portable to multiple platforms. About problem 3: Is the solution simple enough to solve using --build, --host, and/or --target, like configure, or does more gross work need to be done under the covers to make things all work? Thanks! -Garrett From david at ar.media.kyoto-u.ac.jp Thu Mar 12 06:36:21 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 12 Mar 2009 14:36:21 +0900 Subject: [Distutils] Fixing parallel installs with easy_install / setuptools In-Reply-To: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> References: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> Message-ID: <49B89F55.1040004@ar.media.kyoto-u.ac.jp> Garrett Cooper wrote: > Hello distutils folks, > My group at Cisco uses easy_install / setuptools extensively for > installing packages with our make system and there are 2 issues with > using this with parallel make: > 1. Duplicate dependencies cause corruption when installing the > same package twice. > 2. easy-install.pth is never updated according to the installed > packages. Thus when we install 8+ packages and they all depend on > python being installed > 3. Cross-compilation is impossible (at least with what I've seen > -- either that or the folks who hacked the sources before I got to it > didn't understand that feeding in the correct variables would ensure > that things could cross-compile). Then again many things Python don't > cross-compile from what I've seen :(... the interpreter itself is a > prime example =\. > My goal is to fix these issues and contribute back the fixes, but > I want to ensure that I use the best technical solution for the > problems I mentioned. > About problems 1 + 2: > I'm short on time so I'm going to implement a simple locking > mechanism around easy-install.pth. I also realize that .pth files are > chosen because they are flat files and are included simply from within > python with setup.py. Would it make more sense to use a backend > database like pysqlite to store the package data though? That would > require some reworking with setup.py, but considering that sqlite3 is > _already_ included with 2.5+ and it's a public domain licensed piece > of OSS software, would it make more sense to store packaging data with > a stable system like SQLite, especially when it would make removal a > trivial task? Why would using sqlite make remove a trivial task ? I don't see anything that sqlite would solve compared to a file for uninstall. I think the main problem of uninstall is that there are many possible installation locations, and for one package there may be multiple installations. I am afraid I don't see a reliable way to do this without python's help (one 'registry' / python installation). Although note setuptools needs to support python < 2.5 (a lot of platforms do not have python > 2.4, for example). > About problem 3: > Is the solution simple enough to solve using --build, --host, > and/or --target, like configure, or does more gross work need to be > done under the covers to make things all work? > If you want to do it as quickly as possible, then hacking something in distutils may be possible, but options handling is a bit of a pain in distutils (each command is independent and has its own option set). If you want something reliable, the only way is to bypass the build part IMHO (the build_* commands). It is not meant as a general thing, but you may find my project numscons helpful. It adds a scons distutils command, so that you can build your extensions with scons instead of distutils - you then have a sane build system. It is far from perfect, but it can build non trivial codebase on many platforms (windows included), and you can control flags, change compilers, etc... in scons. http://github.com/cournape/numscons/tree/master the code to plug into distutils is ugly and here: http://projects.scipy.org/numpy/browser/trunk/numpy/distutils/command/scons.py) cheers, David From yanegomi at gmail.com Thu Mar 12 07:09:03 2009 From: yanegomi at gmail.com (Garrett Cooper) Date: Wed, 11 Mar 2009 23:09:03 -0700 Subject: [Distutils] Fixing parallel installs with easy_install / setuptools In-Reply-To: <49B89F55.1040004@ar.media.kyoto-u.ac.jp> References: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> <49B89F55.1040004@ar.media.kyoto-u.ac.jp> Message-ID: <364299f40903112309t404fcf21qbb9f2f4d3c1f8021@mail.gmail.com> On Wed, Mar 11, 2009 at 10:36 PM, David Cournapeau wrote: > Garrett Cooper wrote: >> Hello distutils folks, >> ? ? My group at Cisco uses easy_install / setuptools extensively for >> installing packages with our make system and there are 2 issues with >> using this with parallel make: >> ? ? 1. Duplicate dependencies cause corruption when installing the >> same package twice. >> ? ? 2. easy-install.pth is never updated according to the installed >> packages. Thus when we install 8+ packages and they all depend on >> python being installed >> ? ? 3. Cross-compilation is impossible (at least with what I've seen >> -- either that or the folks who hacked the sources before I got to it >> didn't understand that feeding in the correct variables would ensure >> that things could cross-compile). Then again many things Python don't >> cross-compile from what I've seen :(... the interpreter itself is a >> prime example =\. >> ? ? My goal is to fix these issues and contribute back the fixes, but >> I want to ensure that I use the best technical solution for the >> problems I mentioned. >> ? ? About problems 1 + 2: >> ? ? ? ? I'm short on time so I'm going to implement a simple locking >> mechanism around easy-install.pth. I also realize that .pth files are >> chosen because they are flat files and are included simply from within >> python with setup.py. Would it make more sense to use a backend >> database like pysqlite to store the package data though? That would >> require some reworking with setup.py, but considering that sqlite3 is >> _already_ included with 2.5+ and it's a public domain licensed piece >> of OSS software, would it make more sense to store packaging data with >> a stable system like SQLite, especially when it would make removal a >> trivial task? > > Why would using sqlite make remove a trivial task ? I don't see anything > that sqlite would solve compared to a file for uninstall. I think the > main problem of uninstall is that there are many possible installation > locations, and for one package there may be multiple installations. I am > afraid I don't see a reliable way to do this without python's help (one > 'registry' / python installation). > > Although note setuptools needs to support python < 2.5 (a lot of > platforms do not have python > 2.4, for example). > >> ? ? About problem 3: >> ? ? ? ? Is the solution simple enough to solve using --build, --host, >> and/or --target, like configure, or does more gross work need to be >> done under the covers to make things all work? >> > > If you want to do it as quickly as possible, then hacking something in > distutils may be possible, but options handling is a bit of a pain in > distutils (each command is independent and has its own option set). > > If you want something reliable, the only way is to bypass the build part > IMHO (the build_* commands). It is not meant as a general thing, but you > may find my project numscons helpful. It adds a scons distutils command, > so that you can build your extensions with scons instead of distutils - > you then have a sane build system. It is far from perfect, but it can > build non trivial codebase on many platforms (windows included), and you > can control flags, change compilers, etc... in scons. > > http://github.com/cournape/numscons/tree/master > > the code to plug into distutils is ugly and here: > > http://projects.scipy.org/numpy/browser/trunk/numpy/distutils/command/scons.py) Thanks for the pointers. The only thing I'm concerned about is while scons and waf are both Python based, they're still not accepted packages in Python proper. That being said, neither is setuptools. Has there been talk of a standard python packaging tool being integrated into the interpreter suite? Also, has any serious thought been put into maybe taking the package name, producing specific mnemonic based .pth files for the particular package, and just installing this way, e.g.: pexpect -> pexpect.pth nose -> nose.pth etc. I have seen some packages do this and maybe this is the quicker / dirtier route to do this, but it's also the simplest route to go whilst avoiding collisions with packages, from my point of view, and it's not incredibly complex at all. Furthermore, it kind of lends itself to other packaging methods like pkg_install (FreeBSD), pkgconfig, etc etc. This would be good especially because easy_install doesn't allow multiple versions by default... The only real loss is that the interpreter would have to open up a number of .pth files which would potentially slow down the machine because of I/O access, but the number of python packages on a heavily populated system should be under 20~50 I'd think, so the number seems negligible Any more thoughts? -Garrett From david at ar.media.kyoto-u.ac.jp Thu Mar 12 07:13:52 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 12 Mar 2009 15:13:52 +0900 Subject: [Distutils] Fixing parallel installs with easy_install / setuptools In-Reply-To: <364299f40903112309t404fcf21qbb9f2f4d3c1f8021@mail.gmail.com> References: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> <49B89F55.1040004@ar.media.kyoto-u.ac.jp> <364299f40903112309t404fcf21qbb9f2f4d3c1f8021@mail.gmail.com> Message-ID: <49B8A820.1060608@ar.media.kyoto-u.ac.jp> Garrett Cooper wrote: > > Thanks for the pointers. The only thing I'm concerned about is while > scons and waf are both Python based, they're still not accepted > packages in Python proper. That being said, neither is setuptools. > Yes, that's why I meant by "this is not meant as a general thing". I was not sure whether you wanted to solve the problem at large or only for your own application. > Has there been talk of a standard python packaging tool being > integrated into the interpreter suite? > A lot or not really depending on what you mean by packaging tool, but this is a difficult topic. For once, the general rule for a package to be included into python stdlib is to have already a relatively big user-base - which is inherently difficult for a distribution tool. The only way to get a userbase is to be backward compatible, but being backward compatible with distutils means recreating all its flaws (command-based, option handling, mixing metadata and code, etc....). This is the main practical difficulty (besides implementing a good tool, of course - but those tools already exist or could be relatively easily integrated). cheers, David From regebro at gmail.com Thu Mar 12 08:35:16 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 12 Mar 2009 08:35:16 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> On Thu, Mar 12, 2009 at 03:38, P.J. Eby wrote: > That's not a catch 22. ?You simply run a 2.x setup.py with options that > cause the conversion to take place before running 3.x over the converted > result. ?Now you have a 3.x version. How do you run this? What is the command you would use? -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Thu Mar 12 08:38:55 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 12 Mar 2009 08:38:55 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> Message-ID: <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.com> On Thu, Mar 12, 2009 at 08:35, Lennart Regebro wrote: > On Thu, Mar 12, 2009 at 03:38, P.J. Eby wrote: >> That's not a catch 22. ?You simply run a 2.x setup.py with options that >> cause the conversion to take place before running 3.x over the converted >> result. ?Now you have a 3.x version. > > How do you run this? What is the command you would use? Eh, whatever, ignore the question. I'm tired of arguing. I obviously can't make myself understood, and I'm not going to do anything about this before PyCon anyway, and I'm of the opinion that setuptools should be merged into disutils anyway. maybe people like that suggestion better, and then this argument becomes moot. :) -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From tseaver at palladion.com Thu Mar 12 13:47:13 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Mar 2009 08:47:13 -0400 Subject: [Distutils] Releasing zc.buildout Message-ID: <49B90451.7080804@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm doing most of my development these days with Python 2.6, and would like to get a deprecation-warning-free version of zc.buildout released if possible for the projects which use buildout. I would be glad to make the release myself, if desired: just bless me with the PyPI stick and let me know. ;) Thanks! Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJuQRR+gerLs4ltQ4RAg7cAJ9E6B99dOw1oWWK2lP4Dmk6YdO+BgCfdKAu ky5VUwFfP6cwqFziwwxoZBE= =5CBo -----END PGP SIGNATURE----- From jim at zope.com Thu Mar 12 15:54:07 2009 From: jim at zope.com (Jim Fulton) Date: Thu, 12 Mar 2009 10:54:07 -0400 Subject: [Distutils] Releasing zc.buildout In-Reply-To: <49B90451.7080804@palladion.com> References: <49B90451.7080804@palladion.com> Message-ID: On Mar 12, 2009, at 8:47 AM, Tres Seaver wrote: ... > I'm doing most of my development these days with Python 2.6, and would > like to get a deprecation-warning-free version of zc.buildout released > if possible for the projects which use buildout. OK > I would be glad to make the release myself, if desired: just bless me > with the PyPI stick and let me know. ;) That's fine. You are blessed. :) I also blessed you for zc.recipe.cmmi, since you were threatening to fix deprecation warnings there too. Note that I happen to be working on buildout atm and was planning to make a new release soon. I'd be happy to address 2.6 in that release, but I'm happy to have you deal with 2.6. Jim -- Jim Fulton Zope Corporation From zooko at zooko.com Thu Mar 12 16:37:40 2009 From: zooko at zooko.com (zooko) Date: Thu, 12 Mar 2009 09:37:40 -0600 Subject: [Distutils] classifier for setuptools plugins In-Reply-To: References: <200903102213.14925.arne_bab@web.de> <20090311150141.687203A4116@sparrow.telecommunity.com> <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> <200903111746.00014.arne_bab@web.de> Message-ID: <5AD28EE8-51A1-4A14-96E4-98403B3772A9@zooko.com> > I stumbled on the "Googling for setuptools plus Mercurial leads to > an outdated plugin" problem the other day. I e-mailed the author > and asked him to update the page with a link to setuptools_hg, but > haven't gotten response yet. Alternatively, people could give a nod > to setuptools_hg with a link in their blog, etc. and eventually > this should skew the Google ranking :P > > http://pypi.python.org/pypi/setuptools_hg Please add the "Framework :: Setuptools Plugin" classifier to your metadata so that setuptools_hg will appear on this search: http://pypi.python.org/pypi?:action=browse&c=524 There follows a patch that I submitted to recommend this in the setuptools docs. I thought that PJE had already accepted this patch, but it isn't there. PJE: please accept the following patch. Regards, Zooko diff -rN -u old-dw/setuptools.txt new-dw/setuptools.txt --- old-dw/setuptools.txt 2008-08-11 11:38:07.000000000 -0600 +++ new-dw/setuptools.txt 2008-08-11 11:38:07.000000000 -0600 @@ -2384,6 +2384,16 @@ how setuptools' own commands are installed: the setuptools project's setup script defines entry points for them! +If your command would be useful to other Python developers, then consider +uploading it to the `Python Package Index`_. If you do, please include the +Trove Classifier "Framework :: Setuptools Plugin" in the classifiers so that +your command will appear in the list of `packages which are setuptools +plugins`_. + +.. _Python Package Index: http://pypi.python.org/pypi +.. _packages which are setuptools plugins: http://pypi.python.org/ pypi?:action=browse&c=524 + + Adding ``setup()`` Arguments ---------------------------- @@ -2546,6 +2556,17 @@ control system itself. It *may*, however, use ``distutils.log.warn()`` to inform the user of the missing program(s). +Please model your naming conventions and documentation after the other +setuptools revision control plugins: `setuptools_bzr`_, `setuptools_mtn`_,' +`setuptools_darcs`_. Please include the Trove Classifier "Framework :: +Setuptools Plugin" in the classifiers so that your command will appear in the +list of `packages which are setuptools plugins`_. + +.. _setuptools_bzr: http://pypi.python.org/pypi/setuptools_bzr +.. _setuptools_mtn: http://pypi.python.org/pypi/setuptools_mtn +.. _setuptools_darcs: http://pypi.python.org/pypi/setuptools_darcs +.. _packages which are setuptools plugins: http://pypi.python.org/ pypi?:action=browse&c=524 + Subclassing ``Command`` ----------------------- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doc-classifier-and-rcs.diff.txt URL: From tseaver at palladion.com Thu Mar 12 17:01:11 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Mar 2009 12:01:11 -0400 Subject: [Distutils] Releasing zc.buildout In-Reply-To: References: <49B90451.7080804@palladion.com> Message-ID: <49B931C7.1070306@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: > On Mar 12, 2009, at 8:47 AM, Tres Seaver wrote: > ... >> I'm doing most of my development these days with Python 2.6, and would >> like to get a deprecation-warning-free version of zc.buildout released >> if possible for the projects which use buildout. > > OK > >> I would be glad to make the release myself, if desired: just bless me >> with the PyPI stick and let me know. ;) > > > That's fine. You are blessed. :) I also blessed you for > zc.recipe.cmmi, since you were threatening to fix deprecation warnings > there too. > > Note that I happen to be working on buildout atm and was planning to > make a new release soon. I'd be happy to address 2.6 in that release, > but I'm happy to have you deal with 2.6. The tests all pass for me on 2.6, except the bootstrap one which barfs on its own shoelaces because the already-released egg used to do the bootstrapping emits a DeprecationWarning. I was just planning on releasing it as 1.1.2, at which point the tests should pass. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJuTHH+gerLs4ltQ4RApwXAJ9/RkSSrpcuGN1sTkdrYgYn3Kid0QCeKNgW gq0RikVa3mH14R3RAl9k9pE= =ZdJ8 -----END PGP SIGNATURE----- From pje at telecommunity.com Thu Mar 12 17:51:50 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 12 Mar 2009 12:51:50 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.co m> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <20090311180542.E8AC33A40A5@sparrow.telecommunity.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.com> Message-ID: <20090312164933.30B5F3A40A5@sparrow.telecommunity.com> At 08:38 AM 3/12/2009 +0100, Lennart Regebro wrote: >On Thu, Mar 12, 2009 at 08:35, Lennart Regebro wrote: > > On Thu, Mar 12, 2009 at 03:38, P.J. Eby wrote: > >> That's not a catch 22. You simply run a 2.x setup.py with options that > >> cause the conversion to take place before running 3.x over the converted > >> result. Now you have a 3.x version. > > > > How do you run this? What is the command you would use? > >Eh, whatever, ignore the question. I'm tired of arguing. I'm not aware anyone was arguing. I'm simply trying to understand what the problem is. There appears to be some unstated assumptions that we don't share, and I've been trying to find out what they are. I've suggested that perhaps my assumption that both the 2.x and 3.x interpreters are available was the one we don't share, but you didn't comment on that. Another possible assumption we don't share might be that a single source distribution with a single, "bilingual" (biversional?) setup.py is required. (I assume this is not required... but then again, it could perhaps be worked around by having a version-detecting setup.py that then invokes a setup2.py or setup3.py, accordingly.) So I still don't know what the problem is, because you've not exposed all of your assumptions (as far as I can tell), nor corrected any of mine. From eric at swenson.org Thu Mar 12 18:08:49 2009 From: eric at swenson.org (Eric Swenson) Date: Thu, 12 Mar 2009 10:08:49 -0700 Subject: [Distutils] setuptools 0.6c9 for python 2.6 on windows Message-ID: I note that there still isn't a win32 version of the setuptool installer for python 2.6. Is this in the works? For those who do not have VC++ 2008, it is tricky to get setuptools installed. I have built the .exe myself (and could provide it to you if you wanted), but I wondered why, after all this time, and since python 2.6 is the preferred version of python to install these days, why there isn't an installer for windows yet? -- Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Thu Mar 12 18:20:22 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 12 Mar 2009 18:20:22 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090312164933.30B5F3A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.com> <20090312164933.30B5F3A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903121020s3b6054bej9a19d9fd2929c05c@mail.gmail.com> On Thu, Mar 12, 2009 at 17:51, P.J. Eby wrote: > I'm not aware anyone was arguing. OK, wrong word "explaining" then. :) >?I've suggested that > perhaps my assumption that both the 2.x and 3.x interpreters are available > was the one we don't share, but you didn't comment on that. I though I did comment on the question about only Python 3 being available. And easy_install -U will not work for python 3 if you need to use Python 2 for the first step of the installation. And buildout sometimes requires updates to distutils, which currently is done automatically, that won't work. And installation on Python 3 will be a complicated multi-step affair. > ?Another > possible assumption we don't share might be that a single source > distribution with a single, "bilingual" (biversional?) setup.py is required. Well, no, but that is a relatively minor problem. Unless you suggest that we have complety separate setups, with one python 2.x install and setup command, and one python 3 install and setup command. That seems kinda silly, IMO. > So I still don't know what the problem is, because you've not exposed all of > your assumptions (as far as I can tell), nor corrected any of mine. I don't have many assumptions. I just want the setuptools install and tests to work as expected under both python2 and python3. And that means that python3.0 setup.py install should work. And python3.0 setup.py test would be nice too, although it's less important. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From pje at telecommunity.com Thu Mar 12 19:02:31 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 12 Mar 2009 14:02:31 -0400 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <319e029f0903121020s3b6054bej9a19d9fd2929c05c@mail.gmail.co m> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <319e029f0903111116i5fe18e2ne37fe3c75fd93d69@mail.gmail.com> <20090311204528.649E73A40A5@sparrow.telecommunity.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.com> <20090312164933.30B5F3A40A5@sparrow.telecommunity.com> <319e029f0903121020s3b6054bej9a19d9fd2929c05c@mail.gmail.com> Message-ID: <20090312180013.2E1CF3A40A5@sparrow.telecommunity.com> At 06:20 PM 3/12/2009 +0100, Lennart Regebro wrote: >I don't have many assumptions. I just want the setuptools install and >tests to work as expected under both python2 and python3. And that >means that python3.0 setup.py install should work. And python3.0 >setup.py test would be nice too, although it's less important. In which case, either separate source distros are required, or 2to3 will need to be present, and the main setup.py will need to detect python3 and run 2to3 on itself, then execfile the setup script of the newly-generated, python3 version of the same package. Please note that even if you make setuptools "not depend on itself", all you are really doing is *moving* this problem to a different level. Whatever it is that it depends on, will still need to be duplicated for python2 and python3... and what's more, having a way worked out to do this will be of benefit to other packages besides setuptools, especially if setuptools provides some native support for the feature. (Like a standardized "setup.py" and "setup3.py" that delegate to the user's original 'setup2.py'.) From regebro at gmail.com Thu Mar 12 19:21:21 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 12 Mar 2009 19:21:21 +0100 Subject: [Distutils] Setuptools should not depend on setuptools. In-Reply-To: <20090312180013.2E1CF3A40A5@sparrow.telecommunity.com> References: <319e029f0903110850w5ca26866ubdf2710e509444b9@mail.gmail.com> <319e029f0903111414v14221380rbfbb5febb935994a@mail.gmail.com> <20090311215435.B5B5D3A40A5@sparrow.telecommunity.com> <319e029f0903111504w3b5667f2vba284ce10ebf2d7a@mail.gmail.com> <20090312023546.A11BF3A40A5@sparrow.telecommunity.com> <319e029f0903120035s2a4d5229g66832fe73b9b0604@mail.gmail.com> <319e029f0903120038s4a6010e3t64f3947f6302ed23@mail.gmail.com> <20090312164933.30B5F3A40A5@sparrow.telecommunity.com> <319e029f0903121020s3b6054bej9a19d9fd2929c05c@mail.gmail.com> <20090312180013.2E1CF3A40A5@sparrow.telecommunity.com> Message-ID: <319e029f0903121121u3ee0c36t5565c65d31e1d51d@mail.gmail.com> On Thu, Mar 12, 2009 at 19:02, P.J. Eby wrote: > In which case, either separate source distros are required Which I would like to avoid. > or 2to3 will need to be present It is, as it's included with Python. > and the main setup.py will need to detect python3 and > run 2to3 on itself, then execfile the setup script of the newly-generated, > python3 version of the same package. Yup, that's the idea. > Please note that even if you make setuptools "not depend on itself", all you > are really doing is *moving* this problem to a different level. ?Whatever it > is that it depends on, will still need to be duplicated for python2 and > python3... Well, the build_py, for python 2 and the build_py_2to3 commands are already in distutils, so I'm not sure exactly what would need to be duplicated. >?and what's more, having a way worked out to do this will be of > benefit to other packages besides setuptools Well, that's what I'm trying to do. ;) -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From jannis at leidel.info Thu Mar 12 19:34:53 2009 From: jannis at leidel.info (Jannis Leidel) Date: Thu, 12 Mar 2009 19:34:53 +0100 Subject: [Distutils] classifier for setuptools plugins In-Reply-To: <5AD28EE8-51A1-4A14-96E4-98403B3772A9@zooko.com> References: <200903102213.14925.arne_bab@web.de> <20090311150141.687203A4116@sparrow.telecommunity.com> <4C6C2632-2657-4C7E-8C09-575E76DF8E6E@leidel.info> <200903111746.00014.arne_bab@web.de> <5AD28EE8-51A1-4A14-96E4-98403B3772A9@zooko.com> Message-ID: <662F8BF2-0F7D-4FBF-8484-0A14BA96D627@leidel.info> > Please add the "Framework :: Setuptools Plugin" classifier to your > metadata so that setuptools_hg will appear on this search: > > http://pypi.python.org/pypi?:action=browse&c=524 Done, thanks! Best, Jannis From tseaver at palladion.com Thu Mar 12 19:44:07 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Mar 2009 14:44:07 -0400 Subject: [Distutils] Releasing zc.buildout In-Reply-To: <49B931C7.1070306@palladion.com> References: <49B90451.7080804@palladion.com> <49B931C7.1070306@palladion.com> Message-ID: <49B957F7.608@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote: > Jim Fulton wrote: >> On Mar 12, 2009, at 8:47 AM, Tres Seaver wrote: >> ... >>> I'm doing most of my development these days with Python 2.6, and would >>> like to get a deprecation-warning-free version of zc.buildout released >>> if possible for the projects which use buildout. >> OK > >>> I would be glad to make the release myself, if desired: just bless me >>> with the PyPI stick and let me know. ;) > >> That's fine. You are blessed. :) I also blessed you for >> zc.recipe.cmmi, since you were threatening to fix deprecation warnings >> there too. > >> Note that I happen to be working on buildout atm and was planning to >> make a new release soon. I'd be happy to address 2.6 in that release, >> but I'm happy to have you deal with 2.6. > > The tests all pass for me on 2.6, except the bootstrap one which barfs > on its own shoelaces because the already-released egg used to do the > bootstrapping emits a DeprecationWarning. I was just planning on > releasing it as 1.1.2, at which point the tests should pass. I spoke too soon: the 'boostrap' failure looks like what I described, but another failfure is a doctest which barfs due to a major change in exception formatting in Python 2.6. I tried to get ELLIPSIS to hide the differences, but my doctest-fu is to weak: - ------------------------------- 8< --------------------------- File "/home/tseaver/projects/Zope-CVS/zc.buildout-trunk/src/zc/buildout/tests.py", line 1725, in zc.buildout.tests.install_source_dist_with_bad_py Failed example: print system(buildout), # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS Expected: Installing eggs. Getting distribution for 'badegg'. Got badegg 1. Installing bo. Generated script '/sample-buildout/bin/bo'. ... SyntaxError:...'return' outside function... ... SyntaxError:...'return' outside function... Got: Installing eggs. Getting distribution for 'badegg'. Got badegg 1. Installing bo. Generated script '/sample-buildout/bin/bo'. SyntaxError: ("'return' outside function", ('build/bdist.linux-i686/egg/badegg/scripts/one.py', 2, None, 'return 1/n')) SyntaxError: ("'return' outside function", ('/sample-buildout/eggs/tmpSUUYdy/badegg-1-pyN.N.egg/badegg/scripts/one.py', 2, None, 'return 1/n')) - ------------------------------- 8< --------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJuVf3+gerLs4ltQ4RAut1AJ0czxULPGx4E8oiiDvGcUweRzOU4ACgyQi9 Dyk04yDcaPbd3otFN+RbmqU= =mAcA -----END PGP SIGNATURE----- From tseaver at palladion.com Thu Mar 12 19:44:07 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Mar 2009 14:44:07 -0400 Subject: [Distutils] Releasing zc.buildout In-Reply-To: <49B931C7.1070306@palladion.com> References: <49B90451.7080804@palladion.com> <49B931C7.1070306@palladion.com> Message-ID: <49B957F7.608@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote: > Jim Fulton wrote: >> On Mar 12, 2009, at 8:47 AM, Tres Seaver wrote: >> ... >>> I'm doing most of my development these days with Python 2.6, and would >>> like to get a deprecation-warning-free version of zc.buildout released >>> if possible for the projects which use buildout. >> OK > >>> I would be glad to make the release myself, if desired: just bless me >>> with the PyPI stick and let me know. ;) > >> That's fine. You are blessed. :) I also blessed you for >> zc.recipe.cmmi, since you were threatening to fix deprecation warnings >> there too. > >> Note that I happen to be working on buildout atm and was planning to >> make a new release soon. I'd be happy to address 2.6 in that release, >> but I'm happy to have you deal with 2.6. > > The tests all pass for me on 2.6, except the bootstrap one which barfs > on its own shoelaces because the already-released egg used to do the > bootstrapping emits a DeprecationWarning. I was just planning on > releasing it as 1.1.2, at which point the tests should pass. I spoke too soon: the 'boostrap' failure looks like what I described, but another failfure is a doctest which barfs due to a major change in exception formatting in Python 2.6. I tried to get ELLIPSIS to hide the differences, but my doctest-fu is to weak: - ------------------------------- 8< --------------------------- File "/home/tseaver/projects/Zope-CVS/zc.buildout-trunk/src/zc/buildout/tests.py", line 1725, in zc.buildout.tests.install_source_dist_with_bad_py Failed example: print system(buildout), # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS Expected: Installing eggs. Getting distribution for 'badegg'. Got badegg 1. Installing bo. Generated script '/sample-buildout/bin/bo'. ... SyntaxError:...'return' outside function... ... SyntaxError:...'return' outside function... Got: Installing eggs. Getting distribution for 'badegg'. Got badegg 1. Installing bo. Generated script '/sample-buildout/bin/bo'. SyntaxError: ("'return' outside function", ('build/bdist.linux-i686/egg/badegg/scripts/one.py', 2, None, 'return 1/n')) SyntaxError: ("'return' outside function", ('/sample-buildout/eggs/tmpSUUYdy/badegg-1-pyN.N.egg/badegg/scripts/one.py', 2, None, 'return 1/n')) - ------------------------------- 8< --------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJuVf3+gerLs4ltQ4RAut1AJ0czxULPGx4E8oiiDvGcUweRzOU4ACgyQi9 Dyk04yDcaPbd3otFN+RbmqU= =mAcA -----END PGP SIGNATURE----- From marius at pov.lt Thu Mar 12 19:54:41 2009 From: marius at pov.lt (Marius Gedminas) Date: Thu, 12 Mar 2009 20:54:41 +0200 Subject: [Distutils] Fixing parallel installs with easy_install / setuptools In-Reply-To: <364299f40903112309t404fcf21qbb9f2f4d3c1f8021@mail.gmail.com> References: <364299f40903112132n29b823f3vfaf8ef5741f5490b@mail.gmail.com> <49B89F55.1040004@ar.media.kyoto-u.ac.jp> <364299f40903112309t404fcf21qbb9f2f4d3c1f8021@mail.gmail.com> Message-ID: <20090312185441.GB9775@fridge.pov.lt> On Wed, Mar 11, 2009 at 11:09:03PM -0700, Garrett Cooper wrote: > Also, has any serious thought been put into maybe taking the package > name, producing specific mnemonic based .pth files for the particular > package, and just installing this way, e.g.: > > pexpect -> pexpect.pth > nose -> nose.pth > > etc. I have seen some packages do this and maybe this is the quicker / > dirtier route to do this, but it's also the simplest route to go > whilst avoiding collisions with packages, from my point of view, and > it's not incredibly complex at all. Furthermore, it kind of lends > itself to other packaging methods like pkg_install (FreeBSD), > pkgconfig, etc etc. This would be good especially because easy_install > doesn't allow multiple versions by default... The only real loss is > that the interpreter would have to open up a number of .pth files > which would potentially slow down the machine because of I/O access, > but the number of python packages on a heavily populated system should > be under 20~50 I'd think, so the number seems negligible That's a rather optimistic assumption. $ ls /usr/lib/python2.5/site-packages/|wc -l 276 Admittedly, those are all installed with apt-get and not with easy_install. But here's a Zope-3-based Python app: $ wc -l /home/mg/src/schooltool/eggs/easy-install.pth 114 /home/mg/src/schooltool/eggs/easy-install.pth Marius Gedminas -- We have an advanced scalable groupware communication environment (email) -- Alan Cox -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From rudd-o at rudd-o.com Thu Mar 12 22:11:21 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 16:11:21 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B886F5.9090500@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112226.43210.rudd-o@rudd-o.com> <49B886F5.9090500@verizon.net> Message-ID: <200903121611.21771.rudd-o@rudd-o.com> El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Ok, Mandriva comes to mind. It's policy is different and allows more > flexibility than Fedora's. Well, would you be so kind to link the Mandriva policy for us to read about it? > > Besides, policies are meant for humans. And policies change. We don't > need distro policies hard-coded into Distutils. That makes for brittle > code. So what you're saying, in effect, translates to "sorry, man, you cannot build your Plone packages for RPM distributions, nor can anybody else build any sort of pre-release eggs either". Well, sorry, man, that's not acceptable. > Why? A distro's policy is defined for 'human' behavior for users of > that distro. Users of distro X should not be hostage to policies of > distro Y. 'version' and 'release' should be formatted by humans for > whatever distro is the target. There should be no enforcement of any > distro's 'packaging policy' over the fields. Policy schmolicy, the current distutils behavior builds pre-release RPMs that UPGRADE final release packages, which is broken and stupid because distutils ought to be taking care of the impedance mismatch between python-style versioning and RPM-style lexicographical versioning, but it doesn't. I have a solution that works in fedora, rhel and centos, and likely works just as well on other RPM distros including Mandriva and SUSE. Do you have an alternative solution? No? Then let's stop discussing this because what you're saying sounds like a baseless complaint on the grounds of abstract pureness instead of practical results. > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Excellent day to have a rotten day. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Thu Mar 12 23:48:24 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 18:48:24 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903121611.21771.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112226.43210.rudd-o@rudd-o.com> <49B886F5.9090500@verizon.net> <200903121611.21771.rudd-o@rudd-o.com> Message-ID: <49B99138.1010202@verizon.net> Manuel Amador (Rudd-O) wrote: > El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > > > > Ok, Mandriva comes to mind. It's policy is different and allows more > > flexibility than Fedora's. > > > Well, would you be so kind to link the Mandriva policy for us to read > about it? Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal You're failing to keep in mind that Fedora is not the only distro out there that is RPM-based. Mandriva is just one of many other RPM-based distros that have packaging policies different from Fedora. Again, the core software should not enforce one distros packaging policies. If you want to perform some type of check for a particular distro then that should be done through an extension module and there should be a special command line option to trigger it --enforce-fedora-packaging-policy. > > > > > Besides, policies are meant for humans. And policies change. We don't > > need distro policies hard-coded into Distutils. That makes for brittle > > code. > > > So what you're saying, in effect, translates to "sorry, man, you > cannot build your Plone packages for RPM distributions, nor can > anybody else build any sort of pre-release eggs either". Well, sorry, > man, that's not acceptable. Why do you keep saying this? What is preventing you (the human) from filling in the version and release fields with "fedora-compliant strings"? The core software does not have to know anything about fedora. But you do. > > > Why? A distro's policy is defined for 'human' behavior for users of > > that distro. Users of distro X should not be hostage to policies of > > distro Y. 'version' and 'release' should be formatted by humans for > > whatever distro is the target. There should be no enforcement of any > > distro's 'packaging policy' over the fields. > > > Policy schmolicy, the current distutils behavior builds pre-release > RPMs that UPGRADE final release packages, which is broken and stupid > because distutils ought to be taking care of the impedance mismatch > between python-style versioning and RPM-style lexicographical > versioning, but it doesn't. That's a lexical ordering problem. That should be fixed by the human making sure that he declares the version and release with proper strings. > I have a solution that works in fedora, rhel and centos, and likely > works just as well on other RPM distros including Mandriva and SUSE. Your solution LIMITS the version and release strings to ONLY fedora packaging style. Mandriva users don't want that. They want to build RPM according to their own packaging policy for Mandriva but your patch will not let them because it enforces fedora policy. THAT'S WRONG. > Do you have an alternative solution? Yes, I do. Remove all artificial restriction on formatting on the 'version' and 'release' strings. All that's necessary is for both the 'version' and 'release' strings to be available to all the distribution command which it is not at the moment. That's it. Nothing else is necessary. If you want to do some policy enforcing thing, then put it in some type of optional extension called from a special command line option. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Fri Mar 13 02:01:38 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 20:01:38 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B99138.1010202@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903121611.21771.rudd-o@rudd-o.com> <49B99138.1010202@verizon.net> Message-ID: <200903122001.41744.rudd-o@rudd-o.com> > Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal Thanks for providing the link. Which is a PROPOSAL, not a polcy. On a user- editable WIKI. And on top of that, what you say about versioning style is in NO WAY backed by that document. Now we know for a fact that my patch is neither in conflict with Mandriva policy, nor will fail to work there. This is the kind of fact-based exchange I enjoy. > Again, the core software should not enforce one distros packaging > policies. We already established that the Mandriva policy is not in conflict with Mandriva policy or any other policy that you have quoted so far (none, really), so the prescriptive argument you are making has no basis in fact -- It's just dogma. > Why do you keep saying this? What is preventing you (the human) from > filling in the version and release fields with "fedora-compliant > strings"? The core software does not have to know anything about > fedora. But you do. If you need to know: cheese shop eggs come using a particular python policy for naming them, a policy that was defined by setuptools' dependency handling. This python policy is incompatible with RPM lexicographical order. This is, in a nutshell, the source of the problem. Therefore, the bdist_rpm patch needs to adapt the version to an RPM lexicographically valid form. My patches do exactly this -- they merely adapt the bdist_rpm version and release in very specific cases (pre-release packages) which ought not to matter for a distributor making a stable release. In addition to that, my code works across distributions and is not in contradiction to any policy. Without my patches, there needs to be a way to override the version of the package in setup.cfg, and in addition to that, each package in the cheese shop would need to be modified in order to be buildable. That's thousands of eggs. You're invited to talk to all the python egg developers yourself if you disagree with me. Me? I prefer practical solutions, thus the patch. > That's a lexical ordering problem. That should be fixed by the human > making sure that he declares the version and release with proper strings. Well, I'm sorry but the version is not overridable in setup.cfg. So you either use my patch, or chao eggs. > > I have a solution that works in fedora, rhel and centos, and likely > > works just as well on other RPM distros including Mandriva and SUSE. > Your solution LIMITS the version and release strings to ONLY fedora > packaging style. Mandriva users don't want that. FYI: There is nothing in the Mandriva policy supporting this statement. > They want to build RPM according to their own packaging policy for Mandriva but your patch will not let them because it enforces fedora policy. THAT'S WRONG. Again, the policy link you gave provides no basis for your argument. > > > Do you have an alternative solution? > > Yes, I do. Remove all artificial restriction on formatting on the > 'version' and 'release' strings. I have not introduced any artificial restrictions at all. This is just a lie. I'd appreciate it very much if, in the future, you refrain from telling lies about my work. > All that's necessary is for both the > 'version' and 'release' strings to be available to all the distribution > command which it is not at the moment. That's it. Nothing else is > necessary. You are encouraged to write that patch, then make every single egg developer put stuff in their eggs' setup.cfg files. Have fun draining the ocean. > If you want to do some policy enforcing thing, then put > it in some type of optional extension called from a special command line > option. Look, this back-and-forth is pointless. * If you in some capacity have some control on what goes into the distutils repo, then reject the patches. The Fedora/RH people will probably pick them up anyway, because it has value for tens of millions of users out there. * If you are a Mandriva developer, and there is real basis for your argument (despite the policy document you quoted which does not support it), then you are welcome to include a patch to the patch in your Python sources (you already include several patches, so it should not be a problem) which should be a change of a few characters in my patch to confirm to Mandriva policy. * If not, then either produce code that has the same net results as mine, or let's stop this argument. This bickering can be reduced to the following: I have working code that will work on Mandriva too. You have a baseless complaint with a proposal that would require thousands of people to do extra work, and no code to boot. luck, -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Eiffel 65 - Sopra un palco per tutto il mondo Blessed is the man who, having nothing to say, abstains from giving wordy evidence of the fact. -- George Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From brian at vanguardistas.net Fri Mar 13 02:17:56 2009 From: brian at vanguardistas.net (Brian Sutherland) Date: Fri, 13 Mar 2009 02:17:56 +0100 Subject: [Distutils] Adding a sub-command to the setup.py "build" and "develop" commands Message-ID: <20090313011756.GA31927@brian-sutherlands-macbook.local> Hi, I've just released a setuptools/distutils extension that makes the process of compiling translations (i.e. .po -> .mo files) quite automatic. http://pypi.python.org/pypi/van.potomo/ However, one major problem is that to modify the function of the setup.py "build" and "develop" commands one needs to do this in the setup.py: from setuptools import setup, find_packages from van.potomo import develop, build setup( name = "HelloWorld", cmdclass = {'build': build, 'develop': develop}, setup_requires = ["van.potomo"], version = "0.1", packages = find_packages(), ) Meaning that you have to manually install van.potomo before tools like buildout can run the setup.py to figure out the dependencies. Is there any way to make that more automatic, especially so that buildout can have a chance? >From my experience with Debian, there's a "Build Dependencies" field in the control file where one can specify such things. I'm kindof hoping there's a setuptools equivalent? I've searched for a while and most solutions involve monkey patching or too much code in the setup.py. -- Brian Sutherland From greno at verizon.net Fri Mar 13 02:52:48 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 21:52:48 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903122001.41744.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903121611.21771.rudd-o@rudd-o.com> <49B99138.1010202@verizon.net> <200903122001.41744.rudd-o@rudd-o.com> Message-ID: <49B9BC70.4020307@verizon.net> Manuel Amador (Rudd-O) wrote: > > Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal > > > Thanks for providing the link. Which is a PROPOSAL, not a polcy. On a > user-editable WIKI. And on top of that, what you say about versioning > style is in NO WAY backed by that document. That is the ONLY policy there is for Mandriva. So, if you like, then you can say the policy is whatever 'version' and 'release' naming that will work in an RPM spec file. And that's way more flexible than Fedora's policy. And Mandriva is just one example. You have other distros that do not use Fedora naming policy and you have all sorts of derivatives of these distros, none of which are using Fedora packaging policy. > > Now we know for a fact that my patch is neither in conflict with > Mandriva policy, nor will fail to work there. This is the kind of > fact-based exchange I enjoy. It does conflict. See above. > > > Again, the core software should not enforce one distros packaging > > policies. > > > We already established that the Mandriva policy is not in conflict > with Mandriva policy or any other policy that you have quoted so far > (none, really), so the prescriptive argument you are making has no > basis in fact -- It's just dogma. Again, your patch makes changes to 'version' and 'release' based on Fedora naming policy. Not every distro follows Fedora package naming policy. > > > Why do you keep saying this? What is preventing you (the human) from > > filling in the version and release fields with "fedora-compliant > > strings"? The core software does not have to know anything about > > fedora. But you do. > > > If you need to know: cheese shop eggs come using a particular python > policy for naming them, a policy that was defined by setuptools' > dependency handling. This python policy is incompatible with RPM > lexicographical order. This is, in a nutshell, the source of the problem. If that's the problem then fix it where the problem lies. In the egg naming policy in setuptools. > > Therefore, the bdist_rpm patch needs to adapt the version to an RPM > lexicographically valid form. My patches do exactly this -- they > merely adapt the bdist_rpm version and release in very specific cases > (pre-release packages) which ought not to matter for a distributor > making a stable release. In addition to that, my code works across > distributions and is not in contradiction to any policy. Manuel, please put this policy stuff in an extension that will override the core. Not everybody uses eggs or cares about eggs. Or what problems egg people have with RPM. > > Without my patches, there needs to be a way to override the version of > the package in setup.cfg, and in addition to that, each package in the > cheese shop would need to be modified in order to be buildable. That's > thousands of eggs. Again, put this logic in an extension so it doesn't affect the core and call it from a commandline option. > > You're invited to talk to all the python egg developers yourself if > you disagree with me. Me? I prefer practical solutions, thus the patch. > > > > That's a lexical ordering problem. That should be fixed by the human > > making sure that he declares the version and release with proper > strings. > > > Well, I'm sorry but the version is not overridable in setup.cfg. So > you either use my patch, or chao eggs. Again, put the patch logic in an extension called from a commandline option and leave the core alone. > > > > I have a solution that works in fedora, rhel and centos, and likely > > > works just as well on other RPM distros including Mandriva and SUSE. > > > > Your solution LIMITS the version and release strings to ONLY fedora > > packaging style. Mandriva users don't want that. > > > FYI: There is nothing in the Mandriva policy supporting this statement. Yes, there is. Mandriva naming is whatever will work in a spec file. Much less restrictive than Fedora. > > > They want to build RPM according to their own packaging policy for > Mandriva but your patch will not let them because it enforces fedora > policy. THAT'S WRONG. > > > Again, the policy link you gave provides no basis for your argument. See above. > > > > > > Do you have an alternative solution? > > > > Yes, I do. Remove all artificial restriction on formatting on the > > 'version' and 'release' strings. > > > I have not introduced any artificial restrictions at all. This is just > a lie. I'd appreciate it very much if, in the future, you refrain from > telling lies about my work. Manuel, here is part of your patch: + version = self.distribution.get_version() + release = self.release.replace('-','_') + import re ; regex = '[^0-9\.]' + splits = re.split(regex,version,maxsplit=1) + firstnonalnumchar = re.findall(regex,version) + if len(splits) == 1: + pass + else: + version = splits[0] + release = "0." + release + "." + firstnonalnumchar[0] + splits[1] spec_file = [ '%define name ' + self.distribution.get_name(), '%define version ' + self.distribution.get_version().replace('-','_'), - '%define release ' + self.release.replace('-','_'), + '%define version ' + version, + '%define release ' + release, Your patch is enforcing restrictions on 'version' and 'release' that are directly from Fedora. If some other distro user creates 'version' string that happens to match your pattern, his strings will be changed. That's why I said that the better way to do what you want is to leave the core code alone and put your policy logic in an extension that overrides the core and call it from a commandline option. This would be more community friendly and that way you have what you want and you don't impact other distro users. > > > All that's necessary is for both the > > 'version' and 'release' strings to be available to all the distribution > > command which it is not at the moment. That's it. Nothing else is > > necessary. > > > You are encouraged to write that patch, then make every single egg > developer put stuff in their eggs' setup.cfg files. Have fun draining > the ocean. That patch needs written and if necessary I will write it. > > > If you want to do some policy enforcing thing, then put > > it in some type of optional extension called from a special command line > > option. > > Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Fri Mar 13 03:29:19 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 21:29:19 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9BC70.4020307@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122001.41744.rudd-o@rudd-o.com> <49B9BC70.4020307@verizon.net> Message-ID: <200903122129.19895.rudd-o@rudd-o.com> > That is the ONLY policy there is for Mandriva. So, if you like, then > you can say the policy is whatever 'version' and 'release' naming that > will work in an RPM spec file. And that's way more flexible than > Fedora's policy. If the policy is "whatever works", then my patches comply with the Mandriva policy (in other words, they cause bdist_rpm to generate specfiles pursuant to the policy) and there is no need to discuss this any further. > And Mandriva is just one example. You have other distros that do not > use Fedora naming policy and you have all sorts of derivatives of these > distros, none of which are using Fedora packaging policy. Show them to me and I will rewrite my patches to accomodate them. > Again, put the patch logic in an extension called from a commandline > option and leave the core alone. My patches do not touch core distutils. Stop lying about my work. > Yes, there is. Mandriva naming is whatever will work in a spec file. > Much less restrictive than Fedora. ------------------ The core of the issue is simple: * As of today, bdist_rpm generates invalid spec files when supplied with a pre-release python package (egg or standard distutils one). What I mean is that it generates RPMs that would UPGRADE final releases. Which is broken behavior. * My patch fixes this bug in bdist_rpm, and the fix works in mandriva, fedora, centos, and other RPM distributions, due to the fact that the RPM version algorithm is the same in librpm.so everywhere. Now, onto this discussion itself -- let's go meta: I have had enough of your lies and misrepresentations. Now, you had your chance to make a case as to why my patches break anything in Mandriva, and I have conclusively shown they do not break anything there. You have had plenty of chances to make your case, present examples, show code, improve the patches, one-up my work in MANY WAYS. Alas, you haven't done any of that -- all you did was complain about "policy" -- and, frankly, I have worked very hard to make this work and I do not appreciate you chiming in with pointless bitching that has no basis in engineering reasons. So further mails from you will be directed to /dev/null. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: DJ J?rgen presenting Alice Deejay - Better off alone (club vocal mix) One meets his destiny often on the road he takes to avoid it. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 04:51:15 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 23:51:15 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903122129.19895.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122001.41744.rudd-o@rudd-o.com> <49B9BC70.4020307@verizon.net> <200903122129.19895.rudd-o@rudd-o.com> Message-ID: <49B9D833.3050703@verizon.net> Manuel, In RPM the only restriction on 'version' is that it must be made of alphanumeric or period. Ref: http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-tags.html Just for a small test I ran a perfectly legal version pattern, that is allowable in distros other than Fedora, through your patch logic. version = A2 release = 0_rc1 version-release = A2-0_rc1 # this is what we expected to see version-release = -0.0_rc1.A2 # this is what we get instead as strings are modified by patch logic This is why I suggest that you make an extension module triggered by a commandline option so there are no restrictions or unexpected side-effects introduced to non-Fedora distro users. Right now, I'm a big -1 on the patch. If it would be made as an extension triggered by commandline option then I would support it. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 05:00:35 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 23:00:35 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903122300.37492.rudd-o@rudd-o.com> OK, now we're talking. You have given me a test case that makes my patch fail and I will address it after How I Met Your Mother. El Jueves 12 Marzo 2009, Gerry Reno escribi?: > Manuel, > In RPM the only restriction on 'version' is that it must be made of > alphanumeric or period. Ref: > http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-tags.html > Just for a small test I ran a perfectly legal version pattern, that is > allowable in distros other than Fedora, through your patch logic. > > version = A2 > release = 0_rc1 > version-release = A2-0_rc1 # this is what we expected to see > version-release = -0.0_rc1.A2 # this is what we get instead as strings > are modified by patch logic > > > This is why I suggest that you make an extension module triggered by a > commandline option so there are no restrictions or unexpected > side-effects introduced to non-Fedora distro users. > Right now, I'm a big -1 on the patch. If it would be made as an > extension triggered by commandline option then I would support it. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Mix's Law: There is nothing more permanent than a temporary building. There is nothing more permanent than a temporary tax. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From pje at telecommunity.com Fri Mar 13 06:38:28 2009 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 13 Mar 2009 01:38:28 -0400 Subject: [Distutils] Adding a sub-command to the setup.py "build" and "develop" commands In-Reply-To: <20090313011756.GA31927@brian-sutherlands-macbook.local> References: <20090313011756.GA31927@brian-sutherlands-macbook.local> Message-ID: <20090313053610.A3EFF3A4109@sparrow.telecommunity.com> At 02:17 AM 3/13/2009 +0100, Brian Sutherland wrote: > http://pypi.python.org/pypi/van.potomo/ > >However, one major problem is that to modify the function of the >setup.py "build" and "develop" commands one needs to do this in the >setup.py: > > from setuptools import setup, find_packages > from van.potomo import develop, build > > setup( > name = "HelloWorld", > cmdclass = {'build': build, > 'develop': develop}, > setup_requires = ["van.potomo"], > version = "0.1", > packages = find_packages(), > ) > >Meaning that you have to manually install van.potomo before tools like >buildout can run the setup.py to figure out the dependencies. Is there >any way to make that more automatic, especially so that buildout can >have a chance? > > >From my experience with Debian, there's a "Build Dependencies" field in >the control file where one can specify such things. I'm kindof hoping >there's a setuptools equivalent? You can use aliases to do this, sort of. In your setup.cfg, do something like: [aliases] build = vp_build develop = vp_develop And then register these commands in the van.potomo package's entry points (i.e., as vp_build and vp_develop). Unfortunately, this only works for *command-line commands* - it's not a general replacement for the commands, the way cmdclass is. Probably a better solution would be to add the ability in setuptools to specify a *string* in cmdclass, instead of a class, so that this approach could work generally, instead of only on the command line. From rudd-o at rudd-o.com Fri Mar 13 06:45:44 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 00:45:44 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903130045.46626.rudd-o@rudd-o.com> OK, Gerry, I have addressed your concerns. 1) Note that the release number is still squelched in the case of RPMs compiled from SVN checkouts of distutils packages. This is so that alpha, beta or release candidate packages can upgrade SVN checkouts without extra intervention from the packager. Alpha, beta and rc packages do obey the release variable just fine. The whole intention here is to put a sensible release variable and a sensible version variable when no release variable was specified. 2) I have used a relsep variable that you can tweak to use a dot or an elephant if your package policy wants that (note that the Mandrake policy says nothing about that). 3) If anything non-numeric is specified in the release, then you as a packager are indicating that you do not want this automatic behavior and the modifications are automatically skipped. Note that the whole point of the patch is to AVOID this extra work of specifying the release for prerelease packages. 4) The cases you spoke about are addressed. Attached is the patch. Here is a map table with the default release (which happens to be "1": ----------------------- [rudd-o at tobey ~]$ for a in `cat versions` ; do python bdist_rpm.py rewrite $a 1 ; done 1.0-1 1.0-1 1.1-1 1.1-1 1.1a-1 1.1-0.1.a 1.1a2-1 1.1-0.1.a2 1.1b1-1 1.1-0.1.b1 1.1dev-1 1.1-0.0.dev 1.1svn-1 1.1-0.0.svn 1.1dev-r56554-1 1.1-0.0.56554dev 1.1svn-r400-1 1.1-0.0.400svn 1.1svn-r401-1 1.1-0.0.401svn 1.2svn-r34-1 1.2-0.0.34svn 5.5rc2-1 5.5-0.1.rc2 0.9-1 0.9-1 0.9.8-1 0.9.8-1 20080706-1 20080706-1 A2005-1 A2005-1 ABCDEF15-1 ABCDEF15-1 abc45-1 abc45-1 0.9alpha23-1 0.9-0.1.alpha23 A2-1 A2-1 -------------------- Now here is a map table with a release with a non-integer release: ---------------- [rudd-o at tobey ~]$ for a in `cat versions` ; do python bdist_rpm.py rewrite $a 0_1 ; done 1.0-0_1 1.0-0_1 1.1-0_1 1.1-0_1 1.1a-0_1 1.1a-0_1 1.1a2-0_1 1.1a2-0_1 1.1b1-0_1 1.1b1-0_1 1.1dev-0_1 1.1dev-0_1 1.1svn-0_1 1.1svn-0_1 1.1dev-r56554-0_1 1.1dev_r56554-0_1 1.1svn-r400-0_1 1.1svn_r400-0_1 1.1svn-r401-0_1 1.1svn_r401-0_1 1.2svn-r34-0_1 1.2svn_r34-0_1 5.5rc2-0_1 5.5rc2-0_1 0.9-0_1 0.9-0_1 0.9.8-0_1 0.9.8-0_1 20080706-0_1 20080706-0_1 A2005-0_1 A2005-0_1 ABCDEF15-0_1 ABCDEF15-0_1 abc45-0_1 abc45-0_1 0.9alpha23-0_1 0.9alpha23-0_1 A2-0_1 A2-0_1 ------------------ I hope the next note you have about this patch is as constructive as the last one. The intention here is to solve a problem, not to bikeshed. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ If puns were deli meat, this would be the wurst. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist_rpm-rpmversion-lexicalorder.patch Type: text/x-patch Size: 2511 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Fri Mar 13 09:39:48 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 03:39:48 -0500 Subject: [Distutils] Automatic RPM dependencies based on egg metadata In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903130339.51154.rudd-o@rudd-o.com> Based on my earlier work, it brings me much pleasure to contribute the attached patch (which requires the earlier, lexicographical-order patch to be applied first). The following attached file makes distutils automatically add dependencies to the Requires: field in the RPM spec file, if the egg metadata specifies any primary dependency. With this patch, it is no longer needed to manually specify your dependencies in both setup.cfg and requires.txt. DRY FTW! regards, -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Some people have a way about them that seems to say: "If I have only one life to live, let me live it as a jerk." -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist_rpm-autodeps.patch Type: text/x-patch Size: 3152 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 14:28:10 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 09:28:10 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903130045.46626.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> <200903130045.46626.rudd-o@rudd-o.com> Message-ID: <49BA5F6A.8030309@verizon.net> Manuel, Here's what I get with this patch: $ python test2.py rewrite 3.3beta1 4333 3.3beta1-4333 3.3-0.4333.beta1 $ python test2.py rewrite 3.3rc1 4333 3.3rc1-4333 3.3-0.4333.rc1 It is REWRITING the strings when we don't want them rewritten. And that was the point about putting this in an extension called by a commandline arg. Instead of going through all these callesthenics writing obscure code in bdist_rpm.py just write your simpler code into an override module and call it via a commandline option. That way people who work on bdist_rpm in the future don't have to decipher all this Fedora-specific code. Why do you resist doing it this way? Why do you feel you have the right to come in and take over bdist_rpm with Fedora-specific modifications? Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 15:28:36 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 09:28:36 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA5F6A.8030309@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> Message-ID: <200903130928.42700.rudd-o@rudd-o.com> OK, I am at the limit of my patience, so this will be the last mail I will write about this to you, ever. --------- First of all, my work has already been given good reviews by people involved with distutils. --------- Second, I will absolutely not build an extension that must be enabled explicitly by the packager. This is intended to be default behavior. There is good reason for that decision. The current default behavior *breaks RPM package managers like RPM, Yum and Smart*. Let me reiterate that: the current distutils is BRO-KEN. My patches merely fixing this breakage, which is common to ALL distributions. When this gets into distutils, casual packagers won't have to troll the documentation to figure this out, or be baffled when their "beta" RPMs upgrade their stable releases because "Oh, I didn't know I had to pass this --unbreak- me option to bdist_rpm!" There is no engineering reason to make a bug fix optional. This is a bug fix. It must replace the broken behavior. Heeding your baseless advice would defeat the purpose of my work. --------- Third: Any future people looking into distutils have *nothing* to decipher -- the code is documented and the rationale is explained a bit in the code and well in the bug reports and across the Internet in terms of the RPM versioning algorithm. -------- Fourth: The modifications aren't Fedora-specific. They work correctly across all RPM distributions, whether you want to admit it or not. I am not barging in and taking over distutils for Fedora, and for you to frame the discussion in terms of "Fedora vs. The Others" is just corrupt and dishonest. IOW: I REALLY REALLY do not want to hear you lie "But this is Fedoraism" again. And finally, even if some wacko distributor wants to put a BIG ASCII PENIS on the release tag (which, by the way, is a valid string), they can always patch my work out of distutils just as all distributions have been doing all along. Enjoy your broken behavior -- the rest of us actually want good RPM packages. --------- Fifth: you have been given a mechanism to disable the rewriting which you so loathe. No rewriting (thus reverting to the old, BROKEN bdist_rpm behavior) happens if: 1) your package is a final stable release 2) you put a non-numeric character in your setup.cfg's release= config item, such as 2mdk or something like that. This ought to cover any hypothetical policy that is ass-backwards and likes to generate broken prerelease packages that upgrade stable ones. See big ASCII penis comment above, it fits perfectly here. --------- Sixth: you complain about the rewriting: $ python test2.py rewrite 3.3beta1 4333 3.3beta1-4333 ? ? ? ? ? 3.3-0.4333.beta1 How deficient are you that you can't understand how without the rewriting, 3.3beta1 RPM would upgrade 3.3-1 first final release? TRY IT. DO THE LEGWORK. Do what I did, build two packages of the same code, one with beta in the version name, another without it, and see which one upgrades which other. --------- In summary: * bdist_rpm generates broken RPM packages if supplied with pre-release modules. It has always been broken. * My patches FIX this breakage in a manner that is compatible with all RPM distributions out there. This is not a new feature but a BUG FIX. * They do not require extra action by the packager because making a bug fix require manual intervention from the operator would be the epitome of imbecility. * They have been reviewed positively and I have been thanked personally for making them. The prognosis for their inclusion looks rather positive. * I have addressed every single of your complaints to the limit of my abilities, both with code and with detailed, reasoned justifications for the code. * Yet you object to them fiercely, with absolutely NO BASIS in engineering, and only a made-up distro rivalry that is simply false. In addition, on the basis of this patch, I have created new distutils code that auto-detects dependencies based on egg metadata and adds them to the RPM. These patches have already been submitted to the various bug trackers, and without this patch you so much object to, it would have been technically impossible to deliver on that. Before this patch, people had to specify their dependencies twice, once in the requires.txt and another in the setup.cfg file -- a monumental, error-prone task to be done for tens of thousands of eggs -- and now, we have this. If you had your way, we'd be back in the stone age of manually preparing spec files for each and every Python module. OK. You have had repeated chances to make your case, and all you have come up with is this complaint that stinks of crab mentality, evidences a severe lack of technical ability to understand the underlying problem and its importance, and a false allegation that the patch engages in distribution favoritism just because it uses a fucking dot (which is now configurable, but doesn't really impact any distros negatively) for a separator. I have had enough of this. From now on, you will be redacted from the recipient list in future conversations about this topic. I think it's time for you to visit http://www.sadtrombone.com/ . Now have some respect for my time and stop barging in on my inbox. El Viernes 13 Marzo 2009, Gerry Reno escribi?: > Manuel, > Here's what I get with this patch: > $ python test2.py rewrite 3.3beta1 4333 > 3.3beta1-4333 3.3-0.4333.beta1 > $ python test2.py rewrite 3.3rc1 4333 > 3.3rc1-4333 3.3-0.4333.rc1 > > It is REWRITING the strings when we don't want them rewritten. And that > was the point about putting this in an extension called by a commandline > arg. Instead of going through all these callesthenics writing obscure > code in bdist_rpm.py just write your simpler code into an override > module and call it via a commandline option. That way people who work > on bdist_rpm in the future don't have to decipher all this > Fedora-specific code. Why do you resist doing it this way? Why do you > feel you have the right to come in and take over bdist_rpm with > Fedora-specific modifications? > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ ether leak -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 16:27:20 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 11:27:20 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903130928.42700.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> <200903130928.42700.rudd-o@rudd-o.com> Message-ID: <49BA7B58.70303@verizon.net> Manuel, 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the 'version' and 'release' strings. They do stupid things like put version='3.0' and release='rc1' and then wonder why their final release cannot update the release candidate. THIS IS A TRAINING ISSUE. Their poorly planned sequence of release designations is not in lexical order. They should be putting version='3.0' and release='0.rc1' or '0_rc1' and then their final release of version='3.0' and release='1' WILL update their release candidate. AGAIN, this is not something that is broken with 'bdist_rpm' IT IS A TRAINING ISSUE. Packagers need to understand the importance of lexical ordering when using RPM. You cannot make a patch that outlaws stupidity. I vote a big -1 on this patch because it interjects Fedora-specific modifications into bdist_rpm. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 16:56:05 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 10:56:05 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA7B58.70303@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> Message-ID: <200903131056.08524.rudd-o@rudd-o.com> 1) The Python StrictVersion and LooseVersion classes in Setuptools used to validate and compare versions are incompatible with RPM's lexicographical order. Even the setuptools metadata documentation, which is in full force and implemented, says alpha and beta and rc are expressly permitted (and ENCOURAGED) in the version name, and setuptools' dependency resolution actually uses that information. It is clear that the only solution that unbreaks bdist_rpm is version rewriting of prerelease packages. 2) We already know this "fedora-specific" blahblah is false. My patches work fine in other distros and can be disabled easily. This guy can't tell his ass from his head, and he has the audacity to come here with this ignorant chicanery, trying to convince us that the world is flat that the PEPs and the deployed setuptools code do not work the way they are implemented, that "training" is the solution (see above for technical reasons why "training" is not only draining the ocean but technically impossible) and that we're committing a crime against nature by including these patches. I strongly urge those of you with the knowledge and the power, to do the right thing, and ignore this charlatan. El Viernes 13 Marzo 2009, Gerry Reno escribi?: > Manuel, > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > 'version' and 'release' strings. They do stupid things like put > version='3.0' and release='rc1' and then wonder why their final release > cannot update the release candidate. THIS IS A TRAINING ISSUE. Their > poorly planned sequence of release designations is not in lexical > order. They should be putting version='3.0' and release='0.rc1' or > '0_rc1' and then their final release of version='3.0' and release='1' > WILL update their release candidate. AGAIN, this is not something that > is broken with 'bdist_rpm' IT IS A TRAINING ISSUE. Packagers need to > understand the importance of lexical ordering when using RPM. You > cannot make a patch that outlaws stupidity. > > > I vote a big -1 on this patch because it interjects Fedora-specific > modifications into bdist_rpm. > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Repartee is something we think of twenty-four hours too late. -- Mark Twain -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 17:08:49 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 12:08:49 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903131056.08524.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> <200903131056.08524.rudd-o@rudd-o.com> Message-ID: <49BA8511.9080604@verizon.net> Manuel Amador (Rudd-O) wrote: > 1) The Python StrictVersion and LooseVersion classes in Setuptools > used to validate and compare versions are incompatible with RPM's > lexicographical order. Even the setuptools metadata documentation, > which is in full force and implemented, says alpha and beta and rc are > expressly permitted (and ENCOURAGED) in the version name, and > setuptools' dependency resolution actually uses that information. Then this is a 'setuptools' ERROR. Go fix it in setuptools. If setuptools creates lexigraphically stupid version and release strings that are incompatible with RPM then that is a 'setuptools' problem, NOT a bdist_rpm problem. Go fix 'setuptools'. Again, I'm a big -1 on this patch. There is nothing broken in 'bdist_rpm'. But there is obviously something broken in 'setuptools'. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Fri Mar 13 17:41:43 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 11:41:43 -0500 Subject: [Distutils] =?iso-8859-15?q?SOLVED=3A_bdist=5Frpm_and_pre-release?= =?iso-8859-15?q?_python_packages_/_eggs_=28was=3A_pre-release_versioning_?= =?iso-8859-15?q?problems_with_sdist=2C_=09bdist=5Frpm_=2C_bdist=5Fdebian?= =?iso-8859-15?q?=29?= In-Reply-To: <49BA8511.9080604@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> Message-ID: <200903131141.47844.rudd-o@rudd-o.com> I told you to STOP WRITING TO ME. Setuptools IS NOT the cause of the invalid prerelease RPMs, it is the DEVELOPERS OF PYTHON PACKAGES, which are merely following Setuptools policy which is ALREADY DOCUMENTED, CODED AND DEPLOYED ON HUNDREDS OF MILLIONS OF MACHINES AROUND THE GLOBE. "Education" is not a solution when the problem has already acquired these proportions. And we already know you're a big -1. You don't need to keep repeating it, you have made it abundantly transparent that you do not understand the issue at hand, nor have you provided a single line of code or even a workable counterproposal, so it's pretty clear you're full of shit and you vote -1 for bigotry rather than for valid engineering reasons. Now go away. To whoever has had the endurance to resist this continuous crapflood that this Reno dude has been flooding us with: you have my admiration, but I won't continue this pointless discussion anymore. Do whatever you want with the patches, they're yours to butcher, reject or accept, and I will continue building more features (I have already proposed a new one in the python bug tracker) -- I am not gonna let this troll break my stride. Peace out. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ The famous politician was trying to save both his faces. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From distutils at celestial.com Fri Mar 13 18:05:44 2009 From: distutils at celestial.com (Bill Campbell) Date: Fri, 13 Mar 2009 10:05:44 -0700 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA7B58.70303@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> Message-ID: <20090313170544.GA15144@ayn.mi.celestial.com> On Fri, Mar 13, 2009, Gerry Reno wrote: > Manuel, > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > 'version' and 'release' strings. They do stupid things like put > version='3.0' and release='rc1' and then wonder why their final release > cannot update the release candidate. THIS IS A TRAINING ISSUE... This is a pretty good description of the cause of the problem, but doesn't address a solution that may be used in software. I deal mostly with building RPMs used under the OpenPKG portable package management system which generally uses sane naming conventions, at least as far as the release designations while versions depend on the original package versions. The attached bit of code is basically what we use to determine the most recent packages, and works by splitting the version and release into tuples of numeric and non-numeric parts, then comparing these tuples. This is based on a recipe in the O'Reilly Python Cookbook to sort file names containing numerics. This seems to solve most of the problems of sorting RPM packages on version and release, but might fail in places where there are issues of case-sensitivity. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 During times of universal deceit, telling the truth becomes a revolutionary act. --George Orwell -------------- next part -------------- # Class to facilitate sorting RPM packages based on version and # release by splitting the version and release into tuples with # numeric and non-numeric parts. # # This is based on code from the O'Reilly Python Cookbook, which # I don't have handy to cite the page number. # import os, re rpmcmd = '/bin/rpm' _fmt = r"--queryformat='%{NAME}:%{VERSION}:%{RELEASE}:%{ARCH}\n'" _digits = re.compile(r'(\d+)'); class RPM(object): def _ver(self, s): parts = [] for part in s.split('.'): r = _digits.split(part) r[1::2] = [int(p) for p in r[1::2]] parts.append(tuple(r)) return(tuple(parts)) def __init__(self, file=None, name=None, query=None): if not query: if file: cmd = '%s -qp %s %s' % (rpmcmd, _fmt, file) self.file = file else: cmd = '%s -q %s %s' % (rpmcmd, _fmt, name) self.file = 'installed' fh = os.popen(cmd) query = fh.readline().strip() fh.close() self.query = query (self.name, version, release, self.arch) = query.split(':') self._version = version self._release = release self.version = self._ver(version) self.release = self._ver(release) self.ver_rel = (self.version, self.release) def __cmp__(self, other): assert isinstance(other, RPM) return(cmp(self.ver_rel, other.ver_rel)) if __name__ == '__main__': print 'OK'; print rpmcmd print _fmt rpm = RPM(name='python') print rpm.query; print rpm.name print rpm.version print rpm.release print rpm.ver_rel print rpm.arch print rpm.__cmp__(rpm) From greno at verizon.net Fri Mar 13 18:36:46 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 13:36:46 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903131141.47844.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> <200903131141.47844.rudd-o@rudd-o.com> Message-ID: <49BA99AE.0@verizon.net> Manuel, For years I see developers trying to export problems from the source domain over to some other domain because they don't want to fix the problem at its source. And this ends up creating unnecessary dependencies in software which makes it brittle and difficult to evolve the software. What you are proposing is an example of people "throwing things over the fence". This is a bad practice. In this particular instance the SOURCE of the problem is 'setuptools' naming conventions. And therefore the solution needs to be taken in setuptools. By taking problems like this and throwing them over the fence, you create more problems by making the software more tightly coupled, with more dependencies, and that makes software brittle and easily broken, hard to maintain, difficult of evolve. As far as offering alternative solutions to your patch. I've already done that but you want to ignore them. Here's the two solutions: 1. Go fix the problem in 'setuptools'. 2. Put your string rewriter into an extension that overrides bdist_rpm and call it from a special commandline option. This way it's in Distutils but does not impact every user. Those that need it can call it using the commandline option. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 19:18:07 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 13:18:07 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <20090313170544.GA15144@ayn.mi.celestial.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> <20090313170544.GA15144@ayn.mi.celestial.com> Message-ID: <200903131318.20170.rudd-o@rudd-o.com> Bill, Thank you very much for sharing your code with us. I am pretty sure it works quite well because it's more-or-less what RPM does (bar general alphabetical sort of the non-numeric parts of the release). It'd be pretty great if this code had been the foundation for distutils from the start. Regrettably, we need to cope with the few thousand eggs that already follow the existing naming convention based on the Python setuptools implementation, and the widespread deployment of existing setuptools implementations as well. That makes it pretty clear that ascribing the cause of the problem to "bad training only" is misleading at best. Dynamically rewriting the version numbers at RPM biuld time -- exclusively for pre-release packages -- sort of patches over the issue with fairly good results and no need to change a single line of deployed code, a single line of existing eggs, the need for retraining or cumbersome per-package intervention. Now, the process isn't complete. We already have the correct version rewriting technology, and on top of that the auto-dependency technology as well, but there are a few loose ends that we still need to tie, and three things come to my mind now: 1. Pure python modules should be built to install in site-python, so they can be imported everywhere (regardless of architecture, available python interpreter version or distribution) without N*M rebuilds. I already volunteered to write this, which amounts to a two-liner patch. 2. (this is only Fedora-specific and I think it's fixed already) bdist_rpm should auto-create the .pyo files so they have selinux security contexts. 3. - the python compilation miniscript that bdist_rpm uses to install .pyc files fails in a number of packages that contain pseudo-python files (files that have the extension .py but do not really contain python code -- this is prevalent in Plone products) -- the INSTALLED_FILES contains the list of py and pyc files but does not take into account the failure to build some pyc files, thus the RPM build process bombs out because it does not find the auto- specified files at all. In my Plone build tree I have worked around all of these issues successfully by using a custom RPM %install shellscript, so I have real confidence that I can port these shellscripts directly into distutils (they should be at most a three four-liner patches, and of course the fedora-specific fix does not need to go into distutils at all). Oh, and the good news is, I have built Zope and Plone 3.2.2 for (at least) CentOS (and other RPM distros should just work, if not, msg me privately to report it as a bug and I will fix it). They are at: http://yum.rudd-o.com/ together with the scripted build process available there too. And thanks for breathing some common sense in this discussion. El Viernes 13 Marzo 2009, Bill Campbell escribi?: > On Fri, Mar 13, 2009, Gerry Reno wrote: > > Manuel, > > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > > 'version' and 'release' strings. They do stupid things like put > > version='3.0' and release='rc1' and then wonder why their final release > > cannot update the release candidate. THIS IS A TRAINING ISSUE... > > This is a pretty good description of the cause of the problem, > but doesn't address a solution that may be used in software. > > I deal mostly with building RPMs used under the OpenPKG portable > package management system which generally uses sane naming > conventions, at least as far as the release designations while > versions depend on the original package versions. > > The attached bit of code is basically what we use to determine > the most recent packages, and works by splitting the version and > release into tuples of numeric and non-numeric parts, then > comparing these tuples. This is based on a recipe in the > O'Reilly Python Cookbook to sort file names containing numerics. > > This seems to solve most of the problems of sorting RPM packages > on version and release, but might fail in places where there are > issues of case-sensitivity. > > Bill -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Women are more easily and more deeply terrified ... generating more sheer horror than the male of the species. -- Spock, "Wolf in the Fold", stardate 3615.4 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From ziade.tarek at gmail.com Sat Mar 14 08:54:15 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 14 Mar 2009 08:54:15 +0100 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm , bdist_debian) In-Reply-To: <200903131141.47844.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> <200903131141.47844.rudd-o@rudd-o.com> Message-ID: <94bdd2610903140054w4aef07a4q4ff44683a9ade5f9@mail.gmail.com> Please guys stop this now. While this is a complex topic, there's no need to be rude or impolite here. I am -1 on this kind of attitude. If the solutions were so obvious they would be applied for years. Thanks. On Fri, Mar 13, 2009 at 5:41 PM, Manuel Amador (Rudd-O) wrote: > I told you to STOP WRITING TO ME. > > Setuptools IS NOT the cause of the invalid prerelease RPMs, it is the > DEVELOPERS OF PYTHON PACKAGES, which are merely following Setuptools policy > which is ALREADY DOCUMENTED, CODED AND DEPLOYED ON HUNDREDS OF MILLIONS OF > MACHINES AROUND THE GLOBE. "Education" is not a solution when the problem > has already acquired these proportions. > > And we already know you're a big -1. You don't need to keep repeating it, > you have made it abundantly transparent that you do not understand the issue > at hand, nor have you provided a single line of code or even a workable > counterproposal, so it's pretty clear you're full of shit and you vote -1 > for bigotry rather than for valid engineering reasons. > > Now go away. > > To whoever has had the endurance to resist this continuous crapflood that > this Reno dude has been flooding us with: you have my admiration, but I > won't continue this pointless discussion anymore. Do whatever you want with > the patches, they're yours to butcher, reject or accept, and I will continue > building more features (I have already proposed a new one in the python bug > tracker) -- I am not gonna let this troll break my stride. > > Peace out. > -- > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > The famous politician was trying to save both his faces. > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From marius at pov.lt Sat Mar 14 13:26:54 2009 From: marius at pov.lt (Marius Gedminas) Date: Sat, 14 Mar 2009 14:26:54 +0200 Subject: [Distutils] Releasing zc.buildout In-Reply-To: <49B957F7.608@palladion.com> References: <49B90451.7080804@palladion.com> <49B931C7.1070306@palladion.com> <49B957F7.608@palladion.com> Message-ID: <20090314122654.GA27310@fridge.pov.lt> On Thu, Mar 12, 2009 at 02:44:07PM -0400, Tres Seaver wrote: > I spoke too soon: the 'boostrap' failure looks like what I described, > but another failfure is a doctest which barfs due to a major change in > exception formatting in Python 2.6. I tried to get ELLIPSIS to hide the > differences, but my doctest-fu is to weak: Doctests have some caveats. > ------------------------------- 8< --------------------------- > File > "/home/tseaver/projects/Zope-CVS/zc.buildout-trunk/src/zc/buildout/tests.py", > line 1725, in zc.buildout.tests.install_source_dist_with_bad_py > Failed example: > print system(buildout), # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS > Expected: > Installing eggs. > Getting distribution for 'badegg'. > Got badegg 1. > Installing bo. > Generated script '/sample-buildout/bin/bo'. > ... > SyntaxError:...'return' outside function... > ... > SyntaxError:...'return' outside function... > Got: > Installing eggs. > Getting distribution for 'badegg'. > Got badegg 1. > Installing bo. > Generated script '/sample-buildout/bin/bo'. > SyntaxError: ("'return' outside function", > ('build/bdist.linux-i686/egg/badegg/scripts/one.py', 2, None, 'return 1/n')) > > SyntaxError: ("'return' outside function", > ('/sample-buildout/eggs/tmpSUUYdy/badegg-1-pyN.N.egg/badegg/scripts/one.py', > 2, None, 'return 1/n')) > > ------------------------------- 8< --------------------------- Let's look at this again: > Generated script '/sample-buildout/bin/bo'. > ... > SyntaxError:...'return' outside function... does not match > Generated script '/sample-buildout/bin/bo'. > SyntaxError: ("'return' outside function", because of an unfortunate interaction between ELLIPSIS and NORMALIZE_WHITESPACE. NORMALIZE_WHITESPACE converts your pattern into Generated script '/sample-buildout/bin/bo'. ... SyntaxError:... and the matched text to Generated script '/sample-buildout/bin/bo'. SyntaxError:... Then ELLIPSIS has a mismatch because even if you substitute the '...' with an empty string, the pattern wants two spaces where the text has one. I tried to report this as https://bugs.launchpad.net/zope.testing/+bug/126169 once, but was unable to convince Jim that this behaviour trips people up. (Also, I did not try to provide a patch to fix the issue.) HTH, Marius Gedminas -- This sentence contradicts itself -- no actually it doesn't. -- Douglas Hofstadter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From setuptools at bugs.python.org Sat Mar 14 19:08:18 2009 From: setuptools at bugs.python.org (Skip Montanaro) Date: Sat, 14 Mar 2009 18:08:18 +0000 Subject: [Distutils] [issue63] setuptools & pyjamas In-Reply-To: <1237054098.2.0.063169915229.issue63@psf.upfronthosting.co.za> Message-ID: <1237054098.2.0.063169915229.issue63@psf.upfronthosting.co.za> New submission from Skip Montanaro : This is more a reference submission than anything else. I submitted a patch for pyjamas to remove its installer's hard-coding of /usr/share. In my opinion the root for all installed files should be whatever --prefix is (whether explicit or implicit). The pyjamas ticket is . Please have a look at it and add a comment if you so desire. Skip ---------- messages: 251 nosy: skip.montanaro priority: feature status: unread title: setuptools & pyjamas _______________________________________________ Setuptools tracker _______________________________________________ From greno at verizon.net Sun Mar 15 01:42:46 2009 From: greno at verizon.net (Gerry Reno) Date: Sat, 14 Mar 2009 20:42:46 -0400 Subject: [Distutils] Distutils: distclass cmdclass Message-ID: <49BC4F06.70707@verizon.net> Hi all, I've run into a situation where I need to have translations using L10nAppDistribution and we need to also have a special command class. What I'm finding is that I can have one or the other but not both. Does anyone know how to be able to use both distclass and cmdclass at the same time? Regards, Gerry Reno From greno at verizon.net Sun Mar 15 02:35:45 2009 From: greno at verizon.net (Gerry Reno) Date: Sat, 14 Mar 2009 21:35:45 -0400 Subject: [Distutils] Distutils: distclass cmdclass In-Reply-To: <49BC4F06.70707@verizon.net> References: <49BC4F06.70707@verizon.net> Message-ID: <49BC5B71.7020501@verizon.net> Gerry Reno wrote: > Hi all, > I've run into a situation where I need to have translations using > L10nAppDistribution and we need to also have a special command class. > What I'm finding is that I can have one or the other but not both. > Does anyone know how to be able to use both distclass and cmdclass at > the same time? > Right now the only way I've found to do this is to execv to another setup file. Please let me know if this is possible with just one setup(). Thanks. Regards, Gerry From regebro at gmail.com Sun Mar 15 10:31:41 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 15 Mar 2009 10:31:41 +0100 Subject: [Distutils] zc.buildout development catch 22 Message-ID: <319e029f0903150231qb8715dan89640485727da73f@mail.gmail.com> Typical zc.buildout development installs zope.testing to run the tests. Typical zope.testing development uses zc.buildout. As you see, porting this to Python 3 gets well... tricky. ;) So the question (mostly to Jim, I guess) is if there is a way to easily run the tests for either zope.* or zc.buildout without using the zope testrunner et al? -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From jim at zope.com Sun Mar 15 20:39:54 2009 From: jim at zope.com (Jim Fulton) Date: Sun, 15 Mar 2009 15:39:54 -0400 Subject: [Distutils] zc.buildout development catch 22 In-Reply-To: <319e029f0903150231qb8715dan89640485727da73f@mail.gmail.com> References: <319e029f0903150231qb8715dan89640485727da73f@mail.gmail.com> Message-ID: <56D12089-8544-4764-988A-5AD24F013E97@zope.com> On Mar 15, 2009, at 5:31 AM, Lennart Regebro wrote: > Typical zc.buildout development installs zope.testing to run the > tests. > Typical zope.testing development uses zc.buildout. > > As you see, porting this to Python 3 gets well... tricky. ;) Not necessarily. If you wanted to deal with both together you could use a develop egg of zope.testing in the buildout working directory. This may not be elegant, but it's workable. :) > So the question (mostly to Jim, I guess) is if there is a way to > easily run the tests for either zope.* or zc.buildout without using > the zope testrunner et al? It should be possible to run the zc.buildout tests without the test runner. Zope packages that use test layers will need the Zope test runner. Jim -- Jim Fulton Zope Corporation From regebro at gmail.com Sun Mar 15 20:54:07 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 15 Mar 2009 20:54:07 +0100 Subject: [Distutils] zc.buildout development catch 22 In-Reply-To: <56D12089-8544-4764-988A-5AD24F013E97@zope.com> References: <319e029f0903150231qb8715dan89640485727da73f@mail.gmail.com> <56D12089-8544-4764-988A-5AD24F013E97@zope.com> Message-ID: <319e029f0903151254p3b3f3823o391cc9fae0a6a44@mail.gmail.com> On Sun, Mar 15, 2009 at 20:39, Jim Fulton wrote: > It should be possible to run the zc.buildout tests without the test runner. Possibly, but not without zope.testing, it seems. I've put it on hold for the time being, I'm not going to have time to work on it until at least after PyCon. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From fadhley.salim at uk.calyon.com Mon Mar 16 16:35:07 2009 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Mon, 16 Mar 2009 15:35:07 -0000 Subject: [Distutils] Using UNC paths as arguments to --find-links Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483FC8@MXCU10MX1.MSX.CIB> I'd like to do something like this: http://python.pastebin.com/m283c0376 Basically, I'm invoking the easy_install command from a script. Where I specify a number ofn arguments to the --find-links option, some of which will be web-servers and at least one will be a local filesystem path, or possibly a network path un UNC format. It seems to work just fine when I do not provide a --find-links option (in this case it just picks up the defaults from distutils.cfg), when I use the option all I get is: Traceback (most recent call last): File "D:\workspace\pythonscripts_trunk\javapy_egg\Scripts\test_javapy.py", line 20, in ? result = pyproxy.requireEgg( eggspec , True, hosts ) File "d:\workspace\pythonscripts_trunk\javapy_egg\src\calyon\javapy\pyproxy.p y", line 141, in requireEgg pkg_resources.require(eggname) File "d:\python24\lib\site-packages\setuptools-0.6c9-py2.4.egg\pkg_resources. py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "d:\python24\lib\site-packages\setuptools-0.6c9-py2.4.egg\pkg_resources. py", line 524, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: myproject==trunk-99 Can somebody confirm the correct way to do this? For example do I use Windows or UNIX slashes? What character must be used to seperate multiple URLs? I'm using setuptools 0.6c9 on Windows32 Thansk -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit? des Etablissements de Cr?dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From fadhley.salim at uk.calyon.com Wed Mar 18 18:57:50 2009 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Wed, 18 Mar 2009 17:57:50 -0000 Subject: [Distutils] zc.buildout development catch 22 References: <319e029f0903150231qb8715dan89640485727da73f@mail.gmail.com><56D12089-8544-4764-988A-5AD24F013E97@zope.com> <319e029f0903151254p3b3f3823o391cc9fae0a6a44@mail.gmail.com> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483FDC@MXCU10MX1.MSX.CIB> I'd like to do something like this: http://python.pastebin.com/m283c0376 Basically, I'm invoking the easy_install command from a script. Where I specify a number ofn arguments to the --find-links option, some of which will be web-servers and at least one will be a local filesystem path, or possibly a network path un UNC format. It seems to work just fine when I do not provide a --find-links option (in this case it just picks up the defaults from distutils.cfg), when I use the option all I get is: Traceback (most recent call last): File "D:\workspace\pythonscripts_trunk\javapy_egg\Scripts\test_javapy.py", line 20, in ? result = pyproxy.requireEgg( eggspec , True, hosts ) File "d:\workspace\pythonscripts_trunk\javapy_egg\src\calyon\javapy\pyproxy.p y", line 141, in requireEgg pkg_resources.require(eggname) File "d:\python24\lib\site-packages\setuptools-0.6c9-py2.4.egg\pkg_resources. py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "d:\python24\lib\site-packages\setuptools-0.6c9-py2.4.egg\pkg_resources. py", line 524, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: myproject==trunk-99 Can somebody confirm the correct way to do this? For example do I use Windows or UNIX slashes? What character must be used to seperate multiple URLs? I'm using setuptools 0.6c9 on Windows32 -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit? des Etablissements de Cr?dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From wkerzendorf at googlemail.com Thu Mar 19 01:52:33 2009 From: wkerzendorf at googlemail.com (Wolfgang Kerzendorf) Date: Thu, 19 Mar 2009 11:52:33 +1100 Subject: [Distutils] binary executable in package Message-ID: <195838a20903181752v7dc75821k7388f0c8b8dbc147@mail.gmail.com> Hello, I'm am distributing a package with binary executables which are compiled on installation. Everything works so far, except that when the binary executables are copied to the destination that they loose the executable flag. Here's the code: -------- #!/usr/bin/env python from distutils.core import setup from imp import find_module from os import system, path import os #This is a list of files to install, and where #(relative to the 'root' dir, where setup.py is) #You could be more specific. #find_module('pyraf') find_module('ephem') find_module('numpy') find_module('scipy') #find_module('SOAPpy') def makeAll(): curdir=os.getcwd() srcDir=os.path.join(os.path.dirname(os.path.abspath(__file__)), 'src/pywcs/extutil/src') os.chdir(srcDir) os.system('make clean') os.system('make') os.chdir(curdir) makeAll() setup(name = "pywcs", version = "0.91", description = "Collection of tools to use wcs information and fit new wcs to images", author = "Wolfgang Kerzendorf", author_email = "wkerzendorf at gmail.com", url = "http://www.mso.anu.edu.au/~wkerzend/", #Name the folder where your packages live: #(If you have other packages (dirs) or modules (py files) then #put them into the package directory - they will be found #recursively.) packages = ['pywcs'], #'package' package must contain files (see list above) #I called the package 'package' thus cleverly confusing the whole issue... #This dict maps the package name =to=> directories #It says, package *needs* these files. package_dir={'':'src'}, package_data = {'pywcs' : ["conf.d/sex.d/*","extutil/src/*","extutil/starmatch_new","extutil/finalmatch"] }, #'runner' is in the root. scripts = ["addwcs"] #long_description = """Really long text here.""" # #This next part it for the Cheese Shop, look a little down the page. #classifiers = [] ) ------ Thanks for any help Wolfgang -------------- next part -------------- An HTML attachment was scrubbed... URL: From strawman at astraw.com Sun Mar 22 02:41:39 2009 From: strawman at astraw.com (Andrew Straw) Date: Sat, 21 Mar 2009 18:41:39 -0700 Subject: [Distutils] [ANN] stdeb 0.3 released Message-ID: <49C59753.4030303@astraw.com> The latest version of stdeb, the automated Debian (and Ubuntu) package maker, is available for download. Stdeb takes your Python source (either a tarball or an a directory with a setup.py script) and turns it into a Debian source package. From there, it is trivial to turn it into a .deb file. This version adds support for Ubuntu Jaunty and python2.6 in Debian experimental by passing the "--install-layout=deb" option to "python setup.py install". Also in this release is a bugfix to the autofind-depends feature. The homepage: http://github.com/astraw/stdeb Download: http://pypi.python.org/pypi/stdeb/0.3 From ianb at colorstudy.com Sun Mar 22 20:10:49 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 22 Mar 2009 14:10:49 -0500 Subject: [Distutils] PyPI mirroring and Changing the .egg-info PEPs Message-ID: Suggestions: Instead of http://pypi.python.org/mirrors how about http://mirrors.pypi.python.org (or pypimirrors.python.org, etc) -- then the mirror list can be hosted separately, and the likelihood of both the mirrors and the main PyPI server being down are much lower. Admittedly, this makes it harder for other indexes to provide their own mirrors, and the location is just entirely magic. But it seems like a useful magic. In the egg-info PEP: "They both add other files in the `EGG-INFO` or `.egg-info` directory, and create or modify `.pth` files. `pip` also creates one `.pth` file per installed package, which may lead to slow initialisation of Python." I don't believe this is the case. pip zip creates one .pth file for each zip file, but that's different. pip installs packages flat. Unless you use -e, which invokes setup.py develop, which I believe puts the path in easy-install.pth. Oh, you also use British English; "initialization"! I only notice that because my spell check is complaining about the quotation ;) As mentioned in another email, I think RECORD should hold relative paths (and PJE's "/" suggestion also of course seems reasonable). The paths should be relative to the .egg-info directory. "- get_egg_info(pkg_name) -> path or None Scans all site-packages directories and looks for all `pkg_name.egg-info` directories. Returns the directory path that contains a PKG-INFO that matches `pkg_name` for the `name` metadata. Notice that there should be at most one result. If more than one path matches the pkg_name, a DistutilsError is raised. If the directory is not found, returns None." Currently, if there are multiple paths that might be importable, whichever one is found first with sys.path is imported, and conflicts aren't really considered. raising DistutilsError seems like it changes this. " filename can be any value found in `distutils.sdist.EGG_INFO_FILES`." Is there any reason this should be restricted to those filenames? -- Ian Bicking | http://blog.ianbicking.org From regebro at gmail.com Sun Mar 22 21:44:59 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 22 Mar 2009 21:44:59 +0100 Subject: [Distutils] PyPI mirroring and Changing the .egg-info PEPs In-Reply-To: References: Message-ID: <319e029f0903221344r46a41cedvf6f0027d4e067fa9@mail.gmail.com> On Sun, Mar 22, 2009 at 20:10, Ian Bicking wrote: > Suggestions: > > Instead of http://pypi.python.org/mirrors how about > http://mirrors.pypi.python.org (or pypimirrors.python.org, etc) -- > then the mirror list can be hosted separately, and the likelihood of > both the mirrors and the main PyPI server being down are much lower. +1 > Admittedly, this makes it harder for other indexes to provide their > own mirrors It does? Why? I must have missed something. :) It does make it easy to provide local caching though. If you want to cache pypi you can just make a local name for mirrors.pypi.python.org pointing to a site that provides just one mirror: your own local cache. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Mon Mar 23 16:43:11 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 23 Mar 2009 16:43:11 +0100 Subject: [Distutils] PyPI mirroring and Changing the .egg-info PEPs In-Reply-To: References: Message-ID: <94bdd2610903230843o71a20c1fs2a2733ce450f064f@mail.gmail.com> On Sun, Mar 22, 2009 at 8:10 PM, Ian Bicking wrote: > Suggestions: > > Instead of http://pypi.python.org/mirrors how about > http://mirrors.pypi.python.org (or pypimirrors.python.org, etc) -- > then the mirror list can be hosted separately, and the likelihood of > both the mirrors and the main PyPI server being down are much lower. > Admittedly, this makes it harder for other indexes to provide their > own mirrors, and the location is just entirely magic. ?But it seems > like a useful magic. Good idea ! > > In the egg-info PEP: "They both add other files in the `EGG-INFO` or > `.egg-info` directory, and > create or modify `.pth` files. `pip` also creates one `.pth` file > per installed package, which may lead to slow initialisation of Python." > > I don't believe this is the case. ?pip zip creates one .pth file for > each zip file, but that's different. pip installs packages flat. > Unless you use -e, which invokes setup.py develop, which I believe > puts the path in easy-install.pth. > Ok, I'll correct that statement, > Oh, you also use British English; "initialization"! ?I only notice > that because my spell check is complaining about the quotation ;) OK > > As mentioned in another email, I think RECORD should hold relative > paths (and PJE's "/" suggestion also of course seems reasonable). ?The > paths should be relative to the .egg-info directory. OK > > "- get_egg_info(pkg_name) -> path or None > > ?Scans all site-packages directories and looks for all `pkg_name.egg-info` > ?directories. Returns the directory path that contains a PKG-INFO that matches > ?`pkg_name` for the `name` metadata. Notice that there should be at most > ?one result. If more than one path matches the pkg_name, a DistutilsError > ?is raised. > > ?If the directory is not found, returns None." > > Currently, if there are multiple paths that might be importable, > whichever one is found first with sys.path is imported, and conflicts > aren't really considered. ?raising DistutilsError seems like it > changes this. > Yes, while thinking about this API, I thaught it was a good idea to raise if there's a conflict, to make sure there's a healthy environement. > " ?filename can be any value found in `distutils.sdist.EGG_INFO_FILES`." > > Is there any reason this should be restricted to those filenames? I think the question is more : is it a good thing to have that fixed EGG_INFO_FILES file list in the first place ? The reason was to describe a standardized, immutable, egg.info structure, like what we have for the fields in PKG-INFO I think having it can prevent third-party tools to build their own standards on the way .egg.info is used, and keep this convention in Python. But OTHO it makes it harder to evolve. Cheers, Tarek From kevin at bud.ca Tue Mar 24 02:11:16 2009 From: kevin at bud.ca (Kevin Teague) Date: Mon, 23 Mar 2009 18:11:16 -0700 Subject: [Distutils] PEP 376 and the name .egg-info? Message-ID: <8C3865DA-1EDF-4508-A7F8-8D9EAC286139@bud.ca> I have some confusion over the name .egg-info. From what I understand, Eggs are a packaging format that contain metadata. But if you take a Distutils installed package and put a .egg-info file or directory beside it, that doesn't make the package an Egg, does it? To add to this confusion, I always associate the Egg format with the additional metadata fields provided by setuptools. So when I see .egg- info I always expect that the PKG-INFO will contain an 'install_requires' field, etc. It would be more understandable (to me) if this directory was named .pkg-info, .metadata, .install or something. But then I guess that's a somewhat disruptive change for small gains. An alternative to bridging the disconnect between the setuptools metadata fields and the distutils metadata fields would be to add the setuptools fields to distutils as standard package metadata. Adding those metadata fields hasn't been propsed yet as it's own PEP, has it? I guess that PEP would entail a little more work than just managing a set of fields since PyPI would also need to be updated to display those fields. But it would be really lovely to be able to see the 'install_requires' field when browsing PyPI - and click on links in that field to take you to those pakcages PyPI pages. And as an added bonus if the 'namespace_packages' field was clickable such that the link took you to a list of all packages which participated in that namespace, that'd be pretty sweet! From pje at telecommunity.com Tue Mar 24 03:53:16 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 23 Mar 2009 22:53:16 -0400 Subject: [Distutils] PEP 376 and the name .egg-info? In-Reply-To: <8C3865DA-1EDF-4508-A7F8-8D9EAC286139@bud.ca> References: <8C3865DA-1EDF-4508-A7F8-8D9EAC286139@bud.ca> Message-ID: <20090324025055.37DC93A40A5@sparrow.telecommunity.com> At 06:11 PM 3/23/2009 -0700, Kevin Teague wrote: >I have some confusion over the name .egg-info. From what I understand, >Eggs are a packaging format that contain metadata. But if you take a >Distutils installed package and put a .egg-info file or directory >beside it, that doesn't make the package an Egg, does it? Yes, actually, it does. See http://peak.telecommunity.com/DevCenter/EggFormats for details. >To add to this confusion, I always associate the Egg format with the >additional metadata fields provided by setuptools. So when I see >.egg- info I always expect that the PKG-INFO will contain an >'install_requires' field, etc. That information is maintained separately, as there's more tracked by setuptools than distutils and the formats are not compatible. Also, PKG-INFO is generic for a project as a whole, whereas the egg-specific dependency info can vary by python version and/or platform. >I guess that PEP would entail a little more work than just managing a >set of fields since PyPI would also need to be updated to display >those fields. But it would be really lovely to be able to see the >'install_requires' field when browsing PyPI - and click on links in >that field to take you to those pakcages PyPI pages. And as an added >bonus if the 'namespace_packages' field was clickable such that the >link took you to a list of all packages which participated in that >namespace, that'd be pretty sweet! The formats of this information are documented in EggFormats, and it's certainly possible for the PyPI software to pull the data from the sdist and/or egg files. But someone would have to supply the patches to implement this. From zooko at zooko.com Tue Mar 24 04:47:36 2009 From: zooko at zooko.com (zooko) Date: Mon, 23 Mar 2009 21:47:36 -0600 Subject: [Distutils] PEP 376 and the name .egg-info? In-Reply-To: <8C3865DA-1EDF-4508-A7F8-8D9EAC286139@bud.ca> References: <8C3865DA-1EDF-4508-A7F8-8D9EAC286139@bud.ca> Message-ID: <9E367D7E-BDFF-4023-BBB9-70F0C00EF327@zooko.com> I recently made a very similar suggestion in http://bugs.python.org/ issue5480 -- ".egg-info" => ".pkg-info". It seems to me that either the new distutils should use a different name for those files, such as ".pkg-info", or else the new distutils needs to come with a announcement that tells people to pay attention and notice that even though it has "egg" in its name it is now a core Python feature which is supported by the core Python team. Regards, Zooko From pjb at flix.com.au Tue Mar 24 05:26:07 2009 From: pjb at flix.com.au (Peter Bobroff) Date: Tue, 24 Mar 2009 15:26:07 +1100 Subject: [Distutils] No Win exe for Python 2.6 on PyPI References: <200903241523100750.01C1DDF4@mail.flixco.ws> Message-ID: <200903241526070718.01C4913D@mail.flixco.ws> Hi, I am relatively new to Python and recently had to install 2.6 on a coworker's computer. Not having a Windows 2.6 exe on the setuptools page of PyPI caused me considerable heartache and hassle. I eventually found a copy of setuptools-0.6c9.win32.exe which I had managed to build some months ago. I attach a copy in the hope it might shame someone with more experience into puting it or something better onto PyPI IMHO this is a major hurdle for a newcomer trying to use packages if the set up utility itself is missing. Ah! not allowed to attach .exe to this list. Deleted attachment and trying again. Regards, Peter Bobroff From ronaldoussoren at mac.com Tue Mar 24 13:32:55 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 24 Mar 2009 13:32:55 +0100 Subject: [Distutils] PyPI mirroring and Changing the .egg-info PEPs In-Reply-To: <94bdd2610903230843o71a20c1fs2a2733ce450f064f@mail.gmail.com> References: <94bdd2610903230843o71a20c1fs2a2733ce450f064f@mail.gmail.com> Message-ID: <68D63243-A727-4334-9C35-E429991BCA79@mac.com> On 23 Mar, 2009, at 16:43, Tarek Ziad? wrote: > >> >> "- get_egg_info(pkg_name) -> path or None >> >> Scans all site-packages directories and looks for all >> `pkg_name.egg-info` >> directories. Returns the directory path that contains a PKG-INFO >> that matches >> `pkg_name` for the `name` metadata. Notice that there should be at >> most >> one result. If more than one path matches the pkg_name, a >> DistutilsError >> is raised. >> >> If the directory is not found, returns None." >> >> Currently, if there are multiple paths that might be importable, >> whichever one is found first with sys.path is imported, and conflicts >> aren't really considered. raising DistutilsError seems like it >> changes this. >> > > Yes, while thinking about this API, I thaught it was a good idea to > raise if there's a conflict, > to make sure there's a healthy environement. Raising an error would make it a lot harder to temporarily place a directory containing an egg on $PYTHONPATH, for instance to test if a new release of a library will work in an environment before actually installing it. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From ziade.tarek at gmail.com Thu Mar 26 04:58:51 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 26 Mar 2009 04:58:51 +0100 Subject: [Distutils] Packaging Survey first results + Summit schedule Message-ID: <94bdd2610903252058n1a15daa7ic87cd32c7570ba81@mail.gmail.com> Hi, Sorry for the cross-post, but it seemed appropriate since packaging is being discussed in python-dev tonight, - Here are the first results for the packaging survey: http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/ - And tomorrow's Summit schedule for the packaging part : http://tarekziade.wordpress.com/2009/03/26/pycon-language-summit-is-tomorrow/ Please comment (in the appropriate list or in my blog) if you have something you would like to say or see addressed during the Summit and you are unable to be present. (I am already trying to summarize what has been said in python-dev today but I am not sure I'll be able to read everything before tomorrow) Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ianb at colorstudy.com Thu Mar 26 07:16:09 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 26 Mar 2009 01:16:09 -0500 Subject: [Distutils] PyPI mirroring and Changing the .egg-info PEPs In-Reply-To: <319e029f0903221344r46a41cedvf6f0027d4e067fa9@mail.gmail.com> References: <319e029f0903221344r46a41cedvf6f0027d4e067fa9@mail.gmail.com> Message-ID: 2009/3/22 Lennart Regebro : > On Sun, Mar 22, 2009 at 20:10, Ian Bicking wrote: >> Suggestions: >> >> Instead of http://pypi.python.org/mirrors how about >> http://mirrors.pypi.python.org (or pypimirrors.python.org, etc) -- >> then the mirror list can be hosted separately, and the likelihood of >> both the mirrors and the main PyPI server being down are much lower. > > +1 > >> Admittedly, this makes it harder for other indexes to provide their >> own mirrors > > It does? Why? I must have missed something. :) If you want to tell an installation to use another location (e.g., http://mysite.org/packages), with the current proposal you could have a list of mirrors at http://mysite.org/packages/mirrors, while with a domain name it's unclear where the mirrors would go. -- Ian Bicking | http://blog.ianbicking.org From david at ar.media.kyoto-u.ac.jp Thu Mar 26 07:22:04 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 26 Mar 2009 15:22:04 +0900 Subject: [Distutils] Installing built C extensions Message-ID: <49CB1F0C.9050007@ar.media.kyoto-u.ac.jp> Hi, Following one discussion on the python-dev ML, here is one of my recent frustration with distutils. I have a pure C library (pure meaning it does not use the python C API in any way) which I want to install so that other python packages can use it: from distutils.core import setup # How to install libfoo.a/foo.lib in say pkg_dir/lib ? setup(name='yo', libraries=[('foo', {'sources': ['foo.c']})]) I install headers in pkg_dir/include (for example site-packages/foo/include/foo.h), and I want to install foo lib in site-packages/foo/lib. I can't easily retrieve the library file in the build dir from setup.py, so I can't use install_data (and libraries are not data anyway), cheers, David From Fadhley.Salim at uk.calyon.com Thu Mar 26 13:47:49 2009 From: Fadhley.Salim at uk.calyon.com (Fadhley Salim) Date: Thu, 26 Mar 2009 12:47:49 -0000 Subject: [Distutils] Why does setuptools sometimes delete and then re-install the exact same egg? References: <49CB1F0C.9050007@ar.media.kyoto-u.ac.jp> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB20448400B@MXCU10MX1.MSX.CIB> Searching for calyon==1.0-r2009-03-12 Best match: calyon 1.0-r2009-03-12 Processing calyon-1.0_r2009_03_12-py2.4-win32.egg Removing d:\devtools\python24\lib\site-packages\calyon-1.0_r2009_03_12-py2.4-win3 2.egg Copying calyon-1.0_r2009_03_12-py2.4-win32.egg to d:\devtools\python24\lib\site-packages What causes this? I've seen it happen a few times but I'm not sure how to re-produce the behavior. I'm trying to install an egg on a computer where an identical egg already exists. Why does it remove the egg and then re-install it? I'm calling easy_install from a script with the options: ['-vv', '-m', '-f', 'R:/OPTIONS/Stephen/python_eggs', 'calyon==1.0_r2009_03_12'] I'm using setuptools 0.6c9 -------------- next part -------------- This email does not create a legal relationship between any member of the Cr=E9dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. This message and/or any attachments is intended for the sole use of its addressee. If you are not the addressee, please immediately notify the sender and then destroy the message. As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Cr?dit Agricole CIB. All rights reserved. Ce message et ses pi?ces jointes est destin? ? l'usage exclusif de son destinataire. Si vous recevez ce message par erreur, merci d'en aviser imm?diatement l'exp?diteur et de le d?truire ensuite. Le pr?sent message pouvant ?tre alt?r? ? notre insu, CALYON Cr?dit Agricole CIB ne peut pas ?tre engag? par son contenu. Tous droits r?serv?s. From Fadhley.Salim at uk.calyon.com Thu Mar 26 18:08:14 2009 From: Fadhley.Salim at uk.calyon.com (Fadhley Salim) Date: Thu, 26 Mar 2009 17:08:14 -0000 Subject: [Distutils] Continued... Why does setuptools sometimes delete and thenre-install the exact same egg? References: <49CB1F0C.9050007@ar.media.kyoto-u.ac.jp> <7F347D91614EBC48AA7540A2A76D3BB20448400B@MXCU10MX1.MSX.CIB> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB20448400D@MXCU10MX1.MSX.CIB> Further to my last email I've done some investigation: The problem seems to be that on some systems the package index which setuptools builds before it finds the distribtution to install does not include all of the currently installed packages. As a result it goes ahead and deletes + installs the same egg which already exists regardless of whether this is strictly neccecary. So far I've not managed to work out why this happens... any suggestions? Thanks -----Original Message----- From: distutils-sig-bounces+fadhley.salim=uk.calyon.com at python.org [mailto:distutils-sig-bounces+fadhley.salim=uk.calyon.com at python.org] On Behalf Of Salim, Fadhley (CALYON) Sent: 26 March 2009 12:48 To: Distutils-SIG at python.org Subject: [Distutils] Why does setuptools sometimes delete and thenre-install the exact same egg? Searching for calyon==1.0-r2009-03-12 Best match: calyon 1.0-r2009-03-12 Processing calyon-1.0_r2009_03_12-py2.4-win32.egg Removing d:\devtools\python24\lib\site-packages\calyon-1.0_r2009_03_12-py2.4-win3 2.egg Copying calyon-1.0_r2009_03_12-py2.4-win32.egg to d:\devtools\python24\lib\site-packages What causes this? I've seen it happen a few times but I'm not sure how to re-produce the behavior. I'm trying to install an egg on a computer where an identical egg already exists. Why does it remove the egg and then re-install it? I'm calling easy_install from a script with the options: ['-vv', '-m', '-f', 'R:/OPTIONS/Stephen/python_eggs', 'calyon==1.0_r2009_03_12'] I'm using setuptools 0.6c9 -------------- next part -------------- This email does not create a legal relationship between any member of the Cr=E9dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. This message and/or any attachments is intended for the sole use of its addressee. If you are not the addressee, please immediately notify the sender and then destroy the message. As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Cr?dit Agricole CIB. All rights reserved. Ce message et ses pi?ces jointes est destin? ? l'usage exclusif de son destinataire. Si vous recevez ce message par erreur, merci d'en aviser imm?diatement l'exp?diteur et de le d?truire ensuite. Le pr?sent message pouvant ?tre alt?r? ? notre insu, CALYON Cr?dit Agricole CIB ne peut pas ?tre engag? par son contenu. Tous droits r?serv?s. From setuptools at bugs.python.org Fri Mar 27 15:38:52 2009 From: setuptools at bugs.python.org (Martijn Pieters) Date: Fri, 27 Mar 2009 14:38:52 +0000 Subject: [Distutils] [issue64] Subversion 1.6 entries format 'unrecognized' In-Reply-To: <1238164732.16.0.879862864641.issue64@psf.upfronthosting.co.za> Message-ID: <1238164732.16.0.879862864641.issue64@psf.upfronthosting.co.za> New submission from Martijn Pieters : After upgrading to subversion 1.6, the dreaded unrecognized .svn/entries format pops up again. The entries format version number is 10, and if I add that to sdist.entries_finder, things appear to work just fine. Perhaps the test can be updated to parse the integer and use a >= 9 test on it, and just assume that future formats will continue to work until proven otherwise? ---------- messages: 253 nosy: mjpieters priority: bug status: unread title: Subversion 1.6 entries format 'unrecognized' _______________________________________________ Setuptools tracker _______________________________________________ From Fadhley.Salim at uk.calyon.com Fri Mar 27 19:10:23 2009 From: Fadhley.Salim at uk.calyon.com (Fadhley Salim) Date: Fri, 27 Mar 2009 18:10:23 -0000 Subject: [Distutils] setuptools bug, inconsistent replacement of eggs when installing from file:// urls. Message-ID: <7F347D91614EBC48AA7540A2A76D3BB20448401D@MXCU10MX1.MSX.CIB> I think I've found a bug in the way that setuptools selects whether an egg specified by file:// url should be installed over an existing (identical) egg: To re-create the fault: * Install the egg of your choice. * make two local "egg repositories", one with a very long path (e.g. c:/a/) and one with a very long path * In each of the egg repositories place a copy of the egg you just installed, along with an HTML file containing a link to that egg... just it's filename. * Try to easy_install the egg with the options "-mv" and set "-f" to either one of the repositories When you select a short path for the repository you will notice that the egg in your site_packages folder is always replaced, you see something like this: "Removing d:\devtools\python24\lib\site-packages\calyon-1.0_r2009_03_12-py2.4-win3 2.egg" When you select a very long path for the repository you will not see this message, this is because easy_install leaves the egg unmodified. I've found out why this happens: The pkg_resources.Environment uses a hash-tuple to work out the order in which to consider packages to install. If you are trying to install two identical eggs, the 4th element of the tuple -len(self.location or '') becomes significant... longer URLs get preference to shorter URLs. That on it's own might not seem too bad, but consider the the 2nd value of the hash-tuple (precedence) is given an identical value of 3 both for a package which is already installed in pkg_resources and a package which can already be downloaded from a file:// link. I think a better default behaviour would be for it to give stuff that is already installed a higher precedence than stuff which could potentially be installed. Hope all of this makes sense! Sal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- This email does not create a legal relationship between any member of the Cr=E9dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. This message and/or any attachments is intended for the sole use of its addressee. If you are not the addressee, please immediately notify the sender and then destroy the message. As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Cr?dit Agricole CIB. All rights reserved. Ce message et ses pi?ces jointes est destin? ? l'usage exclusif de son destinataire. Si vous recevez ce message par erreur, merci d'en aviser imm?diatement l'exp?diteur et de le d?truire ensuite. Le pr?sent message pouvant ?tre alt?r? ? notre insu, CALYON Cr?dit Agricole CIB ne peut pas ?tre engag? par son contenu. Tous droits r?serv?s. From Fadhley.Salim at uk.calyon.com Fri Mar 27 19:23:42 2009 From: Fadhley.Salim at uk.calyon.com (Fadhley Salim) Date: Fri, 27 Mar 2009 18:23:42 -0000 Subject: [Distutils] Misleading exception thrown when attempting to install a missing .egg from a file:// url Message-ID: <7F347D91614EBC48AA7540A2A76D3BB20448401E@MXCU10MX1.MSX.CIB> Steps to re-create * Set up an environment as per my previous email but without the egg installed in site-packages. * Delete the egg from the egg_repository so that the file we expect to see is missing, but is still indicated in the html file which is used as the index of the egg repository directory. * Try to easy install the egg (as in previous example) and take note of the exception raised. My error is here: http://pastebin.com/m36b84d95 I'm guessing that we get a zipimporterror because the stream returned by the URL fetching fuction is obviously not zip data. However in this case might it be more appropriate to raise DistributionNotFound or some kind of special exception that more correctly describes this situation? Thanks, Sal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- This email does not create a legal relationship between any member of the Cr=E9dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. This message and/or any attachments is intended for the sole use of its addressee. If you are not the addressee, please immediately notify the sender and then destroy the message. As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Cr?dit Agricole CIB. All rights reserved. Ce message et ses pi?ces jointes est destin? ? l'usage exclusif de son destinataire. Si vous recevez ce message par erreur, merci d'en aviser imm?diatement l'exp?diteur et de le d?truire ensuite. Le pr?sent message pouvant ?tre alt?r? ? notre insu, CALYON Cr?dit Agricole CIB ne peut pas ?tre engag? par son contenu. Tous droits r?serv?s. From martin at v.loewis.de Fri Mar 27 22:50:18 2009 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 27 Mar 2009 16:50:18 -0500 Subject: [Distutils] Version strings Message-ID: <49CD4A1A.6050008@v.loewis.de> I just collected the version strings of versions that got released to PyPI, and put up the list on http://www.dcl.hpi.uni-potsdam.de/home/loewis/versions Most of them probably fit into any schema we come up with, but there are certainly some unconventional ones: 1 to 3 1.* Verrsion 1 working proof of concept Regards, Martin From eric at trueblade.com Fri Mar 27 23:28:38 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 27 Mar 2009 17:28:38 -0500 Subject: [Distutils] [Python-Dev] Version strings In-Reply-To: <49CD4A1A.6050008@v.loewis.de> References: <49CD4A1A.6050008@v.loewis.de> Message-ID: <49CD5316.70608@trueblade.com> Martin v. L?wis wrote: > I just collected the version strings of versions that > got released to PyPI, and put up the list on > > http://www.dcl.hpi.uni-potsdam.de/home/loewis/versions That's excellent, thank you. The following chunk of text is present. I don't really care, except that it might mean a bug in your extraction routine. Or maybe it's just a very wacky version string! ---------------- Sysv_ipc gives Python programs access to System V semaphores, shared memory and message queues. Most (all?) Unixes (including OS X) support System V IPC. Windows+Cygwin 1.7 might also work. Sample code is included. This extension is released under the GPL. You might also be interested in the similar POSIX IPC module at: http://semanchuk.com/philip/posix_ipc/ T-0.3.4 (BitTornado) Trunk ---------------- From christian at marquardt.sc Sat Mar 28 00:17:51 2009 From: christian at marquardt.sc (Christian Marquardt) Date: Sat, 28 Mar 2009 00:17:51 +0100 (GMT+01:00) Subject: [Distutils] A problem with setuptools, virtualenv and numpy In-Reply-To: <32464968.1831238195636293.JavaMail.root@athene> Message-ID: <17487744.1851238195871085.JavaMail.root@athene> Hello, I've run into the following problem: - I have numpy (v1.3.0b1) successfully installed in my system's site-packages (for Python 2.5.2). - I have setup a virtual environment (with virtualenv 1.3.3) which uses the systems site-packages; in particular, I can load numpy when working in that environment. - I have a (namespaced) package which requires numpy (via "install_requires = ['numpy']" in the setup() of setuptools (v0.6c9, as coming with virtualenv), and try to build that in the virtual environment using the usual python setup.py install. - When processing the requirements for the packages, setuptools downloads a fresh copy from numpy 1.3.0b1 from Pypi and tries to install it - although numpy is already available in the system wide site-packages directory. To me, that's a bug, but I would like to know if it is a bug in numpy (which might not support setuptools in the correct way) or setuptools which tries to install an already existing package. I have tried to debug this a bit and found that the problem seems to happen in the easy_install.process_distribution() method in easy_install.py, in the following part of the code: log.info("Processing dependencies for %s", requirement) try: distros = WorkingSet([]).resolve( [requirement], self.local_index, self.easy_install ) except [...] if self.always_copy or self.always_copy_from: # Force all the relevant distros to be copied or activated for dist in distros: if dist.key not in self.installed_projects: self.easy_install(dist.as_requirement()) log.info("Finished processing dependencies for %s", requirement) *IF* I understand the code and comments in there correctly, WorkingSet([]).resolve() returns a list of requirement objects representing the requirements which are fulfilled in the existing environment; in my case, that's the package itself as well as numpy: >>> print distros [yaros.gras 0.4.dev-r53 (/home/marq/.virtualenvs/yaros-devel/lib/python2.5/site-packages/yaros.gras-0.4.dev_r53-py2.5.egg), numpy 1.3.0b1 (/opt/apps/lib/python2.5/site-packages)] The paths are also pointing to the correct directories. Next, self.always_copy_from == '.', so the loop at the end of the code above is also run through. The requirement object of the package is filtered out, but the numpy requirement object is passed to self.easy_install(), which then tries to fetch the source code from Pypi. Is that the intended behavior or a bug? Many thanks, Christian. From david at ar.media.kyoto-u.ac.jp Sat Mar 28 09:18:48 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 28 Mar 2009 17:18:48 +0900 Subject: [Distutils] A problem with setuptools, virtualenv and numpy In-Reply-To: <17487744.1851238195871085.JavaMail.root@athene> References: <17487744.1851238195871085.JavaMail.root@athene> Message-ID: <49CDDD68.7070908@ar.media.kyoto-u.ac.jp> Christian Marquardt wrote: > Hello, > > I've run into the following problem: > > - I have numpy (v1.3.0b1) successfully installed in my system's > site-packages (for Python 2.5.2). > - I have setup a virtual environment (with virtualenv 1.3.3) which > uses the systems site-packages; in particular, I can load numpy > when working in that environment. > - I have a (namespaced) package which requires numpy (via > "install_requires = ['numpy']" in the setup() of setuptools > (v0.6c9, as coming with virtualenv), and try to build that in > the virtual environment using the usual python setup.py install. > - When processing the requirements for the packages, setuptools > downloads a fresh copy from numpy 1.3.0b1 from Pypi and tries > to install it - although numpy is already available in the system > wide site-packages directory. > > To me, that's a bug, but I would like to know if it is a bug in numpy > (which might not support setuptools in the correct way) or setuptools > which tries to install an already existing package. > If you want setuptools 'support', I think you should install numpy through the setupegg.py script. cheers, David From ianb at colorstudy.com Sat Mar 28 15:52:15 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 28 Mar 2009 09:52:15 -0500 Subject: [Distutils] What pip uses setuptools for Message-ID: During the discussion at the open space about distutils, it was noted several tools use Setuptools. Right now I think we all agree (probably PJE too) that Setuptools implements more than we can move into the standard library. Since these tools represent clear use cases, figuring out how to get these tools to not depend on Setuptools will, I think, factor out many of the most important APIs. And it so happens that pip uses just a few things, so factoring out this functionality seems feasible: I call setup.py egg_info. This is currently awkward, because where the egg-info information goes is hard to figure out. But it works, and gives me parseable metadata. (Note, all calls to setup.py are done in a subprocess; I think some tools call setup.py in-process, but subprocesses seemed safer to me and I don't see much of a downside, certainly not performance.) I install using Setuptools, which installs the egg-info metadata, and also records the files that were written. (I write these to a temporary location, rewrite the file paths, and put the list of files into the package's egg-info). I install using setup.py develop, using Setuptools. This calls egg_info, and also handles fixing up a .pth file. I use --no-deps, because I never want any of these installation processes to install anything. I've copied some code out of setuptools.command.egg_info for getting the svn revision. I'm fine maintaining this code (since there's also code for other version control systems). The uses of pkg_resources: I look around at all the packages' dependency_links.txt, as this gives pip a hint of where things came from, and this lets me better implement pip freeze. Some people hate dependency_links, but I don't at all. I get a list of all the installed packages (with versions), for pip freeze. It is very helpful that people use -r1234 for svn checkouts, as I detect this and use this to create sensible frozen versions. Handling version numbers for unreleased packages is important. I use pkg_resources to parse versions, test for which is better, and test for inclusion (whether a version satisfies a requirement). Oh, and I parse requirements. I would like if there was a distinction between stable and unstable versions included in the version discussion. I have some code to normalize project names. The code is awkward, and kind of uses pkg_resources. I'd like this handled more nicely. I think Setutools' case insensitivity is useful, as is some other forms of normalization. From ziade.tarek at gmail.com Sat Mar 28 21:18:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 28 Mar 2009 15:18:54 -0500 Subject: [Distutils] Distutils Pycon BoF/Sprints : wiki updated Message-ID: <94bdd2610903281318n7d725a29ub5d895f149b8a49e@mail.gmail.com> Hi Just a quick note for Pycon/BoF attendees, I have changed the front page for each major topic we have been discussed in Pycon see http://wiki.python.org/moin/Distutils each page have already a bit of content, or some ideas to work on, and I am completing them this afternoon. - DistutilsVersionFight : the goal is to come up with a proposal in distutils-SIG for version handling - Distutils/StandardizeEggInfo : the goal is to finalize PEP 376 - Distutils/Metadata : the goal is to finalize PEP 345 - Distutils/StaticMetadata : the goal is to come out with a proposal to split setup.py into a static metadata file + a lighter, almost empty script. - Distutils/Friends : the goal is to try to find a project in each os community that can manage a third-party tool on the top of Distutils The next working session is during the BoF tonight at 6 pm. There are already a lot of proposal going on in the DistutilsVersionFight. Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From kain at kain.org Sun Mar 29 00:53:46 2009 From: kain at kain.org (Bryon Roche) Date: Sat, 28 Mar 2009 23:53:46 +0000 (UTC) Subject: [Distutils] setuptools, other scms, pip? Message-ID: While watching various things at pycon today, I've started implementing support in setuptools for letting package_index.PackageIndex._download_url () use a pluggable entry point for download handler. I was asked, 'why not use pip?' I'm pretty sure this functionality belongs in the direct interface to PyPI, which I understand is setuptools. Is this incorrect? (if interested, I'm publishing my work at git://kain.org/git/setuptools- dev.git, and http://kain.org/git/setuptools-dev.git/ This is very rough and incomplete at the moment, I intend to also implement entry points for approriate version-tagging options for generated .eggs) Thanks, /B From christian at marquardt.sc Sun Mar 29 12:34:05 2009 From: christian at marquardt.sc (Christian Marquardt) Date: Sun, 29 Mar 2009 11:34:05 +0100 (GMT+01:00) Subject: [Distutils] A problem with setuptools, virtualenv and numpy In-Reply-To: <3140739.2031238322764866.JavaMail.root@athene> Message-ID: <1282180.2051238322845566.JavaMail.root@athene> Installing numpy with setupegg.py does indeed install numpy as an egg, but it does not solve the problem I describe here (tested with numpy-1.3.0rc1, and the rest as described below). If numpy is listed as a dependency for another package which is installed into a virtual environment, setuptools tries to download and install numpy instead of just adjusting the easy_install.pth file. Hmm. I tried to install my packages without a virtual environment, and that works ok. So it must be a problem in the interaction between setuptools and virtualenv... I'll cross post this in the virtualenv mailing list... Many thanks, Christian. ----- "David Cournapeau" wrote: > Christian Marquardt wrote: > > Hello, > > > > I've run into the following problem: > > > > - I have numpy (v1.3.0b1) successfully installed in my system's > > site-packages (for Python 2.5.2). > > - I have setup a virtual environment (with virtualenv 1.3.3) which > > uses the systems site-packages; in particular, I can load numpy > > when working in that environment. > > - I have a (namespaced) package which requires numpy (via > > "install_requires = ['numpy']" in the setup() of setuptools > > (v0.6c9, as coming with virtualenv), and try to build that in > > the virtual environment using the usual python setup.py install. > > - When processing the requirements for the packages, setuptools > > downloads a fresh copy from numpy 1.3.0b1 from Pypi and tries > > to install it - although numpy is already available in the > system > > wide site-packages directory. > > > > To me, that's a bug, but I would like to know if it is a bug in > numpy > > (which might not support setuptools in the correct way) or > setuptools > > which tries to install an already existing package. > > > > If you want setuptools 'support', I think you should install numpy > through the setupegg.py script. > > cheers, > > David -- Dr. Christian Marquardt Email: christian at marquardt.sc Wilhelm-Leuschner-Str. 27 Tel.: +49 (0) 6151 95 13 776 64293 Darmstadt Mobile: +49 (0) 179 290 84 74 Germany Fax: +49 (0) 6151 95 13 885 From eric.c2c at gmail.com Sun Mar 29 22:10:04 2009 From: eric.c2c at gmail.com (Eric Lemoine) Date: Sun, 29 Mar 2009 22:10:04 +0200 Subject: [Distutils] install as zip Message-ID: <5ec103de0903291310k2f659d3fx475365802e2caca5@mail.gmail.com> Hi I have a Pylons-based app that I want to distribute through a custom pypi. I use "easy_install -zmaxd somedir" to get the set of eggs required to distribute my app. Some of the .eggs are zips, some others are directories. I'd like zip eggs only, do I have a way to get that? Thanks, Eric -- Envoy? avec mon mobile From ziade.tarek at gmail.com Sun Mar 29 23:47:45 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 29 Mar 2009 16:47:45 -0500 Subject: [Distutils] Setuptools code Message-ID: <94bdd2610903291447t4d55fba8g9cb48e8962991d9e@mail.gmail.com> Hi Phillip One of the task here at Pycon, will consist of splitting pkg_resources.py in smaller bits, to start to see what we are going to put back into Distutils. I think that the best approach is to work on setuptools directly. It also a benefit for setuptools. I am also considering working there on a few patches to make setuptools work with the current Distutils trunk. Are you OK with this approach ? If not, we'll probably start with a copy from the current trunk somewhere else on Python's subversion Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From pje at telecommunity.com Mon Mar 30 01:14:05 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 29 Mar 2009 19:14:05 -0400 Subject: [Distutils] Setuptools code In-Reply-To: <94bdd2610903291447t4d55fba8g9cb48e8962991d9e@mail.gmail.co m> References: <94bdd2610903291447t4d55fba8g9cb48e8962991d9e@mail.gmail.com> Message-ID: <20090329231141.9B3393A406A@sparrow.telecommunity.com> At 04:47 PM 3/29/2009 -0500, Tarek Ziad? wrote: >Hi Phillip > >One of the task here at Pycon, will consist of splitting >pkg_resources.py in smaller bits, to start to see what we are going to >put back into Distutils. > >I think that the best approach is to work on setuptools directly. It >also a benefit for setuptools. >I am also considering working there on a few patches to make >setuptools work with the current Distutils trunk. > >Are you OK with this approach ? Patches that change policies or refactoring should be on a separate branch. Patches that fix bugs and don't introduce new ones can go on the trunk, and hopefully that would include your 2.6-compatibility work. (It should also be understood that "Group X doesn't like the current behavior" doesn't qualify as a bug fix if changing it breaks behavior in use for Group Y", and when in doubt, either assume that Group Y exists, or ask me.) >If not, we'll probably start with a copy from the current trunk >somewhere else on Python's subversion For the breakup or any other significant refactoring or policy change, please start a new branch. I will take responsibility for porting to the trunk and 0.6 branch as appropriate. From pje at telecommunity.com Mon Mar 30 01:15:55 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 29 Mar 2009 19:15:55 -0400 Subject: [Distutils] install as zip In-Reply-To: <5ec103de0903291310k2f659d3fx475365802e2caca5@mail.gmail.co m> References: <5ec103de0903291310k2f659d3fx475365802e2caca5@mail.gmail.com> Message-ID: <20090329231331.24FB93A406A@sparrow.telecommunity.com> At 10:10 PM 3/29/2009 +0200, Eric Lemoine wrote: >Hi I have a Pylons-based app that I want to distribute through a >custom pypi. I use "easy_install -zmaxd somedir" to get the set of >eggs required to distribute my app. Some of the .eggs are zips, some >others are directories. I'd like zip eggs only, do I have a way to get that? Currently, easy_install doesn't re-zip eggs that are already extracted to directories on sys.path. Deleting the installed versions and re-installing them with '-z' should get you fixed up. From eric.c2c at gmail.com Mon Mar 30 06:46:21 2009 From: eric.c2c at gmail.com (Eric Lemoine) Date: Mon, 30 Mar 2009 06:46:21 +0200 Subject: [Distutils] install as zip In-Reply-To: <20090329231331.24FB93A406A@sparrow.telecommunity.com> References: <5ec103de0903291310k2f659d3fx475365802e2caca5@mail.gmail.com> <20090329231331.24FB93A406A@sparrow.telecommunity.com> Message-ID: <5ec103de0903292146k132425edt8bd7c3c9957a3550@mail.gmail.com> Yeah that's it. Thanks. Eric 2009/3/30, P.J. Eby : > At 10:10 PM 3/29/2009 +0200, Eric Lemoine wrote: >>Hi I have a Pylons-based app that I want to distribute through a >>custom pypi. I use "easy_install -zmaxd somedir" to get the set of >>eggs required to distribute my app. Some of the .eggs are zips, some >>others are directories. I'd like zip eggs only, do I have a way to get >> that? > > Currently, easy_install doesn't re-zip eggs that are already > extracted to directories on sys.path. Deleting the installed > versions and re-installing them with '-z' should get you fixed up. > > -- Envoy? avec mon mobile From setuptools at bugs.python.org Mon Mar 30 16:17:11 2009 From: setuptools at bugs.python.org (Salim Fadhley) Date: Mon, 30 Mar 2009 14:17:11 +0000 Subject: [Distutils] [issue65] setuptools bug, inconsistent replacement of eggs when installing from file:// urls. In-Reply-To: <1238422631.04.0.179251234986.issue65@psf.upfronthosting.co.za> Message-ID: <1238422631.04.0.179251234986.issue65@psf.upfronthosting.co.za> New submission from Salim Fadhley : I think I've found a bug in the way that setuptools selects whether an egg specified by file:// url should be installed over an existing (identical) egg: To re-create the fault: * Install the egg of your choice. * make two local "egg repositories", one with a very long path (e.g. c:/a/) and one with a very long path * In each of the egg repositories place a copy of the egg you just installed, along with an HTML file containing a link to that egg... just it's filename. * Try to easy_install the egg with the options "-mv" and set "-f" to either one of the repositories When you select a short path for the repository you will notice that the egg in your site_packages folder is always replaced, you see something like this: "Removing d:\devtools\python24\lib\site-packages\calyon-1.0_r2009_03_12-py2.4-win32.egg" When you select a very long path for the repository you will not see this message, this is because easy_install leaves the egg unmodified. I've found out why this happens: The pkg_resources.Environment uses a hash-tuple to work out the order in which to consider packages to install. If you are trying to install two identical eggs, the 4th element of the tuple -len(self.location or '') becomes significant... longer URLs get preference to shorter URLs. That on it's own might not seem too bad, but consider the the 2nd value of the hash-tuple (precedence) is given an identical value of 3 both for a package which is already installed in pkg_resources and a package which can already be downloaded from a file:// link. I think a better default behaviour would be for it to give stuff that is already installed a higher precedence than stuff which could potentially be installed. ---------- messages: 254 nosy: salimfadhley priority: bug status: unread title: setuptools bug,inconsistent replacement of eggs when installing from file:// urls. _______________________________________________ Setuptools tracker _______________________________________________ From setuptools at bugs.python.org Mon Mar 30 16:19:40 2009 From: setuptools at bugs.python.org (Salim Fadhley) Date: Mon, 30 Mar 2009 14:19:40 +0000 Subject: [Distutils] [issue66] Misleading exception thrown when attempting to installa missing .egg from a file:// url In-Reply-To: <1238422780.52.0.834336929426.issue66@psf.upfronthosting.co.za> Message-ID: <1238422780.52.0.834336929426.issue66@psf.upfronthosting.co.za> New submission from Salim Fadhley : Steps to re-create * Set up an environment as per my previous email but without the egg installed in site-packages. * Delete the egg from the egg_repository so that the file we expect to see is missing, but is still indicated in the html file which is used as the index of the egg repository directory. * Try to easy install the egg (as in previous example) and take note of the exception raised. My error is here: http://pastebin.com/m36b84d95 I'm guessing that we get a zipimporterror because the stream returned by the URL fetching fuction is obviously not zip data. However in this case might it be more appropriate to raise DistributionNotFound or some kind of special exception that more correctly describes this situation? ---------- messages: 255 nosy: salimfadhley priority: wish status: unread title: Misleading exception thrown when attempting to installa missing .egg from a file:// url _______________________________________________ Setuptools tracker _______________________________________________ From larry at hastings.org Mon Mar 30 19:18:45 2009 From: larry at hastings.org (Larry Hastings) Date: Mon, 30 Mar 2009 12:18:45 -0500 Subject: [Distutils] The Ubuntu (Debian?) package maintainers for distutils and setuptools broke PYTHONUSERBASE Message-ID: <49D0FEF5.1000404@hastings.org> I'm trying to add a lightweight "virtualenv" to a local build system using PYTHONUSERBASE. Unfortunately the package maintainers for distutils and setuptools broke it on my Ubuntu laptop. I have incontrovertable proof! PYTHONUSERBASE doesn't scan the exact directory you set it to; you're pointing it at a "prefix" directory, and it looks in directories under there. Well, *directory*, really: PYTHONUSERBASE only causes CPython to scan one additional directory for packages. That directory is: "{prefix}/lib/python2.6/site-packages". This is true whether you use the Debian packaged build or if you build Python yourself from source. AFAICT distutils uses distutils.sysconfig.get_python_lib() to decide where you should install a site package. Depending on the inputs, it will give you one of two directories. If it's a "standard library", it'll give you "{prefix}/lib/python2.6". However! If you call get_python_lib() saying it's *not* a standard library, you get different results. If you build CPython from scratch you'll get "{prefix}/lib/python2.6/site-packages". If you use the Ubuntu Python 2.6 package, you'll get "{prefix}/lib/python2.6/dist-packages". But PYTHONUSERBASE still only looks in the "site-packages" directory. It ignores this directory and therefore doesn't pick up your packages. A more obscure but more widespread bug: consider that PYTHONUSERBASE also ignores the "standard library" directory returned by get_python_lib(). If you wanted to have your own local version of a "standard library", you couldn't put it in "{prefix}/lib/python2.6". This is true whether you build CPython from scratch or use the Ubuntu packages. setuptools under Ubuntu breaks PYTHONUSERBASE in a similar but different way. If you install the setuptools egg by hand, Setuptools will install packages into "{prefix}/lib/python2.6/site-packages". If you use the Ubuntu package for setuptools, setuptools will install packages into "{prefix}/local/lib/python2.6/dist-packages". Again, this is a directory PYTHONUSERBASE ignores. In my opinion the best way to fix this would be to add a new environment variable and deprecate the old one. I nominate "PYTHONUSERBASES"--note the plural. PYTHONUSERBASES would support a local-path-separator-separated list of directories, all of which would be used as site package directories. For each directory on the path, we would add *the same list of subdirectories* that site.addsitepackages() does. Failing that, PYTHONUSERBASE should be at least be changed so it adds the same directories as site.addsitepackages(). I would be happy to contribute a patch to do either of these, for Python 2.x and 3.x. What follows is my test case where I figured all this out. I started by creating a PYTHONUSERBASE directory, which in my late-night hacking fever I called "/home/larry/pwned". Inside I created subdirectories matching every directory I'd ever seen scanned for site packages, like "lib/python2.6/site-packages". Then for each directory I added a do-nothing module named for the directory it was in, with a "UB" on the front so I knew it came from my PYTHONUSERBASE. For example, one file was called "/home/larry/pwned/local/lib/python2.6/dist-packages/UBlocallibpythondistpackages.py". Next, I added those same files to the equivalent directories in "/usr", only with "SYSTEM" on the front. For instance, "/usr/lib/python2.6/site-packages/SYSTEMlibpythonsitepackages.py" Next, I built Python 2.6 from source, with a prefix directory of "/home/larry/src/python/userbase/release", and installed setuptools from the egg on cheeseshop. I then added these same files one last time, and again with SYSTEM on the front, to my locally-built Python's prefix directory. Finally I wrote a Python script that tried to import every one of these modules, and ran it as follows: % PYTHONUSERBASE=/home/larry/pwned python findcookies.py % PYTHONUSERBASE=/home/larry/pwned ./python findcookies.py Here's the output. First, the results from running my built-from-source Python. Hopefully you're reading this in a fixed-point font on a wide screen: -- % PYTHONUSERBASE=/home/larry/pwned /home/larry/src/python/userbase/release/bin/python /home/larry/findcookies.py ------------------------------------------------------------------------------ Where does distutils say we should install? Calling distutils.sysconfig.get_python_lib() with two different prefixes (sys.prefix and "/home/larry/pwned") and all combinations of its two boolean arguments: du.sc.gpl(True , True , '/home/larry/src/python/userbase/release') = '/home/larry/src/python/userbase/release/lib/python2.6' du.sc.gpl(True , True , '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6' du.sc.gpl(True , False, '/home/larry/src/python/userbase/release') = '/home/larry/src/python/userbase/release/lib/python2.6/site-packages' du.sc.gpl(True , False, '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6/site-packages' du.sc.gpl(False, True , '/home/larry/src/python/userbase/release') = '/home/larry/src/python/userbase/release/lib/python2.6' du.sc.gpl(False, True , '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6' du.sc.gpl(False, False, '/home/larry/src/python/userbase/release') = '/home/larry/src/python/userbase/release/lib/python2.6/site-packages' du.sc.gpl(False, False, '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6/site-packages' ------------------------------------------------------------------------------ Where does setup_tools say we should install? Expanding setuptools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"] with two different prefixes (sys.prefix and "/home/larry/pwned"): /home/larry/src/python/userbase/release/lib/python2.6/site-packages /home/larry/pwned/lib/python2.6/site-packages ------------------------------------------------------------------------------ Finally: trying to load a module from every possible site-packages directory. There are five columns in the output; here's what they mean. +----------- Marked with "*" if distutils.sysconfig.get_python_lib | told us to use this directory. | | +--------- Marked with "EZ" if setuptools.command.easy_install.easy_install.INSTALL_SCHEMES | | told us to use this directory. | | | | +------ Could we load this module? | | | | | | +--- What silly name did I give +--- What directory did I | | | | to this module? | stick this module into? | | | | | v v v v v * True SYSTEMlibpython /home/larry/src/python/userbase/release/lib/python2.6 True SYSTEMlibsitepython /home/larry/src/python/userbase/release/lib/site-python False SYSTEMlibdistpython /home/larry/src/python/userbase/release/lib/dist-python * EZ True SYSTEMlibpythonsitepackages /home/larry/src/python/userbase/release/lib/python2.6/site-packages False SYSTEMlibpythondistpackages /home/larry/src/python/userbase/release/lib/python2.6/dist-packages False SYSTEMlocallibpythondistpackages /home/larry/src/python/userbase/release/local/lib/python2.6/dist-packages * False UBlibpython /home/larry/pwned/lib/python2.6 False UBlibsitepython /home/larry/pwned/lib/site-python False UBlibdistpython /home/larry/pwned/lib/dist-python * EZ True UBlibpythonsitepackages /home/larry/pwned/lib/python2.6/site-packages False UBlibpythondistpackages /home/larry/pwned/lib/python2.6/dist-packages False UBlocallibpythondistpackages /home/larry/pwned/local/lib/python2.6/dist-packages -- Next, the output from the CPython and setuptools from the Ubuntu packages: -- % PYTHONUSERBASE=/home/larry/pwned /usr/bin/python /home/larry/findcookies.py ------------------------------------------------------------------------------ Where does distutils say we should install? Calling distutils.sysconfig.get_python_lib() with two different prefixes (sys.prefix and "/home/larry/pwned") and all combinations of its two boolean arguments: du.sc.gpl(True , True , '/usr') = '/usr/lib/python2.6' du.sc.gpl(True , True , '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6' du.sc.gpl(True , False, '/usr') = '/usr/lib/python2.6/dist-packages' du.sc.gpl(True , False, '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6/dist-packages' du.sc.gpl(False, True , '/usr') = '/usr/lib/python2.6' du.sc.gpl(False, True , '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6' du.sc.gpl(False, False, '/usr') = '/usr/lib/python2.6/dist-packages' du.sc.gpl(False, False, '/home/larry/pwned') = '/home/larry/pwned/lib/python2.6/dist-packages' ------------------------------------------------------------------------------ Where does setup_tools say we should install? Expanding setuptools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"] with two different prefixes (sys.prefix and "/home/larry/pwned"): /usr/local/lib/python2.6/dist-packages /home/larry/pwned/local/lib/python2.6/dist-packages ------------------------------------------------------------------------------ Finally: trying to load a module from every possible site-packages directory. There are five columns in the output; here's what they mean. +----------- Marked with "*" if distutils.sysconfig.get_python_lib | told us to use this directory. | | +--------- Marked with "EZ" if setuptools.command.easy_install.easy_install.INSTALL_SCHEMES | | told us to use this directory. | | | | +------ Could we load this module? | | | | | | +--- What silly name did I give +--- What directory did I | | | | to this module? | stick this module into? | | | | | v v v v v * True SYSTEMlibpython /usr/lib/python2.6 False SYSTEMlibsitepython /usr/lib/site-python True SYSTEMlibdistpython /usr/lib/dist-python False SYSTEMlibpythonsitepackages /usr/lib/python2.6/site-packages * True SYSTEMlibpythondistpackages /usr/lib/python2.6/dist-packages EZ True SYSTEMlocallibpythondistpackages /usr/local/lib/python2.6/dist-packages * False UBlibpython /home/larry/pwned/lib/python2.6 False UBlibsitepython /home/larry/pwned/lib/site-python False UBlibdistpython /home/larry/pwned/lib/dist-python True UBlibpythonsitepackages /home/larry/pwned/lib/python2.6/site-packages * False UBlibpythondistpackages /home/larry/pwned/lib/python2.6/dist-packages EZ False UBlocallibpythondistpackages /home/larry/pwned/local/lib/python2.6/dist-packages -- For what it's worth, here's my horrible hacked-together script: -- import distutils.sysconfig import os import sys paths = {} print """ %% PYTHONUSERBASE=%s %s %s ------------------------------------------------------------------------------ Where does distutils say we should install? Calling distutils.sysconfig.get_python_lib() with two different prefixes (sys.prefix and "/home/larry/pwned") and all combinations of its two boolean arguments: """.strip() % (os.environ["PYTHONUSERBASE"], sys.executable, sys.argv[0]) print for platformSpecific in (True, False): for standardLib in (True, False): for prefix in (sys.prefix, "/home/larry/pwned"): path = distutils.sysconfig.get_python_lib(platformSpecific, standardLib, prefix) print (" du.sc.gpl(" + str(platformSpecific).ljust(5) + ", " + str(standardLib).ljust(5) + ", " + repr(prefix) + ")").ljust(69), "=", repr(path) paths[path] = "* " print print print """ ------------------------------------------------------------------------------ Where does setup_tools say we should install? Expanding setuptools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"] with two different prefixes (sys.prefix and "/home/larry/pwned"): """.strip() print import setuptools.command.easy_install as ei for where, base in (("system", sys.prefix), ("userbase", "/home/larry/pwned")): path = ei.easy_install.INSTALL_SCHEMES[os.name]["install_dir"].replace("$base", base).replace("$py_version_short", "2.6") print " ", path if path in paths: paths[path] = "* EZ" else: paths[path] = " EZ" print print """ ------------------------------------------------------------------------------ Finally: trying to load a module from every possible site-packages directory. There are five columns in the output; here's what they mean. +----------- Marked with "*" if distutils.sysconfig.get_python_lib | told us to use this directory. | | +--------- Marked with "EZ" if setuptools.command.easy_install.easy_install.INSTALL_SCHEMES | | told us to use this directory. | | | | +------ Could we load this module? | | | | | | +--- What silly name did I give +--- What directory did I | | | | to this module? | stick this module into? | | | | | v v v v v """.strip() for prefix, pathprefix, where in (("SYSTEM", os.path.normpath(sys.prefix) + "/", "system "), ("UB", "/home/larry/pwned/", "userbase")): for _name, path in ( ("libpython", "lib/python2.6"), ("libsitepython", "lib/site-python"), ("libdistpython", "lib/dist-python"), ("libpythonsitepackages", "lib/python2.6/site-packages"), ("libpythondistpackages", "lib/python2.6/dist-packages"), # nobody ever told me to use one of these four, so I'm removing 'em. # ("locallibpython", "local/lib/python2.6"), # ("locallibsitepython", "local/lib/site-python"), # ("locallibdistpython", "local/lib/dist-python"), # ("locallibpythonsitepackages", "local/lib/python2.6/site-packages"), ("locallibpythondistpackages", "local/lib/python2.6/dist-packages"), ): name = prefix + _name fullpath = os.path.join(pathprefix, path) filename = os.path.join(fullpath, name + ".py") assert os.path.isfile(filename), "Your test is wrong, %s does not exist!" % repr(filename) s = "import {0}".format(name) try: exec s worked = True except ImportError: worked = False starred = paths.get(fullpath, " ") print " ", starred, str(worked).ljust(6), name.ljust(35), fullpath if 0: #print path if 1: name = name.replace("UB", "SYSTEM") if not os.path.isdir(path): os.makedirs(path) filename = path + "/" + name + ".py" filename = filename.replace("/python/", "/python2.6/") #print filename #continue f = open(filename, "wt") f.write('cookie = "PIXIE UNICORN"\n') f.close() print -- If you read this far, I congratulate you on your powers of concentration. Hope this helps, /larry/ From strawman at astraw.com Mon Mar 30 20:35:19 2009 From: strawman at astraw.com (Andrew Straw) Date: Mon, 30 Mar 2009 11:35:19 -0700 Subject: [Distutils] The Ubuntu (Debian?) package maintainers for distutils and setuptools broke PYTHONUSERBASE In-Reply-To: <49D0FEF5.1000404@hastings.org> References: <49D0FEF5.1000404@hastings.org> Message-ID: <49D110E7.6070607@astraw.com> Larry Hastings wrote: > > I'm trying to add a lightweight "virtualenv" to a local build system > using PYTHONUSERBASE. Unfortunately the package maintainers for > distutils and setuptools broke it on my Ubuntu laptop. Then you should file a bug report at http://bugs.launchpad.net/ubuntu/+source/python2.6 From lucid.illusion at gmail.com Tue Mar 31 21:56:40 2009 From: lucid.illusion at gmail.com (K C) Date: Tue, 31 Mar 2009 12:56:40 -0700 Subject: [Distutils] distutils changing interpreter to invalid path Message-ID: In the distutils documentation at http://docs.python.org/distutils/index.html, under the "Installing Scripts" section, it is stated that the location of the python interpreter can be changed. How exactly is this achieved? I am packaging a python application via CDBS which calls the setup.py script and changes my interpreter from "#!/usr/bin/env python" to "#!/usr/local/bin/python". The latter is invalid on Debian. I would prefer that disutils not change the interpreter at all since it is not needed in my case, however I am not sure where to add this "--executable" flag that is mentioned in the documentation or otherwise override the default behaviour. -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpeterson at enthought.com Tue Mar 31 23:13:23 2009 From: dpeterson at enthought.com (Dave Peterson) Date: Tue, 31 Mar 2009 16:13:23 -0500 Subject: [Distutils] A change to how platform name is generated for binary distributions? Message-ID: <49D28773.1070508@enthought.com> We're starting to try and distribute some pre-built binaries on Solaris and have come across an issue with how pkg_resources / distutils generates the platform specification when the distribution includes Python extensions. In particular, we'd like to distribute both x86 and amd64 (or x86_64) binaries for Solaris on Intel since the OS can run in either mode, but all our egg distributions get the same file name no matter what architecture they're actually built for. Diving into how the platform part of a distribution name gets generated, I find that it relies on the results of 'os.uname' via distutils.util.get_platform(). This seems like it would cause problems on more than just Solaris as it should also happen anytime someone is using a 32-bit Python on a 64-bit OS/CPU, no? So it occurs to me that it might be nice to fix up distutils.util.get_platform() to try and rely on the information that can be obtained about the sys.executable itself. After all, distutils / setuptools build extensions using the same compiler and linker flags that the Python environment was generated with so it seems logical to tie them together here too. While it may be hard (impossible?) to cover enough bases of reading compiler flags, many OSes provide tools to gather metadata about executable files which are fairly easy to use. In fact, on OSes that use ELF binaries, there should be alot of commonality in figuring this out I think. So I'm proposing that a change be made to do the above by wrapping all places where distutils.util.get_platform returns a value with something that replaces the last part of the platform spec with the Python architecture *if* it can safely be determine what that architecture is. Something like the following pseudo-code wrapper: def wrapper(platform): if cached_arch is None: cached_arch = _get_python_arch() parts = platform.split('-') parts[-1] = cached_arch platform = '-'.join(parts) return platform def _get_python_arch(): if sys.platform == '': return method_for_finding_python_arch_on_known_os() ... else: return '' I'm much more familiar with setuptools / pkg_resources than I am with distutils, so I've actually tried this out with an implementation where pkg_resources.get_build_platform() is patched and the _get_python_arch() above only returns something for Solaris. Seems to work okay for me, but that's likely more because I don't see many Solaris eggs in the wild than anything I've done explicitly for backwards compatibility. I understand backwards compatibility would be a big sticking point here, so I've tried to structure things where values only change where the implementation is positive it knows what to do, and what values to return. Thoughts? If there is agreement, I can submit a patch. -- Dave P.S. It is my understanding that there is no equivalent to OS X's 'fat' dual-binary mode on Solaris for shipping multiple arch binaries. Nor can I find any difference in filenames or directory names between a 32-bit and 64-bit Python 2.5.4 built from source. Is there some other alternative to building that would solve the above problem without patching distutils/pkg_resources? P.P.S. While you can force the kernel to boot in 32-bit mode, distutils.util.get_platform() returns the same values because the OS's uname does. i.e. whether 64-bit or 32-bit Intel, uname returns "i86pc" as the machine.