From zooko at zooko.com Mon Sep 1 19:10:42 2008 From: zooko at zooko.com (zooko) Date: Mon, 1 Sep 2008 11:10:42 -0600 Subject: [Distutils] "test_suite must be a list" ?? In-Reply-To: <439BAA43-F93D-43FA-84A7-CC483EA7F8A7@zooko.com> References: <439BAA43-F93D-43FA-84A7-CC483EA7F8A7@zooko.com> Message-ID: <7A95B6B7-87D0-4A12-B1A7-DF439E93A903@zooko.com> Here is the ticket on launchpad for Elisa and/or Ubuntu to fix this problem: https://bugs.launchpad.net/ubuntu/+bug/263697 Until that is fixed, the workaround is if your users say "I got an exception saying test_suite must be a list.", then you should ask them to uninstall elisa from their system. You can also use the kludgey work-around that I posted in a previous message, but since you can't do this for all Python packages which you might want to run the unit tests of, I'm not sure that there is a point. Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From m.van.rees at zestsoftware.nl Tue Sep 2 11:23:30 2008 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Tue, 2 Sep 2008 09:23:30 +0000 (UTC) Subject: [Distutils] setup script not found when directory name is 100 characters Message-ID: Hi, I'm sure I have read this somewhere, but my search skills are failing me. When easy installing a package this goes wrong: $ bin/easy_install ../foo.bar/dist/foo.bar-1.0.3.tar.gz Processing foo.bar-1.0.3.tar.gz error: Couldn't find a setup script in ../foo.bar/dist/foo.bar-1.0.3.tar.gz The cause is that there is a directory foo.bar-1.0.3/.../baz for which the length of this path is exactly 100 characters. When I release a version 1.1 instead, this path is reduced to 98 characters and easy_install happily installs it. Does anyone know which version of setuptools or easy_install introduced this problem or fixes it? Or a link where this is explained? Thanks, -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ "This is your day, don't let them take it away." [Barlow Girl] From marius at pov.lt Tue Sep 2 15:30:18 2008 From: marius at pov.lt (Marius Gedminas) Date: Tue, 2 Sep 2008 09:30:18 -0400 Subject: [Distutils] setup script not found when directory name is 100 characters In-Reply-To: References: Message-ID: <20080902133018.GA28974@platonas> On Tue, Sep 02, 2008 at 09:23:30AM +0000, Maurits van Rees wrote: > I'm sure I have read this somewhere, but my search skills are failing > me. When easy installing a package this goes wrong: > > $ bin/easy_install ../foo.bar/dist/foo.bar-1.0.3.tar.gz > Processing foo.bar-1.0.3.tar.gz > error: Couldn't find a setup script in ../foo.bar/dist/foo.bar-1.0.3.tar.gz > > The cause is that there is a directory foo.bar-1.0.3/.../baz for which > the length of this path is exactly 100 characters. When I release a > version 1.1 instead, this path is reduced to 98 characters and > easy_install happily installs it. > > Does anyone know which version of setuptools or easy_install > introduced this problem or fixes it? Or a link where this is > explained? Which Python version is this? I seem to remember some bug in stdlib's tarfile module that couldn't handle paths longer than 100 characters. Google + bugs.python.org tell me there were several bugs http://bugs.python.org/issue1583506 http://bugs.python.org/issue1509889 http://bugs.python.org/issue1609958 http://bugs.python.org/issue1719898 Marius Gedminas -- Always forgive your enemies. Nothing annoys them more. -- Oscar Wilde -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From ianb at colorstudy.com Tue Sep 2 22:47:47 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 02 Sep 2008 15:47:47 -0500 Subject: [Distutils] Weird easy_install behavior when reinstalling Message-ID: <48BDA673.6010303@colorstudy.com> I'm getting some weird behavior for a particular package, installed from svn. To reproduce: virtualenv test-env ./test-env/bin/easy_install http://trac-hacks.org/svn/includemacro/0.11 a bunch of stuff is installed... then run: ./test-env/bin/easy_install http://trac-hacks.org/svn/includemacro/0.11 probably the first time you reinstall (and if not then, try a couple times) you get an exception like: > $ ./T2/bin/easy_install http://trac-hacks.org/svn/includemacro/0.11 > Downloading http://trac-hacks.org/svn/includemacro/0.11 > Doing subversion checkout from http://trac-hacks.org/svn/includemacro/0.11 to /tmp/easy_install-SiGKUf/0.11 > Processing 0.11 > Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-SiGKUf/0.11/egg-dist-tmp-5rw1jx > zip_safe flag not set; analyzing archive contents... > TracIncludeMacro 2.1 is already the active version in easy-install.pth > > Installed /home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/TracIncludeMacro-2.1-py2.4.egg > Processing dependencies for TracIncludeMacro==2.1 > Traceback (most recent call last): > File "./T2/bin/easy_install", line 7, in ? > sys.exit( > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 1671, in main > with_ei_usage(lambda: > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage > return f() > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 1675, in > distclass=DistributionWithoutHelpCommands, **kw > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands > self.run_command(cmd) > File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command > cmd_obj.run() > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 211, in run > self.easy_install(spec, not self.no_deps) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 422, in easy_install > return self.install_item(None, download, tmpdir, deps, True) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 478, in install_item > self.process_distribution(spec, dist, deps) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/easy_install.py", line 518, in process_distribution > distros = WorkingSet([]).resolve( > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 529, in resolve > requirements.extend(dist.requires(req.extras)[::-1]) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 2107, in requires > dm = self._dep_map > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 2099, in _dep_map > for extra,reqs in split_sections(self._get_metadata(name)): > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 2518, in split_sections > for line in yield_lines(s): > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 1813, in yield_lines > for ss in strs: > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 2121, in _get_metadata > for line in self.get_metadata_lines(name): > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 1140, in get_metadata_lines > return yield_lines(self.get_metadata(name)) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 1137, in get_metadata > return self._get(self._fn(self.egg_info,name)) > File "/home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/pkg_resources.py", line 1195, in _get > return self.loader.get_data(path) > zipimport.ZipImportError: bad local file header in /home/ianb/src/PoachEggs/T2/lib/python2.4/site-packages/TracIncludeMacro-2.1-py2.4.egg I assume this is while it is trying to read TracIncludeMacro-xxx.egg/EGG-INFO/requires.txt. I've tried opening that egg with zipfile, and it works fine. This problem seems somewhat specific to TracIncludeMacro, though I can't figure out why. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From m.van.rees at zestsoftware.nl Tue Sep 2 23:26:26 2008 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Tue, 2 Sep 2008 21:26:26 +0000 (UTC) Subject: [Distutils] setup script not found when directory name is 100 characters References: <20080902133018.GA28974@platonas> Message-ID: Marius Gedminas, on 2008-09-02: > > On Tue, Sep 02, 2008 at 09:23:30AM +0000, Maurits van Rees wrote: >> I'm sure I have read this somewhere, but my search skills are failing >> me. When easy installing a package this goes wrong: >> >> $ bin/easy_install ../foo.bar/dist/foo.bar-1.0.3.tar.gz >> Processing foo.bar-1.0.3.tar.gz >> error: Couldn't find a setup script in ../foo.bar/dist/foo.bar-1.0.3.tar.gz >> >> The cause is that there is a directory foo.bar-1.0.3/.../baz for which >> the length of this path is exactly 100 characters. When I release a >> version 1.1 instead, this path is reduced to 98 characters and >> easy_install happily installs it. >> >> Does anyone know which version of setuptools or easy_install >> introduced this problem or fixes it? Or a link where this is >> explained? > > Which Python version is this? I seem to remember some bug in stdlib's > tarfile module that couldn't handle paths longer than 100 characters. Ah right, tarfile is the culprit. I am using python2.4. And one of the bug reports you pasted mentioned that python2.4 is not maintained anymore. So I guess the possible workarounds are: - Make a .egg instead of a .tar.gz for my package. - Restrict myself to using x.y or a.b.c.d as the version of this package. - Give the directory a shorter name... Thanks for clearing this up. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ "This is your day, don't let them take it away." [Barlow Girl] From marius at pov.lt Tue Sep 2 23:55:16 2008 From: marius at pov.lt (Marius Gedminas) Date: Tue, 2 Sep 2008 17:55:16 -0400 Subject: [Distutils] setup script not found when directory name is 100 characters In-Reply-To: References: <20080902133018.GA28974@platonas> Message-ID: <20080902215516.GB23544@platonas> On Tue, Sep 02, 2008 at 09:26:26PM +0000, Maurits van Rees wrote: > Marius Gedminas, on 2008-09-02: > > Which Python version is this? I seem to remember some bug in stdlib's > > tarfile module that couldn't handle paths longer than 100 characters. > > Ah right, tarfile is the culprit. I am using python2.4. And one of > the bug reports you pasted mentioned that python2.4 is not maintained > anymore. > > So I guess the possible workarounds are: > > - Make a .egg instead of a .tar.gz for my package. .egg is a binary distribution; what you probably want is a .zip source distribution. Marius Gedminas -- We don't care. We don't have to. We're the Phone Company. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From tarek.ziade at ingeniweb.com Wed Sep 3 10:09:20 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Wed, 3 Sep 2008 10:09:20 +0200 Subject: [Distutils] setup script not found when directory name is 100 characters In-Reply-To: References: <20080902133018.GA28974@platonas> Message-ID: 2008/9/2 Maurits van Rees > Marius Gedminas, on 2008-09-02: > > > > On Tue, Sep 02, 2008 at 09:23:30AM +0000, Maurits van Rees wrote: > >> I'm sure I have read this somewhere, but my search skills are failing > >> me. When easy installing a package this goes wrong: > >> > >> $ bin/easy_install ../foo.bar/dist/foo.bar-1.0.3.tar.gz > >> Processing foo.bar-1.0.3.tar.gz > >> error: Couldn't find a setup script in > ../foo.bar/dist/foo.bar-1.0.3.tar.gz > >> > >> The cause is that there is a directory foo.bar-1.0.3/.../baz for which > >> the length of this path is exactly 100 characters. When I release a > >> version 1.1 instead, this path is reduced to 98 characters and > >> easy_install happily installs it. > >> > >> Does anyone know which version of setuptools or easy_install > >> introduced this problem or fixes it? Or a link where this is > >> explained? > > > > Which Python version is this? I seem to remember some bug in stdlib's > > tarfile module that couldn't handle paths longer than 100 characters. > > Ah right, tarfile is the culprit. I am using python2.4. And one of > the bug reports you pasted mentioned that python2.4 is not maintained > anymore. > > So I guess the possible workarounds are: > > - Make a .egg instead of a .tar.gz for my package. > > - Restrict myself to using x.y or a.b.c.d as the version of this > package. > > - Give the directory a shorter name... > If this egg is exclusively for a buildout environment, you could also add a patch to your buildout before eggs get collected see: http://tarekziade.wordpress.com/2008/06/19/python-24-tarfile-module-is-buggy-patch-it/ > > > Thanks for clearing this up. > > -- > Maurits van Rees | http://maurits.vanrees.org/ > Work | http://zestsoftware.nl/ > "This is your day, don't let them take it away." [Barlow Girl] > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.van.rees at zestsoftware.nl Wed Sep 3 11:39:23 2008 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Wed, 3 Sep 2008 09:39:23 +0000 (UTC) Subject: [Distutils] setup script not found when directory name is 100 characters References: <20080902133018.GA28974@platonas> <20080902215516.GB23544@platonas> Message-ID: Marius Gedminas, on 2008-09-02: > > On Tue, Sep 02, 2008 at 09:26:26PM +0000, Maurits van Rees wrote: >> Marius Gedminas, on 2008-09-02: >> > Which Python version is this? I seem to remember some bug in stdlib's >> > tarfile module that couldn't handle paths longer than 100 characters. >> >> Ah right, tarfile is the culprit. I am using python2.4. And one of >> the bug reports you pasted mentioned that python2.4 is not maintained >> anymore. >> >> So I guess the possible workarounds are: >> >> - Make a .egg instead of a .tar.gz for my package. > > .egg is a binary distribution; what you probably want is a .zip source > distribution. Ah, right, you can give options to sdist. This indeed works fine: python2.4 setup.py sdist --formats=zip Or this can be put in setup.cfg: [sdist] # guard against tarfile bug by using the zip format formats = zip and then a plain sdist works fine again: python2.4 setup.py sdist Thanks! -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ "This is your day, don't let them take it away." [Barlow Girl] From ziade.tarek at gmail.com Wed Sep 3 16:24:54 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Wed, 3 Sep 2008 07:24:54 -0700 (PDT) Subject: [Distutils] Unicode in distutils meta-data In-Reply-To: <94bdd2610804070132k5ab9e73dn94a676e6f19947de@mail.gmail.com> References: <94bdd2610804070132k5ab9e73dn94a676e6f19947de@mail.gmail.com> Message-ID: <19290360.post@talk.nabble.com> Tarek Ziad? wrote: > > Hello, > > I would like to summarize here http://bugs.python.org/issue2562. > ... > Hello, Just a quick note to say that my patch has been integrated to Python trunk with the help of MAL, So this bug is fixed for the 2.7 series, (doesn't concern 3.x) If you have accented letters in your name you may use them now in your setup.py metadata :-) Tarek -- View this message in context: http://www.nabble.com/Unicode-in-distutils-meta-data-tp16536042p19290360.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From h.goebel at goebel-consult.de Wed Sep 3 16:35:29 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 16:35:29 +0200 Subject: [Distutils] Where to report bugs for setuptools? Message-ID: <48BEA0B1.2050108@goebel-consult.de> Hi, I know I asked this some weeks ago, I did forget :-) Where to report bugs for setuptools? The setuptools website does not give any hint. -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From setuptools at bugs.python.org Wed Sep 3 16:42:25 2008 From: setuptools at bugs.python.org (htgoebel) Date: Wed, 03 Sep 2008 14:42:25 +0000 Subject: [Distutils] [issue39] website: does not tell where to report bugs In-Reply-To: <1220452945.74.0.568902020334.issue39@psf.upfronthosting.co.za> Message-ID: <1220452945.74.0.568902020334.issue39@psf.upfronthosting.co.za> New submission from htgoebel : There seams to be not a single link from the http://peak.telecommunity.com/DevCenter/ wiki to the bug tracker. This makes it hard to report bugs. Please add a link to these pages: * FrontPage * setuptools * EasyInstall ---------- messages: 152 nosy: htgoebel priority: urgent status: unread title: website: does not tell where to report bugs _______________________________________________ Setuptools tracker _______________________________________________ From h.goebel at goebel-consult.de Wed Sep 3 16:43:03 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 16:43:03 +0200 Subject: [Distutils] Where to report bugs for setuptools? In-Reply-To: <48BEA0B1.2050108@goebel-consult.de> References: <48BEA0B1.2050108@goebel-consult.de> Message-ID: <48BEA277.3090504@goebel-consult.de> Hartmut Goebel schrieb: > Where to report bugs for setuptools? The setuptools website does not > give any hint. Found it: http://bugs.python.org/setuptools -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From setuptools at bugs.python.org Wed Sep 3 16:54:49 2008 From: setuptools at bugs.python.org (htgoebel) Date: Wed, 03 Sep 2008 14:54:49 +0000 Subject: [Distutils] [issue40] development egg: recusion to deep when re-running setup.py develop In-Reply-To: <1220453689.64.0.813959255393.issue40@psf.upfronthosting.co.za> Message-ID: <1220453689.64.0.813959255393.issue40@psf.upfronthosting.co.za> New submission from htgoebel : I'm installing a development egg like this:: PYTHONPATH=.:$PYTHONPATH python setup_.py develop --install-dir . --script-dir . Now when re-running the command above, it will fail:: File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1072, in safe_name File "/usr/lib/python2.5/re.py", line 150, in sub return _compile(pattern, 0).sub(repl, string, count) File "/usr/lib/python2.5/re.py", line 230, in _compile p = _cache.get(cachekey) RuntimeError: maximum recursion depth exceeded in cmp This seams to be a problem with *.egg-link. If I add these lines to teh fron tom my setup.py, re-running is fine: import glob, os for fn in glob.glob('*.egg-link'): os.remove(fn) ---------- messages: 153 nosy: htgoebel priority: bug status: unread title: development egg: recusion to deep when re-running setup.py develop _______________________________________________ Setuptools tracker _______________________________________________ From h.goebel at goebel-consult.de Wed Sep 3 16:58:01 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 16:58:01 +0200 Subject: [Distutils] development-egg in current directory Message-ID: <48BEA5F9.7010502@goebel-consult.de> Hi, I'm currently installing development eggs into the current directory (the checkout directory) like this: PYTHONPATH=.:$PYTHONPATH \ python setup.py develop --install-dir . --script-dir . Since this has problems when re-running (see issue40), I wonder whether this is the correct use of the 'develop' egg. Any hints? -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From pje at telecommunity.com Wed Sep 3 17:17:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 03 Sep 2008 11:17:29 -0400 Subject: [Distutils] development-egg in current directory In-Reply-To: <48BEA5F9.7010502@goebel-consult.de> References: <48BEA5F9.7010502@goebel-consult.de> Message-ID: <20080903151629.DEC093A4105@sparrow.telecommunity.com> At 04:58 PM 9/3/2008 +0200, Hartmut Goebel wrote: >Hi, > >I'm currently installing development eggs into the current directory >(the checkout directory) like this: > > PYTHONPATH=.:$PYTHONPATH \ > python setup.py develop --install-dir . --script-dir . > >Since this has problems when re-running (see issue40), I wonder whether >this is the correct use of the 'develop' egg. Any hints? It's definitely not the correct use, especially since, depending on your layout, it could potentially overwrite your scripts. develop is intended for installing to a directory that's normally *already on* sys.path. Why are you doing that? From h.goebel at goebel-consult.de Wed Sep 3 17:43:25 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 17:43:25 +0200 Subject: [Distutils] development-egg in current directory In-Reply-To: <20080903151629.DEC093A4105@sparrow.telecommunity.com> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> Message-ID: <48BEB09D.9030200@goebel-consult.de> Phillip J. Eby schrieb: > At 04:58 PM 9/3/2008 +0200, Hartmut Goebel wrote: >> Hi, >> >> I'm currently installing development eggs into the current directory >> (the checkout directory) like this: >> >> PYTHONPATH=.:$PYTHONPATH \ >> python setup.py develop --install-dir . --script-dir . >> >> Since this has problems when re-running (see issue40), I wonder whether >> this is the correct use of the 'develop' egg. Any hints? > > It's definitely not the correct use, especially since, depending on your > layout, it could potentially overwrite your scripts. develop is > intended for installing to a directory that's normally *already on* > sys.path. Why are you doing that? Because I defined script-entry-points which I need to test. If there is another way to so this, I'll happily change :-) -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From tseaver at palladion.com Wed Sep 3 18:07:38 2008 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 03 Sep 2008 12:07:38 -0400 Subject: [Distutils] development-egg in current directory In-Reply-To: <48BEB09D.9030200@goebel-consult.de> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> <48BEB09D.9030200@goebel-consult.de> Message-ID: <48BEB64A.1020605@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hartmut Goebel wrote: > Phillip J. Eby schrieb: >> At 04:58 PM 9/3/2008 +0200, Hartmut Goebel wrote: >>> Hi, >>> >>> I'm currently installing development eggs into the current directory >>> (the checkout directory) like this: >>> >>> PYTHONPATH=.:$PYTHONPATH \ >>> python setup.py develop --install-dir . --script-dir . >>> >>> Since this has problems when re-running (see issue40), I wonder whether >>> this is the correct use of the 'develop' egg. Any hints? >> It's definitely not the correct use, especially since, depending on your >> layout, it could potentially overwrite your scripts. develop is >> intended for installing to a directory that's normally *already on* >> sys.path. Why are you doing that? > > Because I defined script-entry-points which I need to test. If there is > another way to so this, I'll happily change :-) In such I case, I always create a virtualenv[1], and use *its* python to do 'setup.py develop': the scripts are then installed into the 'bin' directory of the virtualenv. [1] http://pypi.python.org/pypi/virtualenv Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIvrZK+gerLs4ltQ4RAjWmAJ0ZNgpTkV+APO3BtRwOCT8uNO3OYwCgji13 sQ3V2oNtjmlWvpc7OFhXaZU= =6Gb2 -----END PGP SIGNATURE----- From tseaver at palladion.com Wed Sep 3 18:07:38 2008 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 03 Sep 2008 12:07:38 -0400 Subject: [Distutils] development-egg in current directory In-Reply-To: <48BEB09D.9030200@goebel-consult.de> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> <48BEB09D.9030200@goebel-consult.de> Message-ID: <48BEB64A.1020605@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hartmut Goebel wrote: > Phillip J. Eby schrieb: >> At 04:58 PM 9/3/2008 +0200, Hartmut Goebel wrote: >>> Hi, >>> >>> I'm currently installing development eggs into the current directory >>> (the checkout directory) like this: >>> >>> PYTHONPATH=.:$PYTHONPATH \ >>> python setup.py develop --install-dir . --script-dir . >>> >>> Since this has problems when re-running (see issue40), I wonder whether >>> this is the correct use of the 'develop' egg. Any hints? >> It's definitely not the correct use, especially since, depending on your >> layout, it could potentially overwrite your scripts. develop is >> intended for installing to a directory that's normally *already on* >> sys.path. Why are you doing that? > > Because I defined script-entry-points which I need to test. If there is > another way to so this, I'll happily change :-) In such I case, I always create a virtualenv[1], and use *its* python to do 'setup.py develop': the scripts are then installed into the 'bin' directory of the virtualenv. [1] http://pypi.python.org/pypi/virtualenv Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIvrZK+gerLs4ltQ4RAjWmAJ0ZNgpTkV+APO3BtRwOCT8uNO3OYwCgji13 sQ3V2oNtjmlWvpc7OFhXaZU= =6Gb2 -----END PGP SIGNATURE----- From pje at telecommunity.com Wed Sep 3 18:34:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 03 Sep 2008 12:34:01 -0400 Subject: [Distutils] development-egg in current directory In-Reply-To: <48BEB09D.9030200@goebel-consult.de> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> <48BEB09D.9030200@goebel-consult.de> Message-ID: <20080903163301.EBF153A403D@sparrow.telecommunity.com> At 05:43 PM 9/3/2008 +0200, Hartmut Goebel wrote: >Phillip J. Eby schrieb: > > At 04:58 PM 9/3/2008 +0200, Hartmut Goebel wrote: > >> Hi, > >> > >> I'm currently installing development eggs into the current directory > >> (the checkout directory) like this: > >> > >> PYTHONPATH=.:$PYTHONPATH \ > >> python setup.py develop --install-dir . --script-dir . > >> > >> Since this has problems when re-running (see issue40), I wonder whether > >> this is the correct use of the 'develop' egg. Any hints? > > > > It's definitely not the correct use, especially since, depending on your > > layout, it could potentially overwrite your scripts. develop is > > intended for installing to a directory that's normally *already on* > > sys.path. Why are you doing that? > >Because I defined script-entry-points which I need to test. If there is >another way to so this, I'll happily change :-) So leave off the install-dir and script-dir arguments. By the way, if you specify an install-dir, the script-dir defaults to that directory, so your options are redundant anyway. If your goal is simply to test scripts without affecting anything else, the simple thing to do is: python setup.py develop -md /some/dir The -m means that /some/dir won't have to be on PYTHONPATH, and the -d is short for --install-dir. You can then invoke '/some/dir/myscript' to test. (Or if /some/dir is on your PATH, just run 'myscript'.) From h.goebel at goebel-consult.de Wed Sep 3 19:26:28 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 19:26:28 +0200 Subject: [Distutils] development-egg in current directory In-Reply-To: <20080903163301.EBF153A403D@sparrow.telecommunity.com> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> <48BEB09D.9030200@goebel-consult.de> <20080903163301.EBF153A403D@sparrow.telecommunity.com> Message-ID: <48BEC8C4.8060505@goebel-consult.de> Hi Phillip, Phillip J. Eby schrieb: > So leave off the install-dir and script-dir arguments. By the way, if If I leave both of, it tries to install into /usr/lib/python2.5/site-packages. > If your goal is simply to test scripts without affecting anything else, > the simple thing to do is: > > python setup.py develop -md /some/dir Thanks! I'll use python setup.py develop -md . which looks good for me. > you specify an install-dir, the script-dir defaults to that directory, > so your options are redundant anyway. .... > The -m means that /some/dir won't have to be on PYTHONPATH, I suggest adding this o the Wiki. It is not obvious, that -m means this. (I would do myself, but I'm not allowed to edit the page.) Thanks again! -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From h.goebel at goebel-consult.de Wed Sep 3 19:47:59 2008 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 03 Sep 2008 19:47:59 +0200 Subject: [Distutils] development-egg in current directory In-Reply-To: <48BEC8C4.8060505@goebel-consult.de> References: <48BEA5F9.7010502@goebel-consult.de> <20080903151629.DEC093A4105@sparrow.telecommunity.com> <48BEB09D.9030200@goebel-consult.de> <20080903163301.EBF153A403D@sparrow.telecommunity.com> <48BEC8C4.8060505@goebel-consult.de> Message-ID: <48BECDCF.3050205@goebel-consult.de> Hartmut Goebel schrieb: > Thanks! I'll use > python setup.py develop -md . > > which looks good for me. ... but has the same problem with the *.egg-link. :-( Okay, I'll use another directory. -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3195 bytes Desc: S/MIME Cryptographic Signature URL: From setuptools at bugs.python.org Thu Sep 4 12:25:40 2008 From: setuptools at bugs.python.org (htgoebel) Date: Thu, 04 Sep 2008 10:25:40 +0000 Subject: [Distutils] [issue41] need support for post-install scripts In-Reply-To: <1220523940.06.0.00557540451175.issue41@psf.upfronthosting.co.za> Message-ID: <1220523940.06.0.00557540451175.issue41@psf.upfronthosting.co.za> New submission from htgoebel : easy_install needs some mechanism for running post-installation scripts. This is required by eg. pywin32 (see issue18) and py2exe. If uninstall is implemented (see issue21), pre-/post-uninstall scripts need to be supported, too. I suggest adding new entry-points: - easy-install.post-install - easy-install.pre-uninstall - easy-install.post-uninstall (pre-install does not make sense, since typically the package can not be used prior to installation :-) ---------- messages: 154 nosy: htgoebel priority: feature status: unread title: need support for post-install scripts _______________________________________________ Setuptools tracker _______________________________________________ From sdouche at gmail.com Thu Sep 4 17:40:13 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Thu, 4 Sep 2008 17:40:13 +0200 Subject: [Distutils] [issue35] mercurial support In-Reply-To: <1219687219.16.0.365246980531.issue35@psf.upfronthosting.co.za> References: <1219687219.16.0.365246980531.issue35@psf.upfronthosting.co.za> <1219687219.16.0.365246980531.issue35@psf.upfronthosting.co.za> Message-ID: <5e1183fa0809040840t75f8c49nf32de97299e32bb3@mail.gmail.com> On Mon, Aug 25, 2008 at 20:00, Paul Egan wrote: > > New submission from Paul Egan : > > Patch to add mercurial support to setuptools: > * Extends setuptools.file_findersAdds with hg repository support > * Adds tag-hg-revision & filter-hgignore-files options to the egg-info command Hi Paul, Thanks for your patch. After some tests, like works great. Here, we think seriously to migrate on HG (as many OSS projects). Philipp, are you open to accepting HG/Bzr/git in Setuptools ? Cheers -- Seb From pje at telecommunity.com Thu Sep 4 19:07:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 04 Sep 2008 13:07:29 -0400 Subject: [Distutils] [issue35] mercurial support In-Reply-To: <5e1183fa0809040840t75f8c49nf32de97299e32bb3@mail.gmail.com > References: <1219687219.16.0.365246980531.issue35@psf.upfronthosting.co.za> <1219687219.16.0.365246980531.issue35@psf.upfronthosting.co.za> <5e1183fa0809040840t75f8c49nf32de97299e32bb3@mail.gmail.com> Message-ID: <20080904170629.739D03A403D@sparrow.telecommunity.com> At 05:40 PM 9/4/2008 +0200, Sebastien Douche wrote: >On Mon, Aug 25, 2008 at 20:00, Paul Egan wrote: > > > > New submission from Paul Egan : > > > > Patch to add mercurial support to setuptools: > > * Extends setuptools.file_findersAdds with hg repository support > > * Adds tag-hg-revision & filter-hgignore-files options to the > egg-info command > >Hi Paul, >Thanks for your patch. After some tests, like works great. Here, we >think seriously to migrate on HG (as many OSS projects). > >Philipp, are you open to accepting HG/Bzr/git in Setuptools ? No, but I'm open to patches that modify the core to accept plugins for functions that are currently hard-coded to CVS or SVN. File finding is already possible via plugins, and you can easily find such plugins by browsing the "setuptools plugins" category on PyPI. Revision tag information is the main thing not pluggable at the moment; patches welcome. (See also my notes on the above ticket for more info) From cdcasey at gmail.com Thu Sep 4 22:27:11 2008 From: cdcasey at gmail.com (chris) Date: Thu, 4 Sep 2008 15:27:11 -0500 Subject: [Distutils] access to setup() arguments Message-ID: If I have something like this in a setup.py setup( name = 'foo', ... ) is there a way to access name and get back foo? Thanks, Chris From cdcasey at gmail.com Thu Sep 4 22:28:26 2008 From: cdcasey at gmail.com (chris) Date: Thu, 4 Sep 2008 15:28:26 -0500 Subject: [Distutils] access to setup() arguments In-Reply-To: References: Message-ID: Sorry, should have been more explicit. I meant if I'm writing a custom command, is there a way to access that information in it? On Thu, Sep 4, 2008 at 3:27 PM, chris wrote: > If I have something like this in a setup.py > > setup( > name = 'foo', > ... > ) > > is there a way to access name and get back foo? > > Thanks, > Chris > From cdcasey at gmail.com Thu Sep 4 22:35:20 2008 From: cdcasey at gmail.com (chris) Date: Thu, 4 Sep 2008 15:35:20 -0500 Subject: [Distutils] access to setup() arguments In-Reply-To: References: Message-ID: Nevermind. got it. (self.distribution.get_name()) On Thu, Sep 4, 2008 at 3:28 PM, chris wrote: > Sorry, should have been more explicit. I meant if I'm writing a custom > command, is there a way to access that information in it? > > On Thu, Sep 4, 2008 at 3:27 PM, chris wrote: >> If I have something like this in a setup.py >> >> setup( >> name = 'foo', >> ... >> ) >> >> is there a way to access name and get back foo? >> >> Thanks, >> Chris >> > From michael.schmarck at habmalnefrage.de Fri Sep 5 15:48:25 2008 From: michael.schmarck at habmalnefrage.de (Michael Schmarck) Date: Fri, 5 Sep 2008 15:48:25 +0200 Subject: [Distutils] NameError: global name 'log' is not defined Message-ID: Hello. I'm trying to install Trac, and it failed with a NameError: global name 'log' is not defined error message. Because of that, I found the "the svn >= 1.5 bug with setuptools/command/sdist.py: global name 'log' is not defined" thread from August 2008 at http://mail.python.org/pipermail/distutils-sig/2008-August/009883.html Now I'm trying to solve that problem. I tried: easy_install -U setuptools==dev This fails: --($ ~/Source/Distutils-1.0.2)-- easy_install -U setuptools==dev Searching for setuptools==dev Reading http://pypi.python.org/simple/setuptools/ Best match: setuptools dev Downloading http://svn.python.org/projects/sandbox/trunk/setuptools/#egg=setuptools-dev Doing subversion checkout from http://svn.python.org/projects/sandbox/trunk/setuptools/ to /tmp/easy_install-rVsHxt/setuptools Processing setuptools Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-rVsHxt/setuptools/egg-dist-tmp-F3KSrb unrecognized .svn/entries format; skipping . Traceback (most recent call last): File "/export/home/webservd/.software/Python-2.5.2/bin/easy_install", line 8, in load_entry_point('setuptools==0.6c8', 'console_scripts', 'easy_install')() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 1671, in main File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 1675, in File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 211, in run File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 446, in easy_install File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 476, in install_item File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 655, in install_eggs File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 930, in build_and_install File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 919, in run_setup File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/sandbox.py", line 27, in run_setup File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/sandbox.py", line 63, in run File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/sandbox.py", line 29, in File "setup.py", line 95, in File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/bdist_egg.py", line 167, in run File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/egg_info.py", line 171, in run File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/egg_info.py", line 252, in find_sources File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/egg_info.py", line 306, in run File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/egg_info.py", line 333, in add_defaults File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 45, in walk_revctrl File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 52, in _default_revctrl File "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 98, in entries_finder NameError: global name 'log' is not defined Python 2.5.2 is installed in /export/home/.software/Python-2.5.2. This is a Sun Solaris 10 Sparc system, if it matters. What can be done to solve this problem? Cheers, Michael From ziade.tarek at gmail.com Fri Sep 5 16:21:56 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 5 Sep 2008 16:21:56 +0200 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: References: Message-ID: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> On Fri, Sep 5, 2008 at 3:48 PM, Michael Schmarck < michael.schmarck at habmalnefrage.de> wrote: > Hello. > > I'm trying to install Trac, and it failed with a > NameError: global name 'log' is not defined > error message. Because of that, I found the > "the svn >= 1.5 bug with setuptools/command/sdist.py: > global name 'log' is not defined" thread from > August 2008 at > http://mail.python.org/pipermail/distutils-sig/2008-August/009883.html > > Now I'm trying to solve that problem. I tried: > > easy_install -U setuptools==dev > > This fails: > > --($ ~/Source/Distutils-1.0.2)-- easy_install -U setuptools==dev > Searching for setuptools==dev > [cut] > File > "/export/home/webservd/.software/Python-2.5.2/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", > line 98, in entries_finder > NameError: global name 'log' is not defined > It looks like a egg-and-chicken problem, since the bug occur in the 0.6c8 version; I would try to remove setuptools-0.6c8-py2.5 completely before installing the trunk version Regards Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Fri Sep 5 16:23:48 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 5 Sep 2008 16:23:48 +0200 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> References: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> Message-ID: <94bdd2610809050723i1e1bec36t5db7353188126e65@mail.gmail.com> On Fri, Sep 5, 2008 at 4:21 PM, Tarek Ziad? wrote: > > > It looks like a egg-and-chicken problem, > Sorry, seems to be "chicken and an egg problem" in English. (not in french ;) ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.schmarck at habmalnefrage.de Sat Sep 6 13:12:30 2008 From: michael.schmarck at habmalnefrage.de (Michael Schmarck) Date: Sat, 6 Sep 2008 13:12:30 +0200 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> References: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> Message-ID: Hello! 2008/9/5 Tarek Ziad? : > I would try to remove setuptools-0.6c8-py2.5 completely before installing > the trunk version I noticed that I have the same problem on my ArchLinux installation at home. sudo python ez_setup.py setuptools==dev [...] File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 45, in walk_revctrl File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 52, in _default_revctrl File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", line 98, in entries_finder NameError: global name 'log' is not defined How DO I remove setuptools? Thanks, Michael From marius at pov.lt Sat Sep 6 17:09:47 2008 From: marius at pov.lt (Marius Gedminas) Date: Sat, 6 Sep 2008 11:09:47 -0400 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: References: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> Message-ID: <20080906150947.GA31357@platonas> On Sat, Sep 06, 2008 at 01:12:30PM +0200, Michael Schmarck wrote: > Hello! > > 2008/9/5 Tarek Ziad? : > > > I would try to remove setuptools-0.6c8-py2.5 completely before installing > > the trunk version > > I noticed that I have the same problem on my ArchLinux > installation at home. > > sudo python ez_setup.py setuptools==dev > [...] > File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", > line 45, in walk_revctrl > File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", > line 52, in _default_revctrl > File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", > line 98, in entries_finder > NameError: global name 'log' is not defined > > How DO I remove setuptools? Downgrade subversion to 1.4 (or remove it altogether), upgrade setuptools, upgrade subversion back to 1.5? Marius Gedminas -- C++ is a loaded machine gun helpfully pointed at your feet with the safety off. -- ChaosDiscord on Slashdot -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From michael.schmarck at habmalnefrage.de Sat Sep 6 19:41:09 2008 From: michael.schmarck at habmalnefrage.de (Michael Schmarck) Date: Sat, 6 Sep 2008 19:41:09 +0200 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: <5FCC81C3-F70D-4D51-A2CC-F87B5621B822@durin42.com> References: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> <5FCC81C3-F70D-4D51-A2CC-F87B5621B822@durin42.com> Message-ID: Hi! 2008/9/6 Augie Fackler : > > On Sep 6, 2008, at 6:12 AM, Michael Schmarck wrote: > >> Hello! >> >> 2008/9/5 Tarek Ziad? : >> >>> I would try to remove setuptools-0.6c8-py2.5 completely before installing >>> the trunk version >> >> I noticed that I have the same problem on my ArchLinux >> installation at home. >> >> sudo python ez_setup.py setuptools==dev >> [...] >> File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", >> line 45, in walk_revctrl >> File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", >> line 52, in _default_revctrl >> File "/tmp/setuptools-0.6c8-py2.5.egg/setuptools/command/sdist.py", >> line 98, in entries_finder >> NameError: global name 'log' is not defined >> >> How DO I remove setuptools? > > An alternate solution would be to get the change-svn-wc-format.py script > from the Subversion repository and downgrade your working copy of setuptools > to that format. Hm. I don't know if I even *have* setuptools installed. How do I find out? In site-packages, there's no setuptools egg and a "find /usr/lib/python2.5 -name sandbox.py" returns nothing. So the question is: How do you install setuptools, if Subversion 1.5 is installed? Best regards, Michael From sdouche at gmail.com Sun Sep 7 18:58:39 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Sun, 7 Sep 2008 18:58:39 +0200 Subject: [Distutils] [Setuptools] VCS agnostic Message-ID: <5e1183fa0809070958t6ae3e348vd71a5000673c656f@mail.gmail.com> Hi, I wonder about the way of making setuptools independent of VCS. After a fast reading, 2 functions engage the attention : - prune_file_list (egg_info.py) - copytree (install_egg_info.py) These functions manipulate VCS directories (RCS, CVS, .svn). Instead of listing explicitly each VCS directory, I propose to replace by all hidden directories (.*). Recents VCS uses them (.git, .svn, .bzr, .hg ...). Other ideas ? -- Seb From pje at telecommunity.com Sun Sep 7 19:22:31 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 07 Sep 2008 13:22:31 -0400 Subject: [Distutils] [Setuptools] VCS agnostic In-Reply-To: <5e1183fa0809070958t6ae3e348vd71a5000673c656f@mail.gmail.co m> References: <5e1183fa0809070958t6ae3e348vd71a5000673c656f@mail.gmail.com> Message-ID: <20080907172130.2ECDF3A4072@sparrow.telecommunity.com> At 06:58 PM 9/7/2008 +0200, Sebastien Douche wrote: >Hi, >I wonder about the way of making setuptools independent of VCS. After >a fast reading, 2 functions engage the attention : >- prune_file_list (egg_info.py) >- copytree (install_egg_info.py) > >These functions manipulate VCS directories (RCS, CVS, .svn). Instead >of listing explicitly each VCS directory, I propose to replace by all >hidden directories (.*). Recents VCS uses them (.git, .svn, .bzr, .hg >...). > >Other ideas ? Plugins. (This would also be the case for the revision-number finding code.) We could just add a "setuptools.vcs_filters" entry point group wherein filter functions could be registered. Most VCS support is through plugins anyhow, so it's no big deal for them to define a function and add another entry point. From sdouche at gmail.com Sun Sep 7 19:45:34 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Sun, 7 Sep 2008 19:45:34 +0200 Subject: [Distutils] [Setuptools] VCS agnostic In-Reply-To: <20080907172130.2ECDF3A4072@sparrow.telecommunity.com> References: <5e1183fa0809070958t6ae3e348vd71a5000673c656f@mail.gmail.com> <20080907172130.2ECDF3A4072@sparrow.telecommunity.com> Message-ID: <5e1183fa0809071045g3f781c9dnbcc1e190d27a6880@mail.gmail.com> On Sun, Sep 7, 2008 at 19:22, Phillip J. Eby wrote: > Plugins. (This would also be the case for the revision-number finding > code.) We could just add a "setuptools.vcs_filters" entry point group > wherein filter functions could be registered. Most VCS support is through > plugins anyhow, so it's no big deal for them to define a function and add > another entry point. Hmm, for filters too ? Ok :). -- Seb From michael.schmarck at habmalnefrage.de Mon Sep 8 10:30:18 2008 From: michael.schmarck at habmalnefrage.de (Michael Schmarck) Date: Mon, 8 Sep 2008 10:30:18 +0200 Subject: [Distutils] NameError: global name 'log' is not defined In-Reply-To: <20080906150947.GA31357@platonas> References: <94bdd2610809050721ge560be3ic65578b25170907c@mail.gmail.com> <20080906150947.GA31357@platonas> Message-ID: Hello. 2008/9/6 Marius Gedminas : > Downgrade subversion to 1.4 (or remove it altogether), upgrade > setuptools, upgrade subversion back to 1.5? Okay. I removed Subversion 1.5. I then only had Subversion 1.4 at some other directory. What do you do next? I did: python2.5 ~/Source/ez_setup.py setuptools==dev06 This installed a setuptools-0.6c9dev_r65963-py2.5.egg to my site-packages directory. Now I'm trying to use setuptools: --($ ~/Source/Trac-plugins/emoticonsplugin-0.9)-- python setup.py bdist_egg Traceback (most recent call last): File "setup.py", line 20, in from setuptools import setup, find_packages ImportError: No module named setuptools I also found the setuptools.pth file: --($ ~/.software/Python-2.5.2/lib/python2.5/site-packages)-- cat setuptools.pth ./setuptools-0.6c8-py2.5.egg Where did that file come from? Judging from the date, it must have been a result of "python2.5 ~/Source/ez_setup.py setuptools==dev06". When I do echo ./setuptools-0.6c9dev_r65963-py2.5.egg > setuptools.pth I then can use setuptools (ie. the command "python setup.py bdist_egg" did not return an error). Did I do something wrong? I put back the Svn 1.5 installation and "python setup.py bdist_egg" still works. Thanks, Michael From cz at gocept.com Tue Sep 9 10:05:15 2008 From: cz at gocept.com (Christian Zagrodnick) Date: Tue, 9 Sep 2008 10:05:15 +0200 Subject: [Distutils] the svn >= 1.5 bug with setuptools/command/sdist.py: global name 'log' is not defined References: <20080819035610.GA3012@facebook.com> <20080819052919.GG87062@nexus.in-nomine.org> <5e1183fa0808190544l67e82675ib993b558cca40642@mail.gmail.com> <94bdd2610808190609j302ca465j50a5ed846baffffd@mail.gmail.com> <20080819134019.6DCAD3A4105@sparrow.telecommunity.com> <5e1183fa0808190838l58a44a4btcac7cd0e8a178cc2@mail.gmail.com> Message-ID: On 2008-08-19 17:38:00 +0200, "Sebastien Douche" said: > On Tue, Aug 19, 2008 at 15:41, Phillip J. Eby wrote: >> At 03:09 PM 8/19/2008 +0200, Tarek Ziad? wrote: >>> >>> So, setuptools should use a random, unique, temporary name in this >>> function imho >>> to make buildout happy with any kind of dev package. >> >> Um, no. easy_install does *each* download to a unique temporary director > y. >> If buildout is doing something else, it's buildout that's broken. > > I'm talking about the critical bug of Setuptools with SVN 1.5 ( commit > r65222). So I ask again, why setuptools 0.9c9 is not released ? Right. It would *really* help to have another release. *Please*. -- Christian Zagrodnick ? cz at gocept.com gocept gmbh & co. kg ? forsterstra?e 29 ? 06112 halle (saale) ? germany http://gocept.com ? tel +49 345 1229889 4 ? fax +49 345 1229889 1 Zope and Plone consulting and development From sdouche at gmail.com Tue Sep 9 13:12:12 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Tue, 9 Sep 2008 13:12:12 +0200 Subject: [Distutils] [Setuptools] VCS agnostic In-Reply-To: <20080907172130.2ECDF3A4072@sparrow.telecommunity.com> References: <5e1183fa0809070958t6ae3e348vd71a5000673c656f@mail.gmail.com> <20080907172130.2ECDF3A4072@sparrow.telecommunity.com> Message-ID: <5e1183fa0809090412s5ad2a11ag3eebec850e1c033d@mail.gmail.com> On Sun, Sep 7, 2008 at 19:22, Phillip J. Eby wrote: > Plugins. (This would also be the case for the revision-number finding > code.) We could just add a "setuptools.vcs_filters" entry point group > wherein filter functions could be registered. Most VCS support is through > plugins anyhow, so it's no big deal for them to define a function and add > another entry point. http://bugs.python.org/issue3814 A first attempt. -- Seb From setuptools at bugs.python.org Tue Sep 9 15:09:47 2008 From: setuptools at bugs.python.org (Douche) Date: Tue, 09 Sep 2008 13:09:47 +0000 Subject: [Distutils] [issue42] Add VCS support In-Reply-To: <1220965786.99.0.313747610202.issue42@psf.upfronthosting.co.za> Message-ID: <1220965786.99.0.313747610202.issue42@psf.upfronthosting.co.za> New submission from Douche : This is a basic try (4h) to extract specific SVN code out of the core and add entry points for: - filters (.svn, .hg, ...) - find files in repos - get revision's number Thoughts: - if repos have 2 VCS entries (.svn and .hg for example), the first valid entry point is used. - walk_revctrl function needs more love. I keep the philosophy of iterator but is it the good way ? Notes: - vcs_svn.py & vcs_hg.py are only for demo. The natural place are into Setuptools plugins. - filters must return a list. - revisions must return a int or None (not 0). Critics welcome. ---------- files: setuptools-sd-20080908.patch messages: 160 nosy: sdouche priority: feature status: unread title: Add VCS support Added file: http://bugs.python.org/setuptools/file21/setuptools-sd-20080908.patch _______________________________________________ Setuptools tracker _______________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: setuptools-sd-20080908.patch Type: text/x-diff Size: 12855 bytes Desc: not available URL: From james.seetoo at invitrogen.com Tue Sep 9 19:57:44 2008 From: james.seetoo at invitrogen.com (James Seetoo) Date: Tue, 9 Sep 2008 13:57:44 -0400 Subject: [Distutils] Hoping to network with you Message-ID: <2C4C90681A1548F2A3F77BF0462743C3@xoominfo.com> Dear Anthony, My name is James Seetoo and I'm with Invitrogen Corporation's Talent Acquisition Group. I'm working on a project to bring a Sr. Manager, Release Engineering to Invitrogen for our Frederick MD Data Center. This will be a global position and I am specifically looking for someone who has experience with ERP systems (JDE is a major plus) and/or Enterprise wide CRM systems such as Siebel. I would very much appreciate the chance to network with you on this and would be happy to call you at your convenience. Thanks in advance for any help you can give me. Best, James James Seetoo Interested in this job opportunity? Click here to learn more... Position Title: Sr. Manager, Release Engineering Location: Carlsbad, CA Description: Sr. manager role oversees all aspects of the release engineering and application deployment processes including: coordination of project activities pertaining to software builds, automation and deployments as well as configuration elements within the Quality Assurance, Staging, Customer Test, Training, and Production environments. The individual must be familiar with standard concepts, best practices, and procedures for release management and applications configuration management in global enterprise environments. The individual will be responsible for developing and maintaining global release management policies and procedures. Direct reports to this manager will be responsible for the release management of products including but not limited to: Oracle Enterprise One, Siebel, PeopleSoft HR, Agile, Cognos, Traxi3, Commergent, Hyperion, FAST and custom e_Commerce applications. Experience with these or similar products as well as scripting and programming experience (sh, Perl, C, Java, J2EE, VBasic) is highly desirable. This position requires strong organizational and communications skills to ensure a high level of internal coordination, and customer satisfaction. Typical Responsibilities: Defines and manages the release and build processes for QA, Test, and Production systems. Works with the infrastructure teams, implementation teams, and support teams to set up hosted application environments for development, testing, staging, and quality assurance. Understands and communicates system architectures and capabilities and is able to effectively troubleshoot compile, build, and integration failures. Defines and implements quality procedures to ensure effectiveness, including peer reviews of upgrade plans, queries and result sets. Establishes and maintains source control standards for customer releases (using Perforce or application specific management tools). Provides planning tools and feedback concerning upgrade plans, environment refreshes, promotions, and implementations, to insure successful completion of initiatives. Assists with corporate implementations and training as needed. Performs management responsibilities including performance evaluations, schedule time off, resource evaluation, etc. Job Requirements: Bachelor degree in Computer Science, Computer Engineering or a related field or the equivalent of 10+ years experience with releasing software. Supervisory or management experience required Maintains a relationship with the Infra account manager and support teams as well as manages licenses and tracking. Experience with Incident, Problem, Change, Configuration and Asset Management processes and workflow. Knowledge of ITIL and other IT Best Practices frameworks Excellent verbal communication skills. Excellent documentation and written communication skills. Superior organizational skills and attention to detail, including time management skills and highly tuned problem solving and analytical skills. Demonstrated ability to think out of the box and generate creative solutions. Flexibility that allows effective teamwork with people at all levels of the organization. Experience with Project Management Processes and Methodologies. Organization Culture: Candidate must thrive in a fast-paced, ever changing, entrepreneurial environment. Individual must be able to work independently as well as collaboratively. Must be resourceful and be able to understand the full breadth of how the company works together. Individual must be a creative thinker, confident and able to express ideas in an articulate, well thought out manner. Ability to multi-task and prioritize deadlines is a must. Options: . I would like to learn more . Keep me informed of other relevant opportunities . Forward this opportunity to a friend This message was sent by: Invitrogen Corporation Invitrogen Corporation (NASDAQ: IVGN) is a global company with revenues of $1.15 billion. We provide products and services to pharmaceutical and biotechnology companies, as well as academic and government research institutions. More than 4,300 employees work for Invitrogen and we conduct business in over 70 countries. We offer more than 25,000 unique products and services to support disease research, drug discovery and commercial bio-production. The company's areas of focus include genomics, proteomics, bioinformatics, cell culture, and cell biology, among others. Importantly, we continue to expand our capabilities by partnering with the brightest minds in the world on revolutionary diagnostic methods and therapeutic treatments. Invitrogen's product family includes many of the most widely recommended names in the industry, including Molecular Probes, Gibco, and Dynal Biotech. Our products can be found in nearly every major laboratory in the world. Many of the greatest medical discoveries of the last two decades were made using Invitrogen products, including the discovery of the AIDS virus, advancements in cancer treatment, and the development of tools to assist in stem cell research. Invitrogen's Quest is to better the human condition through innovations in science and technology. Through our Passionate People Making a Difference program, we serve as good corporate citizens in the communities in which we live and work. Founded in 1987, Invitrogen is headquartered in Carlsbad, California. 1620 Faraday Ave. Carlsbad, CA 92008 Invitrogen Corporation respects your online privacy. If you would like to be removed from future Invitrogen Corporation mailings, you may unsubscribe by clicking here . -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdcasey at gmail.com Tue Sep 9 20:59:33 2008 From: cdcasey at gmail.com (chris) Date: Tue, 9 Sep 2008 13:59:33 -0500 Subject: [Distutils] keywords in setup() Message-ID: Is there a specific set of keywords that can go into the setup() function, or is it possible to add arbitrary keywords? From pje at telecommunity.com Tue Sep 9 22:36:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 09 Sep 2008 16:36:42 -0400 Subject: [Distutils] setuptoosl bug tracker? In-Reply-To: <94E2A58978FC324196A142DE8399AB78A9CF9F@chsa1556.share.belu ni.net> References: <94E2A58978FC324196A142DE8399AB78A9CF9F@chsa1556.share.beluni.net> Message-ID: <20080909203541.8CD9B3A4072@sparrow.telecommunity.com> At 08:38 PM 9/9/2008 +0200, Thurner Rupert (KCBD 141) wrote: >hi phil, > >does setuptools have a bug tracker or mailing list? The distutils-sig mailing list, as stated in the setuptools documentation. There is also a bug tracker, but what you've reported is not a bug. (Also, my name is not "phil".) >our problem areas are: >we run http://www.blastwave.org and try to install >http://genshi.edgewall.org which bails out with: > ># ls -l /opt/csw/lib/libpython2.5.so >lrwxrwxrwx 1 root root 19 Jul 18 23:14 >/opt/csw/lib/libpython2.5.so -> libpython2.5.so.1.0 > ># easy_install --prefix /opt/csw/testing >genshi >Searching for genshi >Best match: Genshi 0.6dev-r914 >Processing Genshi-0.6dev_r914-py2.5-solaris-2.10-sun4v.egg >Genshi 0.6dev-r914 is already the active version in easy-install.pth > >Using >/opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914-py2.5-solaris-2.10-sun4v.egg >Processing dependencies for genshi >Finished processing dependencies for genshi >root at chvp011vs035 / ># rm >/opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914-py2.5-solaris-2.10-sun4v.egg >root at chvp011vs035 / ># easy_install --prefix /opt/csw/testing genshi >Searching for genshi >Reading http://pypi.python.org/simple/genshi/ >Couldn't find index page for 'genshi' (maybe misspelled?) >Scanning index of all packages (this may take a while) >Reading http://pypi.python.org/simple/ >Reading http://pypi.python.org/simple/Genshi/ >Reading http://genshi.edgewall.org/ >Reading http://genshi.edgewall.org/wiki/Download >Best match: Genshi 0.5.1 >Downloading http://ftp.edgewall.com/pub/genshi/Genshi-0.5.1.zip >Processing Genshi-0.5.1.zip >Running Genshi-0.5.1/setup.py -q bdist_egg --dist-dir >/tmp/easy_install-kz7muh/Genshi-0.5.1/egg-dist-tmp--y0Vju >warning: no previously-included files found matching 'doc/2000ft.graffle' >warning: no previously-included files matching '*' found under >directory 'doc/logo.lineform' >ld: fatal: library -lpython2.5: not found You are evidently missing the python-dev, python-devel, or whatever package on your OS is required to compile Python extensions. Install that first. >and the second one: ># easy_install >http://svn.ipd.uka.de/repos/javaparty/JP/trac/plugins/redirect-0.11/ >Downloading >http://svn.ipd.uka.de/repos/javaparty/JP/trac/plugins/redirect-0.11/ >error: Unexpected HTML page found at >http://svn.ipd.uka.de/repos/javaparty/JP/trac/plugins/redirect-0.11/ > >(we use svn 1.4.5, the counterparty svn-1.5) This is a known issue that is fixed in the setuptools trunk. You can use "easy_install setuptools==dev06" to upgrade to a version that doesn't have the problem. Once you've upgraded, it will also be safe to use svn 1.5. From skip at pobox.com Wed Sep 10 13:13:38 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 10 Sep 2008 06:13:38 -0500 Subject: [Distutils] Getting source from __file__ in egg? Message-ID: <18631.44002.621475.438948@montanaro-dyndns-org.local> If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't: % python Python 2.4.5 (#4, Apr 12 2008, 09:09:16) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import magnitude >>> magnitude.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3-py2.4.egg/magnitude.pyc' >>> magnitude.__file__[:-1] '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3-py2.4.egg/magnitude.py' >>> import os >>> os.path.exists(magnitude.__file__[:-1]) False How do I actually lay my hands on the .py file source for the magnitude module in this case? Do I have to recognize somehow that .../magnitude-0.9.3-py2.4.egg is a zip archive and resort to extracting the Python source with the zipfile module? Thx, Skip Montanaro From fadhley.salim at uk.calyon.com Wed Sep 10 13:40:24 2008 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Wed, 10 Sep 2008 12:40:24 +0100 Subject: [Distutils] Getting source from __file__ in egg? References: <18631.44002.621475.438948@montanaro-dyndns-org.local> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB201EC4A01@MXCU10MX1.MSX.CIB> Try pkg_resources.resource_string or pkg_resources.resource_file - it allows you to retrieve any object in an egg as a string or file-like object. :-) -----Original Message----- From: distutils-sig-bounces+fadhley.salim=uk.calyon.com at python.org [mailto:distutils-sig-bounces+fadhley.salim=uk.calyon.com at python.org] On Behalf Of skip at pobox.com Sent: 10 September 2008 12:14 To: distutils-sig at python.org Subject: [Distutils] Getting source from __file__ in egg? If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't: % python Python 2.4.5 (#4, Apr 12 2008, 09:09:16) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import magnitude >>> magnitude.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3 -py2.4.egg/magnitude.pyc' >>> magnitude.__file__[:-1] '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3 -py2.4.egg/magnitude.py' >>> import os >>> os.path.exists(magnitude.__file__[:-1]) False How do I actually lay my hands on the .py file source for the magnitude module in this case? Do I have to recognize somehow that .../magnitude-0.9.3-py2.4.egg is a zip archive and resort to extracting the Python source with the zipfile module? Thx, Skip Montanaro _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org http://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Commission Bancaire in France and regulated by the Financial Services Authority for the conduct of business in the United Kingdom. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From rupert.thurner at gmail.com Wed Sep 10 15:25:30 2008 From: rupert.thurner at gmail.com (rupert.thurner) Date: Wed, 10 Sep 2008 06:25:30 -0700 (PDT) Subject: [Distutils] Your confirmation is required to join the Distutils-SIG mailing list In-Reply-To: References: Message-ID: <1ea5aef1-30db-443b-bddb-c75d655b32f9@f36g2000hsa.googlegroups.com> On Sep 10, 3:23?pm, distutils-sig-confirm +30f974c694b56e10af63982110a7bbcb86a5e... at python.org wrote: > Mailing list subscription confirmation notice for mailing list > Distutils-SIG > > We have received a request from 198.240.213.25 for subscription of > your email address, "distutils-sig-garchive-52934 at googlegroups.com", > to the distutils-... at python.org mailing list. ?To confirm that you > want to be added to this mailing list, simply reply to this message, > keeping the Subject: header intact. ?Or visit this web page: > > ? ?http://mail.python.org/mailman/confirm/distutils-sig/30f974c694b56e10... > > Or include the following line -- and only the following line -- in a > message to distutils-sig-requ... at python.org: > > ? ? confirm 30f974c694b56e10af63982110a7bbcb86a5eb1b > > Note that simply sending a `reply' to this message should work from > most mail readers, since that usually leaves the Subject: line in the > right form (additional "Re:" text in the Subject: is okay). > > If you do not wish to be subscribed to this list, please simply > disregard this message. ?If you think you are being maliciously > subscribed to the list, or have any other questions, send them to > distutils-sig-ow... at python.org. From cdcasey at gmail.com Wed Sep 10 17:41:48 2008 From: cdcasey at gmail.com (chris) Date: Wed, 10 Sep 2008 10:41:48 -0500 Subject: [Distutils] passing arguments to self.run_command Message-ID: Is there a way to pass arguments to a command called by self.run_command? Anytime I try to add arguments by putting them inside the quotes I get an invalid command error, and self.run_command itself doesn't take any other arguments. Am I missing something? -Chris From zooko at zooko.com Wed Sep 10 19:27:56 2008 From: zooko at zooko.com (zooko) Date: Wed, 10 Sep 2008 11:27:56 -0600 Subject: [Distutils] Getting source from __file__ in egg? In-Reply-To: <18631.44002.621475.438948@montanaro-dyndns-org.local> References: <18631.44002.621475.438948@montanaro-dyndns-org.local> Message-ID: <947197CE-AD02-42FB-8445-F8C4FABCEDDC@zooko.com> On Sep 10, 2008, at 5:13 AM, skip at pobox.com wrote: > If I have a package installed as a zip file egg and ask for its > __file__ I > might get a reference which looks like a normal file system path > but isn't: I advise people to add to their distutils config file: [easy_install] zip_ok=False In addition, I suggest that setuptools should be changed to install eggs unzipped by default, as there is a list of problems and inconveniences with zipping eggs and, as far as I am aware, no measured benefits. Here is the issue ticket, in which your problem, Skip, is the most recent entry in the list: http://bugs.python.org/setuptools/issue33 Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From rupert.thurner at gmail.com Wed Sep 10 20:04:53 2008 From: rupert.thurner at gmail.com (rupert.thurner) Date: Wed, 10 Sep 2008 11:04:53 -0700 (PDT) Subject: [Distutils] compiling an extension on solaris: genshi, mercurial, ... Message-ID: <48c5fb00-795a-4d0f-89ca-f81260a809cb@l64g2000hse.googlegroups.com> hi, we run http://www.blastwave.org and try to install http://genshi.edgewall.org which bails out with (btw, the same happens with mercurial version control): # ls -l /opt/csw/lib/libpython2.5.so lrwxrwxrwx 1 root root 19 Jul 18 23:14 /opt/csw/lib/ libpython2.5.so -> libpython2.5.so.1.0 # easy_install --prefix /opt/csw/testing genshi Searching for genshi Best match: Genshi 0.6dev-r914 Processing Genshi-0.6dev_r914-py2.5-solaris-2.10-sun4v.egg Genshi 0.6dev-r914 is already the active version in easy-install.pth Using /opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914- py2.5-solaris-2.10-sun4v.egg Processing dependencies for genshi Finished processing dependencies for genshi root at chvp011vs035 / # rm /opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914- py2.5-solaris-2.10-sun4v.egg root at chvp011vs035 / # easy_install --prefix /opt/csw/testing genshi Searching for genshi Reading http://pypi.python.org/simple/genshi/ Couldn't find index page for 'genshi' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ Reading http://pypi.python.org/simple/Genshi/ Reading http://genshi.edgewall.org/ Reading http://genshi.edgewall.org/wiki/Download Best match: Genshi 0.5.1 Downloading http://ftp.edgewall.com/pub/genshi/Genshi-0.5.1.zip Processing Genshi-0.5.1.zip Running Genshi-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/ easy_install-kz7muh/Genshi-0.5.1/egg-dist-tmp--y0Vju warning: no previously-included files found matching 'doc/ 2000ft.graffle' warning: no previously-included files matching '*' found under directory 'doc/logo.lineform' ld: fatal: library -lpython2.5: not found ld: fatal: File processing errors. No output written to build/ lib.solaris-2.10-sun4v-2.5/genshi/_speedups.so ********************************************************************** WARNING: An optional C extension could not be compiled, speedups will not be available. ********************************************************************** error: Setup script exited with error: can't copy 'Genshi.egg-info/ native_libs.txt': doesn't exist or not a regular file anybody has an idea how to convince setuptools to use the python library? python works and it should be in the library path though. From setuptools at bugs.python.org Thu Sep 11 07:43:18 2008 From: setuptools at bugs.python.org (vajda) Date: Thu, 11 Sep 2008 05:43:18 +0000 Subject: [Distutils] [issue43] add force_shared Library option to create shared lib even with use_stubs=False In-Reply-To: <1221111798.16.0.734112264299.issue43@psf.upfronthosting.co.za> Message-ID: <1221111798.16.0.734112264299.issue43@psf.upfronthosting.co.za> New submission from vajda : setuptools is growing the capability to build regular shared libraries (as opposed to python extensions). JCC (http://svn.osafoundation.org/pylucene/trunk/jcc/jcc) uses this capability to build the JCC runtime into a regular shared library shared by all python extensions it builds and by programs embedding python (such as a Java VM when running JCC-built eggs from Apache Tomcat). This bug is about adding another option to the setuptools Library class called force_shared which forces setuptools to create a shared library from a Library instance even though the dl module may not be present to generate stubs. This is important on Linux. Note that using this flag then implies that the library itself is responsible for calling dlopen(buf, RTLD_NOW | RTLD_GLOBAL) on the relevant libpython.so before initializing the python runtime is initialized. A patch against the setuptools 0.6 branch svn is attached. The idea for this patch came from a conversation on IRC: http://chandlerproject.org/script/getIrcTranscript.cgi?channel=chandler&date=20080910&startTime=1729 ---------- assignedto: pje files: patch.st messages: 162 nosy: pje, vajda priority: feature status: unread title: add force_shared Library option to create shared lib even with use_stubs=False Added file: http://bugs.python.org/setuptools/file22/patch.st _______________________________________________ Setuptools tracker _______________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: patch.st Type: application/octet-stream Size: 5934 bytes Desc: not available URL: From pje at telecommunity.com Thu Sep 11 18:03:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 11 Sep 2008 12:03:29 -0400 Subject: [Distutils] Getting source from __file__ in egg? In-Reply-To: <947197CE-AD02-42FB-8445-F8C4FABCEDDC@zooko.com> References: <18631.44002.621475.438948@montanaro-dyndns-org.local> <947197CE-AD02-42FB-8445-F8C4FABCEDDC@zooko.com> Message-ID: <20080911160228.7EDAD3A403D@sparrow.telecommunity.com> Skip's issue isn't going to be resolved by avoiding zipfiles in the easy_install case. Really, none of the other issues you've mentioned in that ticket will be either. Zipfiles are a useful installation target for drop-and-go plugin installation, and they're useful for py2exe, py2app, etc. A library that has problems being deployed in a zipfile will have problems in other places besides easy_install. The fix isn't to stop using zipfiles, it's to fix the places in Python (and 3rd-party tools) where zipfile support isn't finished. easy_install simply forces the errors to show up earlier... and for the community as a whole, that's a *good* thing. (Oh, and while we're suggesting things, zooko, I'd suggest you take a look at the two setuptools bugs in "testing" status that are assigned to you; they're holding up the 0.6c9 release.) At 11:27 AM 9/10/2008 -0600, zooko wrote: >On Sep 10, 2008, at 5:13 AM, skip at pobox.com wrote: > >>If I have a package installed as a zip file egg and ask for its >>__file__ I >>might get a reference which looks like a normal file system path >>but isn't: > >I advise people to add to their distutils config file: > >[easy_install] >zip_ok=False > >In addition, I suggest that setuptools should be changed to install >eggs unzipped by default, as there is a list of problems and >inconveniences with zipping eggs and, as far as I am aware, no >measured benefits. > >Here is the issue ticket, in which your problem, Skip, is the most >recent entry in the list: > >http://bugs.python.org/setuptools/issue33 > >Regards, > >Zooko >--- >http://allmydata.org -- Tahoe, the Least-Authority Filesystem >http://allmydata.com -- back up all your files for $5/month >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig From zooko at zooko.com Thu Sep 11 18:21:25 2008 From: zooko at zooko.com (zooko) Date: Thu, 11 Sep 2008 10:21:25 -0600 Subject: [Distutils] Getting source from __file__ in egg? In-Reply-To: <20080911160228.7EDAD3A403D@sparrow.telecommunity.com> References: <18631.44002.621475.438948@montanaro-dyndns-org.local> <947197CE-AD02-42FB-8445-F8C4FABCEDDC@zooko.com> <20080911160228.7EDAD3A403D@sparrow.telecommunity.com> Message-ID: On Sep 11, 2008, at 10:03 AM, Phillip J. Eby wrote: > (Oh, and while we're suggesting things, zooko, I'd suggest you take > a look at the two setuptools bugs in "testing" status that are > assigned to you; they're holding up the 0.6c9 release.) I will be able to look at those next week. Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From python at venix.com Thu Sep 11 19:06:20 2008 From: python at venix.com (Python) Date: Thu, 11 Sep 2008 13:06:20 -0400 Subject: [Distutils] Getting source from __file__ in egg? In-Reply-To: <20080911160228.7EDAD3A403D@sparrow.telecommunity.com> References: <18631.44002.621475.438948@montanaro-dyndns-org.local> <947197CE-AD02-42FB-8445-F8C4FABCEDDC@zooko.com> <20080911160228.7EDAD3A403D@sparrow.telecommunity.com> Message-ID: <1221152780.2413.289.camel@www.venix.com> On Thu, 2008-09-11 at 12:03 -0400, Phillip J. Eby wrote: > A library that has problems being deployed in a > zipfile will have problems in other places besides easy_install. My problems with libraries in zipfiles have generally revolved around limited user accounts that cannot handle the unzip. A normal python installation provides files than can be used directly. So I've configured the distutils configuration file to always unzip. I've found it easier for the systems I administer to force all installations to be unzipped rather than trying to find those accounts that will have problems with zipped libraries. These errors can be pretty mysterious to a sysadmin who is not fluent in Python. It can also be pretty embarrassing when the test system works and the live system fails because of the change in user account. If there is a better/smarter approach for installing zipped libraries I'd be happy to use it. -- Lloyd Kvam Venix Corp DLSLUG/GNHLUG library http://dlslug.org/library.html http://www.librarything.com/catalog/dlslug http://www.librarything.com/rsshtml/recent/dlslug http://www.librarything.com/rss/recent/dlslug From yves.moisan at boreal-is.com Thu Sep 11 22:55:24 2008 From: yves.moisan at boreal-is.com (Yves Moisan) Date: Thu, 11 Sep 2008 16:55:24 -0400 Subject: [Distutils] Buildout to run setup.py Message-ID: <1221166524.6323.9.camel@dell-desktop.example.com> Hi All, I'm new at buildout and I was wondering if it was possible to emulate a system command like "python setup.py install" in buildout. Use Case : [buildout] parts = MapFishTrunk [MapFishTrunk] recipe = infrae.subversion urls = http://www.mapfish.org/svn/mapfish/trunk/MapFish MapFish interpreter = Scripts\python There is a setup.py file a couple of levels down the directory that I would like to execute to build the eggs (and dependencies) and to store the resulting eggs in a location accessible to Scripts\python (which was generated using virtualenv). I can run Scripts/python setup.py install but I wonder if that can be done within buildout. TIA, Yves Moisan From jim at zope.com Thu Sep 11 23:39:14 2008 From: jim at zope.com (Jim Fulton) Date: Thu, 11 Sep 2008 17:39:14 -0400 Subject: [Distutils] Buildout to run setup.py In-Reply-To: <1221166524.6323.9.camel@dell-desktop.example.com> References: <1221166524.6323.9.camel@dell-desktop.example.com> Message-ID: On Sep 11, 2008, at 4:55 PM, Yves Moisan wrote: > Hi All, > > I'm new at buildout and I was wondering if it was possible to > emulate a > system command like "python setup.py install" in buildout. Use Case : > > [buildout] > parts = > MapFishTrunk > > > [MapFishTrunk] > recipe = infrae.subversion > urls = > http://www.mapfish.org/svn/mapfish/trunk/MapFish MapFish > interpreter = Scripts\python > > There is a setup.py file a couple of levels down the directory that I > would like to execute to build the eggs (and dependencies) and to > store > the resulting eggs in a location accessible to Scripts\python (which > was > generated using virtualenv). I can run Scripts/python setup.py > install > but I wonder if that can be done within buildout. I'm not entirely sure what you're trying to do. Buildout is capable of building develop eggs from source directories. I don't think you can build regular eggs from source directories. You also want to run the setup with certain distributions in the path. There isn't support for doing that presently, although folks would find it useful. It would be possible to write a recipe that did both of these things. You would then define a part for the egg you wanted to install, where that part would use the recipe. You'd want to list that part before other parts that need that egg. Jim -- Jim Fulton Zope Corporation From yves.moisan at boreal-is.com Fri Sep 12 14:28:18 2008 From: yves.moisan at boreal-is.com (Yves Moisan) Date: Fri, 12 Sep 2008 08:28:18 -0400 Subject: [Distutils] Buildout to run setup.py In-Reply-To: References: <1221166524.6323.9.camel@dell-desktop.example.com> Message-ID: <1221222498.6264.15.camel@dell-desktop.example.com> > > I'm not entirely sure what you're trying to do. > > Buildout is capable of building develop eggs from source directories. > I don't think you can build regular eggs from source directories. You > also want to run the setup with certain distributions in the path. > There isn't support for doing that presently, although folks would > find it useful. The infrae recipe allows one do an svn co in parts/arbitrary_folder. Down in the hierarchy of checked out folders, there is a setup.py file. What I wanted to do is create a Python sandbox where to deploy those yet to be developed eggs. What I succeeded in doing now is install a virtualenv sandbox and build the eggs from the svn co using the sandbox Python. When I found out about the Infrae recipe I figured there should be a way to build the eggs from the source. > It would be possible to write a recipe that did both > of these things. You would then define a part for the egg you wanted > to install, where that part would use the recipe. You'd want to list > that part before other parts that need that egg. > Jim > > I'll go hunt for a recipes tutorial :-) Thanx for your time. Yves Moisan From ziade.tarek at gmail.com Fri Sep 12 14:33:42 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 12 Sep 2008 14:33:42 +0200 Subject: [Distutils] Buildout to run setup.py In-Reply-To: <1221222498.6264.15.camel@dell-desktop.example.com> References: <1221166524.6323.9.camel@dell-desktop.example.com> <1221222498.6264.15.camel@dell-desktop.example.com> Message-ID: <94bdd2610809120533j78013660m74d18c422a845b3d@mail.gmail.com> On Fri, Sep 12, 2008 at 2:28 PM, Yves Moisan wrote: > > > > > I'm not entirely sure what you're trying to do. > > > > Buildout is capable of building develop eggs from source directories. > > I don't think you can build regular eggs from source directories. You > > also want to run the setup with certain distributions in the path. > > There isn't support for doing that presently, although folks would > > find it useful. > > The infrae recipe allows one do an svn co in parts/arbitrary_folder. > Down in the hierarchy of checked out folders, there is a setup.py file. > What I wanted to do is create a Python sandbox where to deploy those yet > to be developed eggs. What I succeeded in doing now is install a > virtualenv sandbox and build the eggs from the svn co using the sandbox > Python. When I found out about the Infrae recipe I figured there should > be a way to build the eggs from the source. Other options are: - to set up a svn bundle with those svn links and add some develop directives in the buildout that point to them or - to ask the package team to deal with a dev tag so you can use "==dev" in your buildout (like setutpools does for instance) > > > > It would be possible to write a recipe that did both > > of these things. You would then define a part for the egg you wanted > > to install, where that part would use the recipe. You'd want to list > > that part before other parts that need that egg. > > Jim > > > > > > I'll go hunt for a recipes tutorial :-) Thanx for your time. > > Yves Moisan > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdcasey at gmail.com Fri Sep 12 19:27:49 2008 From: cdcasey at gmail.com (chris) Date: Fri, 12 Sep 2008 12:27:49 -0500 Subject: [Distutils] bdist_egg doesn't call build? Message-ID: I created a custom build command, and was hoping it would be called when I ran bdist_egg. However, what appears to happen is that bdist_egg runs all of build's subcommands directly, and my custom build command never runs. Is this what's supposed to happen? If so, how do I make it call my custom build command short of subclassing bdist_egg and using self.run_command('build')? On a side note, would it be better for bdist_egg (and other commands that do builds) to call build directly, or are there other considerations? It seems like it would make extending easier... Thanks, -Chris From pje at telecommunity.com Fri Sep 12 20:21:22 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 12 Sep 2008 14:21:22 -0400 Subject: [Distutils] bdist_egg doesn't call build? In-Reply-To: References: Message-ID: <20080912182020.A46D23A4072@sparrow.telecommunity.com> At 12:27 PM 9/12/2008 -0500, chris wrote: >I created a custom build command, and was hoping it would be called >when I ran bdist_egg. However, what appears to happen is that >bdist_egg runs all of build's subcommands directly, and my custom >build command never runs. Is this what's supposed to happen? What happens is bdist_egg runs install_lib, install_scripts, and install_data. It's these commands that run any build subcommands that apply. bdist_egg doesn't actually invoke *any* build commands directly. > If so, >how do I make it call my custom build command short of subclassing >bdist_egg and using self.run_command('build')? Short of that? You can't. From rupert.thurner at gmail.com Sat Sep 13 19:39:31 2008 From: rupert.thurner at gmail.com (rupert.thurner) Date: Sat, 13 Sep 2008 10:39:31 -0700 (PDT) Subject: [Distutils] compiling an extension on solaris: genshi, mercurial, ... python-dev installed? In-Reply-To: <48c5fb00-795a-4d0f-89ca-f81260a809cb@l64g2000hse.googlegroups.com> References: <48c5fb00-795a-4d0f-89ca-f81260a809cb@l64g2000hse.googlegroups.com> Message-ID: <2e3bd6a8-903b-439c-aba4-e6e266f81b6e@34g2000hsh.googlegroups.com> On Sep 10, 8:04?pm, "rupert.thurner" wrote: > hi, > > we runhttp://www.blastwave.organd try to installhttp://genshi.edgewall.org > which bails out with (btw, the same happens with mercurial version > control): > > # ls -l /opt/csw/lib/libpython2.5.so > lrwxrwxrwx ? 1 root ? ? root ? ? ? ? ?19 Jul 18 23:14 /opt/csw/lib/ > libpython2.5.so -> libpython2.5.so.1.0 > > # easy_install --prefix /opt/csw/testing genshi > Searching for genshi > Best match: Genshi 0.6dev-r914 > Processing Genshi-0.6dev_r914-py2.5-solaris-2.10-sun4v.egg > Genshi 0.6dev-r914 is already the active version in easy-install.pth > > Using /opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914- > py2.5-solaris-2.10-sun4v.egg > Processing dependencies for genshi > Finished processing dependencies for genshi > root at chvp011vs035 / > # rm /opt/csw/testing/lib/python2.5/site-packages/Genshi-0.6dev_r914- > py2.5-solaris-2.10-sun4v.egg > root at chvp011vs035 / > # easy_install --prefix /opt/csw/testing genshi > Searching for genshi > Readinghttp://pypi.python.org/simple/genshi/ > Couldn't find index page for 'genshi' (maybe misspelled?) > Scanning index of all packages (this may take a while) > Readinghttp://pypi.python.org/simple/ > Readinghttp://pypi.python.org/simple/Genshi/ > Readinghttp://genshi.edgewall.org/ > Readinghttp://genshi.edgewall.org/wiki/Download > Best match: Genshi 0.5.1 > Downloadinghttp://ftp.edgewall.com/pub/genshi/Genshi-0.5.1.zip > Processing Genshi-0.5.1.zip > Running Genshi-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/ > easy_install-kz7muh/Genshi-0.5.1/egg-dist-tmp--y0Vju > warning: no previously-included files found matching 'doc/ > 2000ft.graffle' > warning: no previously-included files matching '*' found under > directory 'doc/logo.lineform' > ld: fatal: library -lpython2.5: not found > ld: fatal: File processing errors. No output written to build/ > lib.solaris-2.10-sun4v-2.5/genshi/_speedups.so > ********************************************************************** > WARNING: > An optional C extension could not be compiled, speedups will not be > available. > ********************************************************************** > error: Setup script exited with error: can't copy 'Genshi.egg-info/ > native_libs.txt': doesn't exist or not a regular file > > anybody has an idea how to convince setuptools to use the python > library? python works and it should be in the library path though. phillip eby suggested that "python-dev" is not installed and this should be the cause. blastwave does not really offer such a thing, so i tried to check the files, and took debian solaris as example: http://packages.debian.org/sid/sparc/python2.5-dev/filelist the header files are there, as well as the .so file /opt/csw/lib/ libpython2.5.so. but /opt/csw/lib/python2.5/config/Setup /opt/csw/lib/python2.5/config/Setup.config /opt/csw/lib/python2.5/config/Setup.local /opt/csw/lib/python2.5/config/config.c /opt/csw/lib/python2.5/config/config.c.in /opt/csw/lib/python2.5/config/install-sh /opt/csw/lib/python2.5/config/libpython2.5-pic.a /opt/csw/lib/python2.5/config/libpython2.5.a /opt/csw/lib/python2.5/config/libpython2.5.so /opt/csw/lib/python2.5/config/makesetup /opt/csw/lib/python2.5/config/python.o is missing. could it be that setuptools would miss this? From pje at telecommunity.com Sat Sep 13 20:23:38 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 13 Sep 2008 14:23:38 -0400 Subject: [Distutils] compiling an extension on solaris: genshi, mercurial, ... python-dev installed? In-Reply-To: <2e3bd6a8-903b-439c-aba4-e6e266f81b6e@34g2000hsh.googlegrou ps.com> References: <48c5fb00-795a-4d0f-89ca-f81260a809cb@l64g2000hse.googlegroups.com> <2e3bd6a8-903b-439c-aba4-e6e266f81b6e@34g2000hsh.googlegroups.com> Message-ID: <20080913182236.D05FD3A403D@sparrow.telecommunity.com> At 10:39 AM 9/13/2008 -0700, rupert.thurner wrote: >phillip eby suggested that "python-dev" is not installed and this >should be the cause. blastwave does not really offer such a thing, so >i tried to check the files, and took debian solaris as example: >http://packages.debian.org/sid/sparc/python2.5-dev/filelist > >the header files are there, as well as the .so file /opt/csw/lib/ >libpython2.5.so. but > >/opt/csw/lib/python2.5/config/Setup >/opt/csw/lib/python2.5/config/Setup.config >/opt/csw/lib/python2.5/config/Setup.local >/opt/csw/lib/python2.5/config/config.c >/opt/csw/lib/python2.5/config/config.c.in >/opt/csw/lib/python2.5/config/install-sh >/opt/csw/lib/python2.5/config/libpython2.5-pic.a >/opt/csw/lib/python2.5/config/libpython2.5.a >/opt/csw/lib/python2.5/config/libpython2.5.so >/opt/csw/lib/python2.5/config/makesetup >/opt/csw/lib/python2.5/config/python.o > >is missing. could it be that setuptools would miss this? It's not setuptools that cares; it's the distutils. Or more likely, the linker. IIUC, these files are not optional. This appears to suggest that there's a bug in the Python configuration on the platform: http://blogs.sun.com/dp/entry/a_screencasting_toolchain You might want to follow up with the person who reported(?) the bug back to Sun or whoever it is. This is not a setuptools problem nor a distutils problem: the missing files are clearly symptomatic of breakage in your platform's Python installation. You might want to consider building your own Python from source, as often these kinds of breakage are caused by platform distributors "cleaning things up" that shouldn't be moved or deleted. (For example, separating the headers and libraries into a separate "lib", "dev", or "devel" package.) If that's what happened here, then building a fresh Python from source (without the platform vendor's "improvements") would fix it. From setuptools at bugs.python.org Sun Sep 14 15:36:08 2008 From: setuptools at bugs.python.org (johannes raggam) Date: Sun, 14 Sep 2008 13:36:08 +0000 Subject: [Distutils] [issue44] distutils.log not imported in sdist.py In-Reply-To: <1221399368.45.0.414130458097.issue44@psf.upfronthosting.co.za> Message-ID: <1221399368.45.0.414130458097.issue44@psf.upfronthosting.co.za> New submission from johannes raggam : please add from distutils import log in setuptools.command.sdist.py setuptools version: setuptools-0.6c8-py2.4.egg i got an error with installing a python module which i checked out from svn. from traceback: """log.warn("unrecognized .svn/entries format in %s", dirname) NameError: global name 'log' is not defined """ ---------- messages: 169 nosy: thet priority: bug status: unread title: distutils.log not imported in sdist.py _______________________________________________ Setuptools tracker _______________________________________________ From setuptools at bugs.python.org Sun Sep 14 15:36:10 2008 From: setuptools at bugs.python.org (johannes raggam) Date: Sun, 14 Sep 2008 13:36:10 +0000 Subject: [Distutils] [issue45] distutils.log not imported in sdist.py In-Reply-To: <1221399370.2.0.291093107215.issue45@psf.upfronthosting.co.za> Message-ID: <1221399370.2.0.291093107215.issue45@psf.upfronthosting.co.za> New submission from johannes raggam : please add from distutils import log in setuptools.command.sdist.py setuptools version: setuptools-0.6c8-py2.4.egg i got an error with installing a python module which i checked out from svn. from traceback: """log.warn("unrecognized .svn/entries format in %s", dirname) NameError: global name 'log' is not defined """ ---------- messages: 170 nosy: thet priority: bug status: unread title: distutils.log not imported in sdist.py _______________________________________________ Setuptools tracker _______________________________________________ From cdcasey at gmail.com Tue Sep 16 00:01:40 2008 From: cdcasey at gmail.com (chris) Date: Mon, 15 Sep 2008 17:01:40 -0500 Subject: [Distutils] subclassing commands in extensions Message-ID: I've subclassed bdist_egg in my setup.py to add an option, "--include-docs", 'i'. It works as I expect it to when run from the command line while it's in the setup.py. Since I have a few projects that may use this, I thought I may try to put it in an extension. However, now when I run python setup.py bdist_egg -i, I get error: command 'bdist_egg' has no such option 'include_docs' Is there something extra that needs to be done when putting a subclassed command in an extension instead of just a setup.py? Thanks, -Chris From pjenvey at underboss.org Tue Sep 16 02:12:18 2008 From: pjenvey at underboss.org (Philip Jenvey) Date: Mon, 15 Sep 2008 17:12:18 -0700 Subject: [Distutils] subclassing commands in extensions In-Reply-To: References: Message-ID: <1233298C-8E37-4D85-9340-ADF8FA941DEC@underboss.org> On Sep 15, 2008, at 3:01 PM, chris wrote: > I've subclassed bdist_egg in my setup.py to add an option, > "--include-docs", 'i'. It works as I expect it to when run from the > command line while it's in the setup.py. > > Since I have a few projects that may use this, I thought I may try to > put it in an extension. However, now when I run python setup.py > bdist_egg -i, I get > > error: command 'bdist_egg' has no such option 'include_docs' > > Is there something extra that needs to be done when putting a > subclassed command in an extension instead of just a setup.py? You need to specify a setup_requires='myplugin' kwarg to your setup(). It's described here: http://peak.telecommunity.com/DevCenter/setuptools#new-and-changed-setup-keywords -- Philip Jenvey From setuptools at bugs.python.org Wed Sep 17 02:36:32 2008 From: setuptools at bugs.python.org (Greg Hazel) Date: Wed, 17 Sep 2008 00:36:32 +0000 Subject: [Distutils] [issue46] always-unzip does not always unzip In-Reply-To: <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co.za> Message-ID: <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co.za> New submission from Greg Hazel : Lots of unexpected behaviour (installed zipped, then claimed a file was present which was not present, then failed to replace the module unzipped): >easy_install pytz Searching for pytz Reading http://pypi.python.org/simple/pytz/ Reading http://pytz.sourceforge.net Reading http://sourceforge.net/project/showfiles.php?group_id=79122 Reading http://www.stuartbishop.net/Software/pytz Reading http://sourceforge.net/projects/pytz/ Best match: pytz 2008c Downloading http://pypi.python.org/packages/2.5/p/pytz/pytz-2008c- py2.5.egg#md5=8fb5f0dcab343ceb497c40138fa8d6c7 Processing pytz-2008c-py2.5.egg Moving pytz-2008c-py2.5.egg to c:\python25\lib\site-packages Adding pytz 2008c to easy-install.pth file Installed c:\python25\lib\site-packages\pytz-2008c-py2.5.egg Processing dependencies for pytz Finished processing dependencies for pytz >python Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pytz >>> print pytz.__file__ c:\python25\lib\site-packages\pytz-2008c-py2.5.egg\pytz\__init__.py >>> ^Z >cat c:\python25\lib\site-packages\pytz-2008c-py2.5.egg\pytz\__init__.py cat: c:\python25\lib\site-packages\pytz-2008c-py2.5.egg\pytz\__init__.py: No such file or directory >easy_install --always-unzip pytz Searching for pytz Best match: pytz 2008c Processing pytz-2008c-py2.5.egg pytz 2008c is already the active version in easy-install.pth Using c:\python25\lib\site-packages\pytz-2008c-py2.5.egg Processing dependencies for pytz Finished processing dependencies for pytz >cat c:\python25\lib\site-packages\pytz-2008c-py2.5.egg\pytz\__init__.py cat: c:\python25\lib\site-packages\pytz-2008c-py2.5.egg\pytz\__init__.py: No such file or directory ---------- messages: 171 nosy: ghazel priority: bug status: unread title: always-unzip does not always unzip _______________________________________________ Setuptools tracker _______________________________________________ From pje at telecommunity.com Wed Sep 17 02:44:26 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 16 Sep 2008 20:44:26 -0400 Subject: [Distutils] [issue46] always-unzip does not always unzip In-Reply-To: <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co .za> References: <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co.za> <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co.za> Message-ID: <20080917005312.DF8393A4105@sparrow.telecommunity.com> At 12:36 AM 9/17/2008 +0000, Greg Hazel wrote: >New submission from Greg Hazel : > >Lots of unexpected behaviour (installed zipped, then claimed a file >was present >which was not present, A module's __file__ can include paths that are within a zipfile. >then failed to replace the module unzipped): >... > >easy_install --always-unzip pytz >Searching for pytz >Best match: pytz 2008c >Processing pytz-2008c-py2.5.egg >pytz 2008c is already the active version in easy-install.pth --always-unzip did nothing here because the egg was already installed. To reinstall an already installed package, use a file, directory, or URL as a target instead of a requirement name, and/or use the -U flag to request an upgrade. From cdcasey at gmail.com Wed Sep 17 03:23:08 2008 From: cdcasey at gmail.com (chris) Date: Tue, 16 Sep 2008 20:23:08 -0500 Subject: [Distutils] subclassing commands in extensions In-Reply-To: References: Message-ID: having DISTUTILS_DEBUG=1 results in the following: $> python setup.py bdist_egg -i Distribution.parse_config_files(): reading /usr/lib/python2.5/distutils/distutils.cfg reading setup.cfg options (after parsing config files): option dict for 'aliases' command: {'release': ('setup.cfg', "egg_info -RDb ''")} option dict for 'build_py' command: {'optimize': ('/usr/lib/python2.5/distutils/distutils.cfg', '0')} option dict for 'egg_info' command: {'tag_build': ('setup.cfg', '.dev'), 'tag_svn_revision': ('setup.cfg', '1')} option dict for 'install' command: {'optimize': ('/usr/lib/python2.5/distutils/distutils.cfg', '0'), 'prefix': ('/usr/lib/python2.5/distutils/distutils.cfg', '/usr/local')} option dict for 'nosetests' command: {'detailed_errors': ('setup.cfg', '1'), 'tests': ('setup.cfg', 'enthought/chaco/tests,enthought/chaco/shell/tests,enthought/chaco2/tests,enthought/chaco2/shell/tests'), 'verbosity': ('setup.cfg', '0'), 'with_coverage': ('setup.cfg', '1'), 'with_doctest': ('setup.cfg', '1')} options (after parsing command line): option dict for 'aliases' command: {'release': ('setup.cfg', "egg_info -RDb ''")} option dict for 'bdist_egg' command: {'include_docs': ('command line', 1)} option dict for 'build_py' command: {'optimize': ('/usr/lib/python2.5/distutils/distutils.cfg', '0')} option dict for 'egg_info' command: {'tag_build': ('setup.cfg', '.dev'), 'tag_svn_revision': ('setup.cfg', '1')} option dict for 'install' command: {'optimize': ('/usr/lib/python2.5/distutils/distutils.cfg', '0'), 'prefix': ('/usr/lib/python2.5/distutils/distutils.cfg', '/usr/local')} option dict for 'nosetests' command: {'detailed_errors': ('setup.cfg', '1'), 'tests': ('setup.cfg', 'enthought/chaco/tests,enthought/chaco/shell/tests,enthought/chaco2/tests,enthought/chaco2/shell/tests'), 'verbosity': ('setup.cfg', '0'), 'with_coverage': ('setup.cfg', '1'), 'with_doctest': ('setup.cfg', '1')} running bdist_egg Distribution.get_command_obj(): creating 'bdist_egg' command object setting options for 'bdist_egg' command: include_docs = 1 (from command line) Traceback (most recent call last): File "setup.py", line 227, in zip_safe = False, File "/usr/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/usr/lib/python2.5/distutils/dist.py", line 992, in run_command cmd_obj = self.get_command_obj(command) File "/usr/lib/python2.5/distutils/dist.py", line 879, in get_command_obj self._set_command_options(cmd_obj, options) File "/usr/lib/python2.5/distutils/dist.py", line 919, in _set_command_options % (source, command_name, option)) distutils.errors.DistutilsOptionError: error in command line: command 'bdist_egg' has no such option 'include_docs' On Mon, Sep 15, 2008 at 5:01 PM, chris wrote: > I've subclassed bdist_egg in my setup.py to add an option, > "--include-docs", 'i'. It works as I expect it to when run from the > command line while it's in the setup.py. > > Since I have a few projects that may use this, I thought I may try to > put it in an extension. However, now when I run python setup.py > bdist_egg -i, I get > > error: command 'bdist_egg' has no such option 'include_docs' > > Is there something extra that needs to be done when putting a > subclassed command in an extension instead of just a setup.py? > > Thanks, > -Chris > From ghazel at gmail.com Wed Sep 17 08:15:37 2008 From: ghazel at gmail.com (ghazel at gmail.com) Date: Wed, 17 Sep 2008 01:15:37 -0500 Subject: [Distutils] [issue46] always-unzip does not always unzip In-Reply-To: <20080917005312.DF8393A4105@sparrow.telecommunity.com> References: <1221611792.19.0.735668766002.issue46@psf.upfronthosting.co.za> <20080917005312.DF8393A4105@sparrow.telecommunity.com> Message-ID: <151643b20809162315w3ad06b66jd97885827be28b8d@mail.gmail.com> Maybe always-unzip is a good default? It really seems like the disk space saved due to zipping is not worth the trouble. I could change the ticket status from "bug" to "enhancement". Using -U flag did not cause it to install unzipped - it gave the same "pytz 2008c is already the active version in easy-install.pth". Which file, directory, or URL should I provide to install pytz? -Greg On Tue, Sep 16, 2008 at 7:44 PM, Phillip J. Eby wrote: > At 12:36 AM 9/17/2008 +0000, Greg Hazel wrote: >> >> New submission from Greg Hazel : >> >> Lots of unexpected behaviour (installed zipped, then claimed a file was >> present >> which was not present, > > A module's __file__ can include paths that are within a zipfile. > >> then failed to replace the module unzipped): >> ... >> >easy_install --always-unzip pytz >> Searching for pytz >> Best match: pytz 2008c >> Processing pytz-2008c-py2.5.egg >> pytz 2008c is already the active version in easy-install.pth > > --always-unzip did nothing here because the egg was already installed. To > reinstall an already installed package, use a file, directory, or URL as a > target instead of a requirement name, and/or use the -U flag to request an > upgrade. > > From setuptools at bugs.python.org Wed Sep 17 23:26:02 2008 From: setuptools at bugs.python.org (Philip Jenvey) Date: Wed, 17 Sep 2008 21:26:02 +0000 Subject: [Distutils] [issue47] [PATCH] avoid the deprecated md5 module in 2.6 In-Reply-To: <1221686762.66.0.949066261355.issue47@psf.upfronthosting.co.za> Message-ID: <1221686762.66.0.949066261355.issue47@psf.upfronthosting.co.za> New submission from Philip Jenvey : Patch to avoid DeprecationWarnings from ez_setup in 2.6 ---------- assignedto: pje files: hashlib_md5-r66389.diff messages: 176 nosy: pje, pjenvey priority: bug status: unread title: [PATCH] avoid the deprecated md5 module in 2.6 Added file: http://bugs.python.org/setuptools/file23/hashlib_md5-r66389.diff _______________________________________________ Setuptools tracker _______________________________________________ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: hashlib_md5-r66389.diff URL: From ianb at colorstudy.com Thu Sep 18 19:27:13 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 18 Sep 2008 12:27:13 -0500 Subject: [Distutils] virtualenv + python 2.6 (setuptools 2.6 egg) In-Reply-To: <48D27937.8040303@clapper.org> References: <48D27937.8040303@clapper.org> Message-ID: <48D28F71.7050603@colorstudy.com> Brian Clapper wrote: > Ian, > > Been playing with Python 2.6rc2. Have you tried virtualenv with it? Doesn't > seem to work for me, since it attempts to download a 2.6 version of the > setuptools egg, which doesn't exist. I glanced through the virtualenv source > (in between extreme busy-ness at work), and it looked to me as though > virtualenv would use a setuptools egg in /path/to/virtualenv.egg/support-files > if one existed, but that doesn't seem to happen. The EZ_SETUP_PY embedded > script seems to be the culprit, but I haven't dug into it any deeper than that > due to work demands. > > It's quite possible I'm just missing something stupid. If so, feel free to > send me on my way. Yes, the missing egg is causing the problem. ez_setup.py doesn't seem to use a tarball fallback (I'm not sure why). Phillip - can there either be a 2.6 egg uploaded, or have ez_setup.py install the tarball when the egg is missing? -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Fri Sep 19 14:30:15 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 19 Sep 2008 14:30:15 +0200 Subject: [Distutils] Setuptools 0.7 release ? Message-ID: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> Hi, We v'e been trying the dev version here and it works great, could it be released soon, Phillip ? The SVN 1.5 compatibility problem drives some developers nuts around me :) I am volunteering for help on this package, if you think that is wise, I can do releases, tests patches, add examples to ticket that don't have it, etc.. Best regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Fri Sep 19 15:38:03 2008 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 19 Sep 2008 09:38:03 -0400 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> Message-ID: <48D3AB3B.5040503@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > We v'e been trying the dev version here and it works great, could it be > released soon, Phillip ? > > The SVN 1.5 compatibility problem drives some developers nuts around me :) > > I am volunteering for help on this package, if you think that is wise, I can > do releases, tests patches, > add examples to ticket that don't have it, etc.. Aren't you really asking for a final 0.6 release? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI06s7+gerLs4ltQ4RAh9NAKCTJsRWhorggsUMwRehXRHhMiY6fACg1M9H YRrGF5d+awphVTh8wqvkCw8= =REam -----END PGP SIGNATURE----- From tarek.ziade at ingeniweb.com Fri Sep 19 15:45:50 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Fri, 19 Sep 2008 15:45:50 +0200 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <48D3AB3B.5040503@palladion.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <48D3AB3B.5040503@palladion.com> Message-ID: 2008/9/19 Tres Seaver > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Tarek Ziad? wrote: > > > We v'e been trying the dev version here and it works great, could it be > > released soon, Phillip ? > > > > The SVN 1.5 compatibility problem drives some developers nuts around me > :) > > > > I am volunteering for help on this package, if you think that is wise, I > can > > do releases, tests patches, > > add examples to ticket that don't have it, etc.. > > Aren't you really asking for a final 0.6 release? I don't know The release of the latest dev version. The trunk is 0.7a1 so I supposed the next release is 0.7 ? what "c" means in 0.6c8 ? candidate ? in that case the trunk should be 0.6c9 or 0.6 ? Can't tell Tarek > > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFI06s7+gerLs4ltQ4RAh9NAKCTJsRWhorggsUMwRehXRHhMiY6fACg1M9H > YRrGF5d+awphVTh8wqvkCw8= > =REam > -----END PGP SIGNATURE----- > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Fri Sep 19 15:46:10 2008 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 19 Sep 2008 09:46:10 -0400 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <94bdd2610809190641x50b535e3m27a8822e6cf92d67@mail.gmail.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <48D3AB3B.5040503@palladion.com> <94bdd2610809190641x50b535e3m27a8822e6cf92d67@mail.gmail.com> Message-ID: <48D3AD22.50702@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > On Fri, Sep 19, 2008 at 3:38 PM, Tres Seaver wrote: >> Aren't you really asking for a final 0.6 release? > > I am a bit lost in the versions in setuptools, > > the current trunk is flagged 0.7a1 so I don't really know what will be the > revision of > the next public release.. I didn't realize you were using the trunk: I thought you were using the head of the 0.6 branch, which also has the fix for the SVN 1.5 issue. I think we should just release 0.6 and be done with it, and then go on to working on the next release (which should probably be 1.0, or something). We can always release a 0.6.1 for any problems which show up later, but getting people unstuck on the SVN bug should happen now. IMNSHO, of course. :) Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI060i+gerLs4ltQ4RAmtnAJ0VEMatwy/fvRxR2uK1NIw04Znp4gCgmtsq s6D3/xHnjqL38MT9YoZmDgI= =tVBV -----END PGP SIGNATURE----- From tarek.ziade at ingeniweb.com Fri Sep 19 15:53:19 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Fri, 19 Sep 2008 15:53:19 +0200 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <48D3AD22.50702@palladion.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <48D3AB3B.5040503@palladion.com> <94bdd2610809190641x50b535e3m27a8822e6cf92d67@mail.gmail.com> <48D3AD22.50702@palladion.com> Message-ID: 2008/9/19 Tres Seaver > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Tarek Ziad? wrote: > > On Fri, Sep 19, 2008 at 3:38 PM, Tres Seaver > wrote: > > >> Aren't you really asking for a final 0.6 release? > > > > I am a bit lost in the versions in setuptools, > > > > the current trunk is flagged 0.7a1 so I don't really know what will be > the > > revision of > > the next public release.. > > I didn't realize you were using the trunk: I thought you were using the > head of the 0.6 branch, which also has the fix for the SVN 1.5 issue. > > I think we should just release 0.6 and be done with it, and then go on > to working on the next release (which should probably be 1.0, or > something). We can always release a 0.6.1 for any problems which show > up later, but getting people unstuck on the SVN bug should happen now. > > IMNSHO, of course. :) > Ah right... so yes, a 0.6 version would be nice you are right +1 on making the current trunk 1.x Tarek > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFI060i+gerLs4ltQ4RAmtnAJ0VEMatwy/fvRxR2uK1NIw04Znp4gCgmtsq > s6D3/xHnjqL38MT9YoZmDgI= > =tVBV > -----END PGP SIGNATURE----- > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Fri Sep 19 16:46:52 2008 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 19 Sep 2008 10:46:52 -0400 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <48D3AB3B.5040503@palladion.com> <94bdd2610809190641x50b535e3m27a8822e6cf92d67@mail.gmail.com> <48D3AD22.50702@palladion.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziade wrote: > 2008/9/19 Tres Seaver > > I didn't realize you were using the trunk: I thought you were using the > head of the 0.6 branch, which also has the fix for the SVN 1.5 issue. > > I think we should just release 0.6 and be done with it, and then go on > to working on the next release (which should probably be 1.0, or > something). We can always release a 0.6.1 for any problems which show > up later, but getting people unstuck on the SVN bug should happen now. > > IMNSHO, of course. :) > > >> Ah right... so yes, a 0.6 version would be nice you are right It's been almost a year since Jim's question on the development process: http://thread.gmane.org/gmane.comp.python.distutils.devel/5434 I think we are well past time to declare 0.6 done (maybe even make *it* 1.0, given how widespread the use is), and then work on new features for another release. >> +1 on making the current trunk 1.x That would be OK with me, too, as long as we release what we have now. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI07tc+gerLs4ltQ4RAj+5AJ4geqxtS5QZ98CMBDagzBhXp3yEdwCfXEPr 8aPUurqZwYDjSSNWiDzHyJA= =h157 -----END PGP SIGNATURE----- From icon at fedoraproject.org Fri Sep 19 17:27:19 2008 From: icon at fedoraproject.org (Konstantin Ryabitsev) Date: Fri, 19 Sep 2008 11:27:19 -0400 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <48D3AB3B.5040503@palladion.com> <94bdd2610809190641x50b535e3m27a8822e6cf92d67@mail.gmail.com> <48D3AD22.50702@palladion.com> Message-ID: On Fri, Sep 19, 2008 at 10:46 AM, Tres Seaver wrote: >>> Ah right... so yes, a 0.6 version would be nice you are right > > It's been almost a year since Jim's question on the development process: > > http://thread.gmane.org/gmane.comp.python.distutils.devel/5434 > > I think we are well past time to declare 0.6 done (maybe even make *it* > 1.0, given how widespread the use is), and then work on new features for > another release. In fact, if you go 0.6c8 to 1.0 instead of going to 0.6, I know I'll be happy as the package maintainer in Fedora/EPEL. As far as RPM is concerned, 0.6 is less than 0.6c8, so I'd have to do creative/gross things with epoch inflation just to make the package update smoothly to 0.6 final. Just a note from the sidelines. :) Cheers, -- Konstantin Ryabitsev Montr?al, Qu?bec From zooko at zooko.com Fri Sep 19 21:54:08 2008 From: zooko at zooko.com (zooko) Date: Fri, 19 Sep 2008 13:54:08 -0600 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> Message-ID: <7F3B9883-4C8B-43D6-AE4B-A6A471847A4F@zooko.com> On Sep 19, 2008, at 6:30 AM, Tarek Ziad? wrote: > I am volunteering for help on this package, if you think that is > wise, I can do releases, tests patches, > add examples to ticket that don't have it, etc.. Thank you for volunteering! PJE mentioned that two tickets that I opened, that he fixed, and that he asked me to test the fixes, are blockers. I believe that's: http://bugs.python.org/setuptools/issue23 # SandboxViolation: mkdir('/ Users/zooko/.python-eggs', 511) {} and http://bugs.python.org/setuptools/issue16 # permission error in untarring pyOpenSSL-0.6.tar.gz I don't know how to reproduce the first one. To reproduce the second one, make yourself a source tarball that has g+s bits on directories, per my msg23. Note that the current pyOpenSSL-0.6 tarball has been fixed to not have g+s bits, so you can't just use that. Thank you! Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From pjenvey at underboss.org Fri Sep 19 22:25:46 2008 From: pjenvey at underboss.org (Philip Jenvey) Date: Fri, 19 Sep 2008 13:25:46 -0700 Subject: [Distutils] Setuptools 0.7 release ? In-Reply-To: <7F3B9883-4C8B-43D6-AE4B-A6A471847A4F@zooko.com> References: <94bdd2610809190530k1594e648w83d31e972cda2043@mail.gmail.com> <7F3B9883-4C8B-43D6-AE4B-A6A471847A4F@zooko.com> Message-ID: On Sep 19, 2008, at 12:54 PM, zooko wrote: > > PJE mentioned that two tickets that I opened, that he fixed, and > that he asked me to test the fixes, are blockers. I believe that's: > > http://bugs.python.org/setuptools/issue23 # SandboxViolation: > mkdir('/Users/zooko/.python-eggs', 511) {} > > and > > http://bugs.python.org/setuptools/issue16 # permission error in > untarring pyOpenSSL-0.6.tar.gz > > I don't know how to reproduce the first one. To reproduce the > second one, make yourself a source tarball that has g+s bits on > directories, per my msg23. Note that the current pyOpenSSL-0.6 > tarball has been fixed to not have g+s bits, so you can't just use > that. http://bugs.python.org/setuptools/issue27 is also pending as Jython really needs it going forward. PJE acknowledged this would make the next release as long as there's tests (which there are). Since he hasn't applied it yet, other reviewers and testers are welcome. I also recommend the incredibly simple http://bugs.python.org/setuptools/issue47 make the next release in preparation for the soon to be released 2.6. -- Philip Jenvey From chris at simplistix.co.uk Tue Sep 23 12:37:53 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 23 Sep 2008 11:37:53 +0100 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <20080917171851.GA27261@logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> Message-ID: <48D8C701.6070707@simplistix.co.uk> (I'll be CC'ing the distutils sig in to these replies as this discussion probably belongs there...) Nicolas Chauvat wrote: >> The slides for my two talks can be found here: >> >> http://www.simplistix.co.uk/presentations > > "Python Package Management Sucks" > > Install debian and get back to productive tasks. This is an almost troll-like answer. See page 35 of the presentation. If you're too lazy, here's a re-hash: - there are lots of different operating systems - even with Linux, there are many different package management systems - all the package management systems behave differently and expect packages to be set up differently for them - expecting package developers to shoulder this burden is unfair and results in various bitrotting repackaged versions of python packages rather than just one up-to-date version maintained by the entity originating the python package - Adobe Photoshop Plugins, Firefox Add-ons, etc do not delegate their management to an OS package manager. Packages are Python's "plugins" and so should get the same type of consistent, cross-platform package management targetted at the application in question, which is Python in this case. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 23 12:40:42 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 23 Sep 2008 11:40:42 +0100 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> Message-ID: <48D8C7AA.201@simplistix.co.uk> John Pinner wrote: > To me, setup tools is missing two essential features of a package > management system: > > 1. Dependency management I'm pretty sure setuptools does this. It could probably do a better job of checking version and dependency clashes, but provided there are none of these, it actually does an OK job. > 2. A means of removing packages. Indeed, but this is because setuptools is layered on top of distutils, and distutils is the root of all evil. >> Well that currently does not work as only a small minority of python >> packages are available in the Debian package archives. What would nice >> would be that the setup tools people in different languages and the >> distributions would work together. E.g.: It's a nice idea but the complexity of making this happen (especially all the silly political ramblings that would be involved) mean it's never going to happen... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 23 12:42:49 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 23 Sep 2008 11:42:49 +0100 Subject: [Distutils] svn for package management In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> Message-ID: <48D8C829.9040806@simplistix.co.uk> Houston, Martin (London) wrote: > Has anyone thought of using subversion to controlled the installed python tree? (as a separate undertaking to any under development sources). It is more cross platform than rpm or dpkg I used to do this with project directories (including Zope instance homes) but the fact that distutils (and hence setuptools, easy_install and buildout) like to destructively re-arrange packages on install it's something I'm moving away from. Buildout is currently the "least insane" way of managing python packages... cheers, -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 23 12:47:03 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 23 Sep 2008 11:47:03 +0100 Subject: [Distutils] "just use debian" In-Reply-To: <20080919174433.GS8000@logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> Message-ID: <48D8C927.4060609@simplistix.co.uk> Nicolas Chauvat wrote: >> Also some of the Debian Python packages are broken or grossly >> out-of-date. > > File a bug report :) Yes, because that automatically frees up the packager's time to work on these issues... > My problem with setup tools is that they come from windows I don't think that's the case at all. > is (almost) no package management system. The consequence is that > their author reinvented the wheel, but limited it to Python, then > moved to eggs and made things worse. Yes, the egg format is annoying. No, I don't think having a package management system that targets only python packages is a bad idea. > My main tool is Python, but I have many other tools on my system. I > do not want to have as many package management utilities as > "subsystems". Then I suggest you volunteer to maintain the debian packages for every single python package. > If I have one tool for Python, one for Java, one for C, > one for Fortran, one for C libraries, one for Gnome, etc. integration > becomes a nightmare. If you have projects this large, then you likely want to roll your own OS packages anyway. > [Please note that for an experienced Debian developer, making the > initial package of a Python module can be a matter of half an hour to > a couple hours and releasing a new version a matter of minutes.] ...and for someone not using Debian or not an experienced Debian developer? Despite being a fan of Debian, I'm well aware of just how "friendly" a community it can be to the new user... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From stephen.pascoe at lirico.co.uk Tue Sep 23 12:59:18 2008 From: stephen.pascoe at lirico.co.uk (Stephen Pascoe) Date: Tue, 23 Sep 2008 11:59:18 +0100 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48D8C927.4060609@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> Message-ID: <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> I'm interested in hearing what you find so annoying about the egg format because for me it's the one part of the setuptools system that I would keep. If I had my way we'd separate out eggs from distutils/setuptools and all the automagical package installation of easy_install and focus on making eggs work as plugins. Eggs are a best effort attempt to make a jar-like system for Python and in many cases they work well (c.f. Trac plugins). Sure, there are some workarounds needed for C-extensions and packages that access __file__, etc. but these problems seem more surmountable than the deeply ingrained problems with distutils. Cheers, Stephen. On Tue, Sep 23, 2008 at 11:47 AM, Chris Withers wrote: > Nicolas Chauvat wrote: > >> Also some of the Debian Python packages are broken or grossly >>> out-of-date. >>> >> >> File a bug report :) >> > > Yes, because that automatically frees up the packager's time to work on > these issues... > > My problem with setup tools is that they come from windows >> > > I don't think that's the case at all. > > is (almost) no package management system. The consequence is that >> their author reinvented the wheel, but limited it to Python, then >> moved to eggs and made things worse. >> > > Yes, the egg format is annoying. > No, I don't think having a package management system that targets only > python packages is a bad idea. > > My main tool is Python, but I have many other tools on my system. I >> do not want to have as many package management utilities as >> "subsystems". >> > > Then I suggest you volunteer to maintain the debian packages for every > single python package. > > If I have one tool for Python, one for Java, one for C, >> one for Fortran, one for C libraries, one for Gnome, etc. integration >> becomes a nightmare. >> > > If you have projects this large, then you likely want to roll your own OS > packages anyway. > > [Please note that for an experienced Debian developer, making the >> initial package of a Python module can be a matter of half an hour to >> a couple hours and releasing a new version a matter of minutes.] >> > > ...and for someone not using Debian or not an experienced Debian developer? > Despite being a fan of Debian, I'm well aware of just how "friendly" a > community it can be to the new user... > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > _______________________________________________ > pyconuk mailing list > pyconuk at python.org > http://mail.python.org/mailman/listinfo/pyconuk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Tue Sep 23 13:38:00 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 23 Sep 2008 13:38:00 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D8C701.6070707@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> Message-ID: <94bdd2610809230438m70869babh6e8a3791d5b661fb@mail.gmail.com> On Tue, Sep 23, 2008 at 12:37 PM, Chris Withers wrote: > (I'll be CC'ing the distutils sig in to these replies as this discussion > probably belongs there...) > > Nicolas Chauvat wrote: > >> The slides for my two talks can be found here: >>> >>> http://www.simplistix.co.uk/presentations >>> >> >> "Python Package Management Sucks" >> >> Install debian and get back to productive tasks. > > haha. :) I just wanted to react on that: I have created a setuptools-enabled package for Pylint (logilab.pylintinstaller, see http://pypi.python.org/pypi/logilab.pylintinstaller) with the bless of Logilab, so people under Windows could use the tool with a one-liner installation process... So if you like Pylint and Python but you don't use Debian you can use that. > >> > This is an almost troll-like answer. > See page 35 of the presentation. > > If you're too lazy, here's a re-hash: > > - there are lots of different operating systems > > - even with Linux, there are many different package management systems > > - all the package management systems behave differently and expect packages > to be set up differently for them > > - expecting package developers to shoulder this burden is unfair and > results in various bitrotting repackaged versions of python packages rather > than just one up-to-date version maintained by the entity originating the > python package > > - Adobe Photoshop Plugins, Firefox Add-ons, etc do not delegate their > management to an OS package manager. Packages are Python's "plugins" and so > should get the same type of consistent, cross-platform package management > targetted at the application in question, which is Python in this case. > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From russel at russel.org.uk Tue Sep 23 13:53:44 2008 From: russel at russel.org.uk (Russel Winder) Date: Tue, 23 Sep 2008 12:53:44 +0100 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> Message-ID: <1222170824.8878.442.camel@balin> On Tue, 2008-09-23 at 11:59 +0100, Stephen Pascoe wrote: > I'm interested in hearing what you find so annoying about the egg > format because for me it's the one part of the setuptools system that > I would keep. > > If I had my way we'd separate out eggs from distutils/setuptools and > all the automagical package installation of easy_install and focus on > making eggs work as plugins. Eggs are a best effort attempt to make a > jar-like system for Python and in many cases they work well (c.f. Trac > plugins). Sure, there are some workarounds needed for C-extensions > and packages that access __file__, etc. but these problems seem more > surmountable than the deeply ingrained problems with distutils. Remember though that jars carry no dependency information, hence Maven, Ivy, OSGi, JSR277, etc. Avoiding the Python equivalent of Jar Hell surely has to be a prime directive. I guess the question in my mind is if the Ruby community have Ruby Gems, what is the Python equivalent, and why doesn't it work? -- Russel. ==================================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: http://www.russel.org.uk/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From zooko at zooko.com Tue Sep 23 15:41:21 2008 From: zooko at zooko.com (zooko) Date: Tue, 23 Sep 2008 07:41:21 -0600 Subject: [Distutils] hooray for distutils/eggs/setuptools/easy_install In-Reply-To: <1222170824.8878.442.camel@balin> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> Message-ID: <1FABC3FB-1457-4AAB-915B-4B05BF49FB9D@zooko.com> On Sep 23, 2008, at 5:53 AM, Russel Winder wrote: > I guess the question in my mind is if the Ruby community have Ruby > Gems, > what is the Python equivalent, and why doesn't it work? I'm fairly satisfied with distutils/eggs/setuptools/easy_install. It isn't perfect, but it's good enough, and it is improving thanks to Philip Eby, Chris Galvan, Philip Jenvey, Tarek Ziad?, and other contributors. Also, even though there is a sizable fraction of Python programmers who don't like it, there is no other tool that has anywhere near the vast universe of compatible Python code. Partially because it is compatible with distutils and partly because it is widely accepted itself. There are currently 4,800 packages listed on http://pypi.python.org , in addition to which there are an uncounted number of publicly available Python packages that are not listed there. (By the way, there are 783 packages in the Debian unstable Python section -- http://packages.debian.org/unstable/python -- and 712 packages in the Ubuntu Hardy Python section -- http://packages.ubuntu.com/hardy/ python ). There are probably at least 4,800 different programmers responsible for writing and maintaining the world's publicly re-usable Python packages. Almost all of these thousands and thousands of packages are seamlessly re-usable by setuptools, and if you use distutils or setuptools to package and distribute your Python code, then your code will be re-usable by those folks (whether they use distutils or setuptools themselves). I get great value from being able to re-use almost any other Python package in my code without having to ask my users to manually deal with more dependencies, and without having to spend time to create platform-specific packages, e.g. Debian packages, Windows installers, etc. But note also that setuptools does not *prevent* me from creating such packages. Currently the standard Tahoe install instructions are generic and apply to all supported platforms, but Tahoe also gets packaged up as .debs and as a Windows app for specific customers. See also the stdeb tool which is a handy way to produce .debs automatically from your Python source code and bbfreeze which, I am told, is a good way to produce Windows packages: http://stdeb.python-hosting.com http://pypi.python.org/pypi/bbfreeze Also, I really like setuptools plugins as a way to make build tools be separately maintained and packaged instead of piling up in your setup.py. Here are the first few packages which use the new classifier "Framework :: Setuptools Plugin": http://pypi.python.org/pypi?:action=browse&c=524 Hopefully in the future more of these packages will get classified as being setuptools plugins that are useful for development: http://pypi.python.org/pypi?% 3Aaction=search&term=setuptools&submit=search My major project, Tahoe, has been using setuptools for more than a year now. Here are the installation instructions for Tahoe. Note that these instructions are the same for all supported platforms, which includes Windows, Cygwin, Mac, Linux, and Solaris. http://allmydata.org/source/tahoe/trunk/docs/install.html Here is the list of Python packages that Tahoe needs (not including the packages that *those* packages need, such as pyOpenSSL and pyutil): http://allmydata.org/trac/tahoe/browser/_auto_deps.py And here is the list of open tickets about how we would like to improve Tahoe packaging: http://allmydata.org/trac/tahoe/report/10 Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From mwhapples at aim.com Tue Sep 23 16:32:22 2008 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 23 Sep 2008 15:32:22 +0100 Subject: [Distutils] Setup tools causes problems with orca finding its python packages Message-ID: <1222180342.11820.47.camel@localhost> I have noticed that easy_install has a easy-install.pth file which seems to be adding the directories specified in it before any other python package directories, including the environment variable PYTHONPATH. This causes a problem with orca which uses a bash script to start itself, and specifies where its python packages should be found by setting the PYTHONPATH variable. This wouldn't cause problems except when a user has more than one version of orca installed (eg. a stable version from the distribution installed in the standard python site-packages directory and one (SVN trunk version) in a custom directory). I found that easy_install had added the standard python site-packages directory (/usr/lib/python2.5/site-packages on my system) to easy-install.pth so making it impossible to use my custom install of orca as the standard location was first in sys.path. Is what easy_install doing really a standard thing? If not then this should be corrected to get things to behave as they should. If it is standard then either can easy_install prevent the standard python site-packages module from being added to easy-install.pth or can you suggest a way that orca can overcome this problem? Michael Whapples From pje at telecommunity.com Tue Sep 23 16:50:10 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 23 Sep 2008 10:50:10 -0400 Subject: [Distutils] Setup tools causes problems with orca finding its python packages In-Reply-To: <1222180342.11820.47.camel@localhost> References: <1222180342.11820.47.camel@localhost> Message-ID: <20080923144904.2CF343A40C2@sparrow.telecommunity.com> At 03:32 PM 9/23/2008 +0100, Michael Whapples wrote: >I have noticed that easy_install has a easy-install.pth file which seems >to be adding the directories specified in it before any other python >package directories, including the environment variable PYTHONPATH. This >causes a problem with orca which uses a bash script to start itself, and >specifies where its python packages should be found by setting the >PYTHONPATH variable. This wouldn't cause problems except when a user has >more than one version of orca installed (eg. a stable version from the >distribution installed in the standard python site-packages directory >and one (SVN trunk version) in a custom directory). I found that >easy_install had added the standard python site-packages directory >(/usr/lib/python2.5/site-packages on my system) to easy-install.pth so >making it impossible to use my custom install of orca as the standard >location was first in sys.path. Eek. That sounds like a bug. Can you provide minimal steps to reproduce it? (i.e., getting a site directory to end up in easy-install.pth) From mwhapples at aim.com Tue Sep 23 17:24:35 2008 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 23 Sep 2008 16:24:35 +0100 Subject: [Distutils] Setup tools causes problems with orca finding its python packages In-Reply-To: <20080923144904.2CF343A40C2@sparrow.telecommunity.com> References: <1222180342.11820.47.camel@localhost> <20080923144904.2CF343A40C2@sparrow.telecommunity.com> Message-ID: <1222183475.24613.14.camel@localhost> This seems to do it (I know that this is not the correct way of doing things, but I have made the mistake): As root install plasTeX (http://plastex.sf.net, also in pypi) from source by using the setup.py script in it. This should add a plasTeX directory to your site-packages directory. Forget to uninstall it (IE. that source installation is still there when you do the following). Running as root to perform the install, give the following command easy_install -Z plasTeX Now look in easy-install.pth in the site packages directory (/usr/lib/python2.5/site-packages/easy-install.pth on my system), I find /usr/lib/python2.5/site-packages listed there after the above command. I don't believe this is specific to plasTeX but this is the one I noticed it with. Also I know this isn't the correct way to do things, we should remember to uninstall source installations first, but sometimes things slip our minds. If plasTeX isn't installed in any form before installing with easy_install this bug doesn't show, a subdirectory is created and the subdirectory is added. Michael Whapples On Tue, 2008-09-23 at 10:50 -0400, Phillip J. Eby wrote: > At 03:32 PM 9/23/2008 +0100, Michael Whapples wrote: > >I have noticed that easy_install has a easy-install.pth file which seems > >to be adding the directories specified in it before any other python > >package directories, including the environment variable PYTHONPATH. This > >causes a problem with orca which uses a bash script to start itself, and > >specifies where its python packages should be found by setting the > >PYTHONPATH variable. This wouldn't cause problems except when a user has > >more than one version of orca installed (eg. a stable version from the > >distribution installed in the standard python site-packages directory > >and one (SVN trunk version) in a custom directory). I found that > >easy_install had added the standard python site-packages directory > >(/usr/lib/python2.5/site-packages on my system) to easy-install.pth so > >making it impossible to use my custom install of orca as the standard > >location was first in sys.path. > > Eek. That sounds like a bug. Can you provide minimal steps to > reproduce it? (i.e., getting a site directory to end up in easy-install.pth) > From jeff at drinktomi.com Tue Sep 23 23:30:11 2008 From: jeff at drinktomi.com (Jeff Younker) Date: Tue, 23 Sep 2008 14:30:11 -0700 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D8C701.6070707@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> Message-ID: <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> I have to say, as a developer, and a system administrator, I like setuptools. It does what I need. Could it be better? Sure. For what I use python for on a day-to-day basis it makes my life a thousand times better than it was before setuptools. Nothing ruins your day more than spending *hours* tracing down package dependencies just to get the *one* package you need to allow you to perform some crucial task. It's even worse when you have to do it on multiple architectures. Perl's package location and installation system (CPAN) is one of the primary facts contributing to its success. Perl is a pig. It's a charming pig that can do lots of tricks, but a pig none the less. What makes it shine is CPAN. And here's the catch: CPAN isn't really any better than setuptools. It's got warts and nuts all over the place, but it works. Without setuptools a lot of people wouldn't be using python. Easily installing packages is critical to the success of a language, and setuptools fills that role admirably. [ I think what we really need to focus on as a community is binary generation generation. There are several tools out there that work for different platforms, but nothing that, well, just works everywhere. I'd far rather have one of those, than bicker over a perfectly functional setuptools. ] -jeff From rwarner at snaplogic.org Tue Sep 23 23:44:29 2008 From: rwarner at snaplogic.org (Rick Warner) Date: Tue, 23 Sep 2008 14:44:29 -0700 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> Message-ID: <48D9633D.6060407@snaplogic.org> Jeff Younker wrote: > I have to say, as a developer, and a system administrator, I like > setuptools. It does > what I need. Could it be better? Sure. For what I use python for on > a day-to-day > basis it makes my life a thousand times better than it was before > setuptools. Nothing > ruins your day more than spending *hours* tracing down package > dependencies > just to get the *one* package you need to allow you to perform some > crucial task. > It's even worse when you have to do it on multiple architectures. > > Perl's package location and installation system (CPAN) is one of the > primary facts > contributing to its success. Perl is a pig. It's a charming pig > that can do lots of tricks, > but a pig none the less. What makes it shine is CPAN. And here's the > catch: CPAN > isn't really any better than setuptools. It's got warts and nuts all > over the place, but > it works. And CPAN has some HUGE advantages over setuptools: it is designed as a repository, and it is replicated. Which means it is dependable. Anyone who suffered through the multiple outages of PyPI (which in not replicated) over the past year or so, or the ongoing outages of the many repositories across the web to which PyPI directs users/processes, can understand why this is important. - rick From jim at zope.com Wed Sep 24 00:20:36 2008 From: jim at zope.com (Jim Fulton) Date: Tue, 23 Sep 2008 18:20:36 -0400 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D9633D.6060407@snaplogic.org> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> Message-ID: On Sep 23, 2008, at 5:44 PM, Rick Warner wrote: > Jeff Younker wrote: >> I have to say, as a developer, and a system administrator, I like >> setuptools. It does >> what I need. Could it be better? Sure. For what I use python for >> on a day-to-day >> basis it makes my life a thousand times better than it was before >> setuptools. Nothing >> ruins your day more than spending *hours* tracing down package >> dependencies >> just to get the *one* package you need to allow you to perform some >> crucial task. >> It's even worse when you have to do it on multiple architectures. >> >> Perl's package location and installation system (CPAN) is one of >> the primary facts >> contributing to its success. Perl is a pig. It's a charming pig >> that can do lots of tricks, >> but a pig none the less. What makes it shine is CPAN. And here's >> the catch: CPAN >> isn't really any better than setuptools. It's got warts and nuts >> all over the place, but >> it works. > > And CPAN has some HUGE advantages over setuptools: it is designed as > a repository, and it is replicated. Which means it is dependable. > Anyone who suffered through the multiple outages of PyPI (which in > not replicated) over the past year or so, or the ongoing outages of > the many repositories across the web to which PyPI directs users/ > processes, can understand why this is important. Actually, PyPI is replicated. See, for example, http://download.zope.org/simple/ . It may be that some of the mirrors should be better advertised. Jim -- Jim Fulton Zope Corporation From rwarner at snaplogic.org Wed Sep 24 00:42:03 2008 From: rwarner at snaplogic.org (Rick Warner) Date: Tue, 23 Sep 2008 15:42:03 -0700 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> Message-ID: <48D970BB.7070504@snaplogic.org> Jim Fulton wrote: > > On Sep 23, 2008, at 5:44 PM, Rick Warner wrote: > >> Jeff Younker wrote: >>> I have to say, as a developer, and a system administrator, I like >>> setuptools. It does >>> what I need. Could it be better? Sure. For what I use python for >>> on a day-to-day >>> basis it makes my life a thousand times better than it was before >>> setuptools. Nothing >>> ruins your day more than spending *hours* tracing down package >>> dependencies >>> just to get the *one* package you need to allow you to perform some >>> crucial task. >>> It's even worse when you have to do it on multiple architectures. >>> >>> Perl's package location and installation system (CPAN) is one of the >>> primary facts >>> contributing to its success. Perl is a pig. It's a charming pig >>> that can do lots of tricks, >>> but a pig none the less. What makes it shine is CPAN. And here's >>> the catch: CPAN >>> isn't really any better than setuptools. It's got warts and nuts >>> all over the place, but >>> it works. >> >> And CPAN has some HUGE advantages over setuptools: it is designed as >> a repository, and it is replicated. Which means it is dependable. >> Anyone who suffered through the multiple outages of PyPI (which in >> not replicated) over the past year or so, or the ongoing outages of >> the many repositories across the web to which PyPI directs >> users/processes, can understand why this is important. > > > Actually, PyPI is replicated. See, for example, > http://download.zope.org/simple/. > > It may be that some of the mirrors should be better advertised. A half-hearted effort. at best, after the problems last year. When I configure a CPAN client (once per user) I create a list of replicas I want to search for any query from a list of hundreds of replicas distributed around the world. From then on the client automatically switches to one of my selected replicas when one does not respond in a timely manner. The minimal set of recent PyPI replicas are neither well advertised, and are not automatically searched, so therefore ineffective. And that is a mere tip of the iceberg, since PyPI is just the index, and the repositories are for the most part not replicated. CPAN sites are both index and repository. Setuptools and PyPI are light years behind CPAN in regards to creating a usable, reliable method of package deployment. - rick From jim at zope.com Wed Sep 24 02:24:51 2008 From: jim at zope.com (Jim Fulton) Date: Tue, 23 Sep 2008 20:24:51 -0400 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D970BB.7070504@snaplogic.org> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> <48D970BB.7070504@snaplogic.org> Message-ID: <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> On Sep 23, 2008, at 6:42 PM, Rick Warner wrote: > Jim Fulton wrote: >> >> On Sep 23, 2008, at 5:44 PM, Rick Warner wrote: >> >>> Jeff Younker wrote: >>>> I have to say, as a developer, and a system administrator, I like >>>> setuptools. It does >>>> what I need. Could it be better? Sure. For what I use python >>>> for on a day-to-day >>>> basis it makes my life a thousand times better than it was before >>>> setuptools. Nothing >>>> ruins your day more than spending *hours* tracing down package >>>> dependencies >>>> just to get the *one* package you need to allow you to perform >>>> some crucial task. >>>> It's even worse when you have to do it on multiple architectures. >>>> >>>> Perl's package location and installation system (CPAN) is one of >>>> the primary facts >>>> contributing to its success. Perl is a pig. It's a charming >>>> pig that can do lots of tricks, >>>> but a pig none the less. What makes it shine is CPAN. And here's >>>> the catch: CPAN >>>> isn't really any better than setuptools. It's got warts and nuts >>>> all over the place, but >>>> it works. >>> >>> And CPAN has some HUGE advantages over setuptools: it is designed >>> as a repository, and it is replicated. Which means it is >>> dependable. Anyone who suffered through the multiple outages of >>> PyPI (which in not replicated) over the past year or so, or the >>> ongoing outages of the many repositories across the web to which >>> PyPI directs users/processes, can understand why this is important. >> >> >> Actually, PyPI is replicated. See, for example, http://download.zope.org/simple/ >> . >> >> It may be that some of the mirrors should be better advertised. > > A half-hearted effort. at best, Hardly, but there's always room for improvement. > after the problems last year. When I configure a CPAN client (once > per user) I create a list of replicas I want to search for any query > from a list of hundreds of replicas distributed around the world. > From then on the client automatically switches to one of my selected > replicas when one does not respond in a timely manner. That's good. That would be nice to add to setuptools. > The minimal set of recent PyPI replicas are neither well advertised, Yes > and are not automatically searched, Setuptools and many tools built on it let you configure the index to use. It's true that you can configure only one, but I've found that to be sufficient for my needs, but I agree, being able to search many would be nice. > so therefore ineffective. Lots of people have found them to be very effective. > And that is a mere tip of the iceberg, since PyPI is just the index, > and the repositories are for the most part not replicated. CPAN > sites are both index and repository. My mirror is just an index, yes. I've found PyPI's repository to be reliable enough for my needs. I've had the most trouble when distributions aren't stored in the PyPI repository. People are building mirrors that mirror both index and repository/ > Setuptools and PyPI are light years behind CPAN in regards to > creating a usable, reliable method of package deployment. Behind, yes. Light years? I don't think so. Jim -- Jim Fulton Zope Corporation From andy at andrewprice.me.uk Wed Sep 24 03:44:31 2008 From: andy at andrewprice.me.uk (Andrew Price) Date: Wed, 24 Sep 2008 01:44:31 +0000 (UTC) Subject: [Distutils] Msgfmt in distutils? Message-ID: Hi, Distutils has made my life easier as an python application author and packager for some time but when it comes to generating and distributing message catalogues for translations, things get unexpectedly tricky. It would be nice to be able to do something like, --------- from distutils.core import setup setup(name="foo", version="1.0", description="A handy, translated application", ... po_files = [('share/locales', ['po/*.po'])] ) ---------- and have distutils do the right thing with the .po files at build time (generate .mo files from them) and at install time (install them into PREFIX/share/locales/LC_MESSAGES/, or wherever the distribution is configured to put them). My big question is, are there plans to add msgfmt functionality into distutils? Googling tells me there was some discussion about this back in 2003 but I can't find any sign of progress since then. Regards, -- Andy Price From ianb at colorstudy.com Wed Sep 24 06:48:39 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 23 Sep 2008 23:48:39 -0500 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D970BB.7070504@snaplogic.org> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> <48D970BB.7070504@snaplogic.org> Message-ID: <48D9C6A7.1010409@colorstudy.com> Rick Warner wrote: >> Actually, PyPI is replicated. See, for example, >> http://download.zope.org/simple/. >> >> It may be that some of the mirrors should be better advertised. > > A half-hearted effort. at best, after the problems last year. When I > configure a CPAN client (once per user) I create a list of replicas I > want to search for any query from a list of hundreds of replicas > distributed around the world. Can someone suggest the best way to search among repositories? For instance, try to connect to one, then stop if it gives Connection Refused? If it gives any unexpected error (5xx)? Timing out is a common failure, and a pain in the butt, but I guess there's that too. What does the CPAN client do? -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From kevin at bud.ca Wed Sep 24 06:24:00 2008 From: kevin at bud.ca (Kevin Teague) Date: Tue, 23 Sep 2008 21:24:00 -0700 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <1222170824.8878.442.camel@balin> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> Message-ID: <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> > > I guess the question in my mind is if the Ruby community have Ruby > Gems, > what is the Python equivalent, and why doesn't it work? > Python Eggs == Ruby Gems, and they both work more or less equally well as a packaging format. Maybe people just hear "Ruby Gems is awesome and Python don't got squat" because Python people are grumpier than Ruby people? Maybe it's because the Python Eggs page sports a PEAK logo that looks like it was made with Corel Draw 5 and this brings backs unpleasant memories of when they got paid $6 bucks an hour making coupons in Corel Draw 5 whereas the Ruby Gems page has an eye-pleasing RubyGem logo? Perhaps we should get Tony Robbins to present, "The Eggscellent Power of Positive Thinking" as a keynote at the next PyCon? And from our communities newfound exuberant positivity, talented graphic designers will be lining up to participate in? OK, joking aside, it's differences in how eggs and gems are distributed and consumed that I think make people claims that one works better than the other. Ruby Gems are installed with the 'gem' script, which installs gems in a versioned cache location, whereas 'easy_install' by default installs Python eggs into a global, versionless location. The documentation for Ruby Gems is also on the whole more approachable then the Python Egg documentation, so when new people are learning the tool and they get stuck, with gems they tend to find their answer and go away happy and evangelical, whereas with eggs they might go away bitter and grumpy. Eggs have a small "Z-shaped" learning curve in that a new developer learns "sudo easy_install some_package" and it works and they say, "yay!". Later on though they want to use two versions of the same package and they realize that they have to learn how-to do things differently *and* they're presented with TIMTOWTDI - either manual management (symlinks or hand-munged .pth files) or setuptools or multiple Python installs or VirtualEnv or Buildout or some combination of approaches. To a certain extent, TIMTOWTDI is necessary with package management, since there are so many different use cases - but it would be very nice if there was an approachable documentation resource to help people explore these different tools and techniqiues more easily though. Or they can just use debian! Any debian developers out there up for the task of packaging up the 1500+ odd packages released from the Zope community? From ianb at colorstudy.com Wed Sep 24 07:31:17 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 24 Sep 2008 00:31:17 -0500 Subject: [Distutils] pyinstall Message-ID: <48D9D0A5.3010305@colorstudy.com> I just announced a new installer I've been working on: http://pypi.python.org/pypi/pyinstall (also the not-very-thorough docs) Announcement post: http://www.openplans.org/projects/topp-engineering/blog/2008/09/24/pyinstall-a-new-hope/ Single-file executable for you to try: https://svn.openplans.org/svn/pyinstall/trunk/pyinstall.py I think it's good. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From lists at zopyx.com Wed Sep 24 08:15:30 2008 From: lists at zopyx.com (Andreas Jung) Date: Wed, 24 Sep 2008 08:15:30 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> Message-ID: <5223B6693FA203527ED9125C@[192.168.1.22]> --On 23. September 2008 18:20:36 -0400 Jim Fulton wrote: > > On Sep 23, 2008, at 5:44 PM, Rick Warner wrote: > >> Jeff Younker wrote: >>> I have to say, as a developer, and a system administrator, I like >>> setuptools. It does >>> what I need. Could it be better? Sure. For what I use python for >>> on a day-to-day >>> basis it makes my life a thousand times better than it was before >>> setuptools. Nothing >>> ruins your day more than spending *hours* tracing down package >>> dependencies >>> just to get the *one* package you need to allow you to perform some >>> crucial task. >>> It's even worse when you have to do it on multiple architectures. >>> >>> Perl's package location and installation system (CPAN) is one of >>> the primary facts >>> contributing to its success. Perl is a pig. It's a charming pig >>> that can do lots of tricks, >>> but a pig none the less. What makes it shine is CPAN. And here's >>> the catch: CPAN >>> isn't really any better than setuptools. It's got warts and nuts >>> all over the place, but >>> it works. >> >> And CPAN has some HUGE advantages over setuptools: it is designed as >> a repository, and it is replicated. Which means it is dependable. >> Anyone who suffered through the multiple outages of PyPI (which in >> not replicated) over the past year or so, or the ongoing outages of >> the many repositories across the web to which PyPI directs users/ >> processes, can understand why this is important. > > > Actually, PyPI is replicated. See, for example, > http://download.zope.org/simple/. > > It may be that some of the mirrors should be better advertised. > > For the logs: we are currently working on a mirroring infrastructure with a set of several full PyPI mirror (based on z3c.pypimirror)....more to be announced and worked on in the middle of October. Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: From david at ar.media.kyoto-u.ac.jp Wed Sep 24 08:27:51 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 24 Sep 2008 15:27:51 +0900 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> Message-ID: <48D9DDE7.6010601@ar.media.kyoto-u.ac.jp> Kevin Teague wrote: > Python Eggs == Ruby Gems, and they both work more or less equally well > as a packaging format. I don't agree. Sure, they are the same idea, but the implementation is vastly different, and that's what matters IMHO, or at least is one big problem. If you look at the ruby gem page, you have one link for a specification; I have not done it, and maybe I would realize I were wrong by trying it, but I got the impression I could generate gems myself from the specification. Can I do that with eggs ? Also, gems and rake/rant are different projects (maybe by different people ?). In practice, it is nice to have everything integrated (and ruby gems certainly feel as integrated as python eggs), but having different packages for different tasks force to have proper behavior, not a behavior which works in some cases, and broke in others. And rant is a proper build system, whereas distutils isn't. There is also the problem that by making some things easy but effectively "magic", when it breaks, you don't know how to fix. Those two problems (everything intermixed and magic) are linked. If several tasks were separated, there would have been a clear specification/API, and less magic. Of course, basing setuptools on the top of distutils make this task nearly impossible (but I understand it was the best if not only choice given Philip requirements when he started setuptools). You have people who ignore the problem eggs are trying to solve (the "install debian and solve real problems" crowd), but I don't think the majority of people who object to eggs in principle. They object to implementation problems. cheers, David From ziade.tarek at gmail.com Wed Sep 24 09:25:43 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 09:25:43 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <2DF68F3D-6DAD-40A9-98DA-C28999A15E04@drinktomi.com> <48D9633D.6060407@snaplogic.org> <48D970BB.7070504@snaplogic.org> <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> Message-ID: <94bdd2610809240025r3510bf05k19a2a8d78e67b347@mail.gmail.com> On Wed, Sep 24, 2008 at 2:24 AM, Jim Fulton wrote: > > On Sep 23, 2008, at 6:42 PM, Rick Warner wrote: > > Jim Fulton wrote: >> >>> >>> On Sep 23, 2008, at 5:44 PM, Rick Warner wrote: >>> >>> Jeff Younker wrote: >>>> >>>>> I have to say, as a developer, and a system administrator, I like >>>>> setuptools. It does >>>>> what I need. Could it be better? Sure. For what I use python for on >>>>> a day-to-day >>>>> basis it makes my life a thousand times better than it was before >>>>> setuptools. Nothing >>>>> ruins your day more than spending *hours* tracing down package >>>>> dependencies >>>>> just to get the *one* package you need to allow you to perform some >>>>> crucial task. >>>>> It's even worse when you have to do it on multiple architectures. >>>>> >>>>> Perl's package location and installation system (CPAN) is one of the >>>>> primary facts >>>>> contributing to its success. Perl is a pig. It's a charming pig that >>>>> can do lots of tricks, >>>>> but a pig none the less. What makes it shine is CPAN. And here's the >>>>> catch: CPAN >>>>> isn't really any better than setuptools. It's got warts and nuts all >>>>> over the place, but >>>>> it works. >>>>> >>>> >>>> And CPAN has some HUGE advantages over setuptools: it is designed as a >>>> repository, and it is replicated. Which means it is dependable. Anyone >>>> who suffered through the multiple outages of PyPI (which in not replicated) >>>> over the past year or so, or the ongoing outages of the many repositories >>>> across the web to which PyPI directs users/processes, can understand why >>>> this is important. >>>> >>> >>> >>> Actually, PyPI is replicated. See, for example, >>> http://download.zope.org/simple/. >>> >>> It may be that some of the mirrors should be better advertised. >>> >> >> A half-hearted effort. at best, >> > > Hardly, but there's always room for improvement. > > after the problems last year. When I configure a CPAN client (once per >> user) I create a list of replicas I want to search for any query from a list >> of hundreds of replicas distributed around the world. From then on the >> client automatically switches to one of my selected replicas when one does >> not respond in a timely manner. >> > > That's good. That would be nice to add to setuptools. Well that is the patch I have proposed right here : let setuptools deal with several indexes http://bugs.python.org/setuptools/issue32 Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Wed Sep 24 10:42:34 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 10:42:34 +0200 Subject: [Distutils] Annoucing collective.eggproxy, the smart mirror for PyPI Message-ID: <94bdd2610809240142v15746b3er28edf2d4a886ed69@mail.gmail.com> Hi, I just wanted to announce the release of collective.eggproxy ( http://pypi.python.org/pypi/collective.eggproxy/0.2.0) collective.eggproxy is a smart mirror for PyPI. It will collect packages on PyPI only when a program like easy_install or zc.buildout asks for it. In other words, unlike some mirrors that act like rsync and get the whole PyPI base (more than 5 gigas) collective.eggproxy will only get what you need. At first run collective.eggproxy downloads pypi index and builds a page of links. When a software asks for a specific package, version, etc. collective.eggproxy downloads it if needed and stores it locally. Want to give a try ? try it in two lines with easy_install: $ easy_install collective.eggproxy $ eggproxy_run That's it ! Cheers Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Wed Sep 24 13:32:23 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 13:32:23 +0200 Subject: [Distutils] Annoucing distribute project Message-ID: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> Hi, I am launching a fork of setuptools. The name is "Distribute". (not definitive) and will be community-driven This fork will remain 100% compatible with setuptools, and follow closely setuptools evolutions. Roadmap ======= - Distribute 0.1 (09/27/2008) The first release of Distribute will be done at the end of this week, targeting setuptools 0.6x, and will provide all latest bugfixes done in the 0.6 branches of setuptools, like Subversion 1.5 compatibility. - Distribute 0.2 (10/27/2008) The 0.2 release will be done in 1 month, targeting setuptools current trunk, and will focus on providing a maximum of bugfixes. You can provide patches, or aks for a particular bug to be fixed. Use the launchpad bug tracker for that https://bugs.launchpad.net/distribute - Distribute 0.3 The 0.3 release will contain new features and improvments, and its release date decide when 0.2 comes out. Project managment =============== Distribute is using launchpad (https://launchpad.net/distribute) and bzr and anyone that wants to contribute is welcome to provide a branch. The branch are merged if they provide tests and if there is a consensus of the community on it. The process is as follow for a contribution: - create a bzr branch - talk about it on distutils-SIG, using [distribute] in the header, asking for review/merge - it is discussed in the mailing list - if no one objects, and if it has tests, it will be merged by a maintainer I am the maintainer at this point but this role will be extended to other people that want to become a maintainer, so I will not become a botlleneck for this project. A maintainer has to be nominated by 100% of the maintainers, and there will be at most 4 maintainers in this project. Maintainers are added or removed after each release. They work together to release the next milestone. Let me know if you want to become a maintainer. Maintainers need to discuss their changes as well on the mailing list, and their only privilege is being able to merge branches on the main branch, because they are trust to do so. Cheers Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From doko at cs.tu-berlin.de Wed Sep 24 13:49:01 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Wed, 24 Sep 2008 13:49:01 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> Message-ID: <18650.10541.768378.823880@gargle.gargle.HOWL> announcing a fork without any objective? what is this fork about? if it stays 100% compatible, isn't it just more frequent releases? Matthias Tarek Ziad? writes: > Hi, > > I am launching a fork of setuptools. The name is "Distribute". (not > definitive) and will be community-driven > > This fork will remain 100% compatible with setuptools, and follow closely > setuptools evolutions. > > Roadmap > ======= > > - Distribute 0.1 (09/27/2008) > > The first release of Distribute will be done at the end of this week, > targeting setuptools 0.6x, > and will provide all latest bugfixes done in the 0.6 branches of setuptools, > like Subversion 1.5 compatibility. > > - Distribute 0.2 (10/27/2008) > > The 0.2 release will be done in 1 month, targeting setuptools current trunk, > and will focus on > providing a maximum of bugfixes. You can provide patches, or aks for a > particular bug to be fixed. > Use the launchpad bug tracker for that https://bugs.launchpad.net/distribute > > - Distribute 0.3 > > The 0.3 release will contain new features and improvments, and its release > date decide when 0.2 comes out. > > Project managment > =============== > > Distribute is using launchpad (https://launchpad.net/distribute) and bzr and > anyone that wants to > contribute is welcome to provide a branch. > > The branch are merged if they provide tests and if there is a consensus of > the community > on it. > > The process is as follow for a contribution: > > - create a bzr branch > - talk about it on distutils-SIG, using [distribute] in the header, asking > for review/merge > - it is discussed in the mailing list > - if no one objects, and if it has tests, it will be merged by a maintainer > > I am the maintainer at this point but this role will be extended to other > people that want to become > a maintainer, so I will not become a botlleneck for this project. > > A maintainer has to be nominated by 100% of the maintainers, and there will > be at most > 4 maintainers in this project. Maintainers are added or removed after each > release. > They work together to release the next milestone. > > Let me know if you want to become a maintainer. Maintainers need to discuss > their changes > as well on the mailing list, and their only privilege is being able to merge > branches on the main branch, > because they are trust to do so. > > Cheers > Tarek > > -- > Tarek Ziad? | Association AfPy | www.afpy.org > Blog FR | http://programmation-python.org > Blog EN | http://tarekziade.wordpress.com/ >
Hi,

I am launching a fork of setuptools. The name is "Distribute". (not definitive) and will be community-driven

This fork will remain 100% compatible with setuptools, and follow closely setuptools evolutions.
>
Roadmap
=======

- Distribute 0.1 (09/27/2008)

The first release of Distribute will be done at the end of this week, targeting setuptools 0.6x,
and will provide all latest bugfixes done in the 0.6 branches of setuptools, like Subversion 1.5 compatibility.
>
- Distribute 0.2 (10/27/2008)
>
The 0.2 release will be done in 1 month, targeting setuptools current trunk, and will focus on
providing a maximum of bugfixes. You can provide patches, or aks for a particular bug to be fixed.
Use the launchpad bug tracker for that https://bugs.launchpad.net/distribute
>
- Distribute 0.3

The 0.3 release will contain new features and improvments, and its release date decide when 0.2 comes out.

Project managment
===============

Distribute is using launchpad (https://launchpad.net/distribute) and bzr and anyone that wants to
> contribute is welcome to provide a branch.

The branch are merged if they provide tests and if there is a consensus of the community
on it.

The process is as follow for a contribution:

- create a bzr branch
> - talk about it on distutils-SIG, using [distribute] in the header, asking for review/merge
- it is discussed in the mailing list
- if no one objects, and if it has tests, it will be merged by a maintainer

I am the maintainer at this point but this role will be extended to other people that want to become
> a maintainer, so I will not become a botlleneck for this project.

A maintainer has to be nominated by 100% of the maintainers, and there will be at most
4 maintainers in this project. Maintainers are added or removed after each release.
> They work together to release the next milestone.

Let me know if you want to become a maintainer. Maintainers need to discuss their changes
as well on the mailing list, and their only privilege is being able to merge branches on the main branch,
> because they are trust to do so.

Cheers
Tarek

--
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>
> _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From ziade.tarek at gmail.com Wed Sep 24 14:00:03 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 14:00:03 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <18650.10541.768378.823880@gargle.gargle.HOWL> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> Message-ID: <94bdd2610809240500v6fbe9941wdd2900851e9bb375@mail.gmail.com> On Wed, Sep 24, 2008 at 1:49 PM, Matthias Klose wrote: > announcing a fork without any objective? what is this fork about? if > it stays 100% compatible, isn't it just more frequent releases? This fork is about having a tool that is build for and by the community, and that does not rely on one single person, that does not have enough spare time to work on it and therefore become a bottleneck. Right now setuptools has one single person with commit privileges, and that maintains it. He is a very busy man and we have been struggling for months to speed up bug fixes and releases. This tool became a very important part of python development in many places, and we need to make it evolve faster. If setuptools development process changes, I will of course, stop the fork. Regards Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdouche at gmail.com Wed Sep 24 14:32:15 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Wed, 24 Sep 2008 14:32:15 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <18650.10541.768378.823880@gargle.gargle.HOWL> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> Message-ID: <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> On Wed, Sep 24, 2008 at 13:49, Matthias Klose wrote: > announcing a fork without any objective? what is this fork about? Python community *must* have a robust, stable and well featured infrastructure like Cpan, Gem or Apt. The actual situation is insane: - Setuptools is not on Python core - single point failure infrastructure (pypi.p.o) - not compatible with Hg / Bzr / Git out of the box - missing some features As far I see, the goal is to speed up the development of a great tool, not just a fork for fun :). PS: within 1 month, Ubuntu 8.10 & Debian 4.2 are out (default svn : 1.5). It's not acceptable to say "use trunk version, stupid!" -- Seb From ziade.tarek at gmail.com Wed Sep 24 14:39:02 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 14:39:02 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> Message-ID: <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> On Wed, Sep 24, 2008 at 2:32 PM, Sebastien Douche wrote: > On Wed, Sep 24, 2008 at 13:49, Matthias Klose > wrote: > > announcing a fork without any objective? what is this fork about? > > Python community *must* have a robust, stable and well featured > infrastructure like Cpan, Gem or Apt. The actual situation is insane: > > - Setuptools is not on Python core I agree this is a shame. The boundary between setuptools and distutils is fuzzy. But if we build together a good tool, I guess it will be included at some point as a distutils replacement. > > - single point failure infrastructure (pypi.p.o) notice that there's a patch at this point that let setuptools use several indexes, it will be discussed for inclusion in Distribute 0.3 > > - not compatible with Hg / Bzr / Git out of the box > Notice that you can write a Manifest.in file to avoid it at this point > > - missing some features > > > As far I see, the goal is to speed up the development of a great tool, > not just a fork for fun :). > exactly ! > > > PS: within 1 month, Ubuntu 8.10 & Debian 4.2 are out (default svn : > 1.5). It's not acceptable to say "use trunk version, stupid!" > > > -- > Seb > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ms at cerenity.org Wed Sep 24 14:31:27 2008 From: ms at cerenity.org (Michael) Date: Wed, 24 Sep 2008 13:31:27 +0100 Subject: [Distutils] [pyconuk] "Python Package Management Sucks" In-Reply-To: <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> References: <48D00408.50705@simplistix.co.uk> <48D970BB.7070504@snaplogic.org> <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> Message-ID: <200809241331.28077.ms@cerenity.org> On Wednesday 24 September 2008 01:24:51 Jim Fulton wrote: > > ?Setuptools and PyPI are light years behind CPAN in regards to ? > > creating a usable, reliable method of package deployment. > > Behind, yes. Light years? I don't think so. People have been complaining about python package management for the entire time I've used python. Now, I'm not old timer for python only 6 and a bit years or so - after having rejected it originally 10 years ago, but people have definitely been complaining for 6 years at least. Now before that I'd been writing perl code for many years, and they'd had this system stolen from the TeX community (CTAN) which they'd called CPAN and that had been actively useful to everyone in the Perl community from the first time I came across perl (which was 10 years ago). Now, with python there's the general ethos: There should be one-- and preferably only one --obvious way to do it. And with perl there's the general ethos: There's more than one way to do it Anyone who's written extensive amounts of code in both languages will know that the latter ethos does cause major problems in practice. However for packaging, with python the rule is * "There's more than one way to do it" And for perl the rule is: * Use CPAN I've always found this difference amusing, and never said much about this because one of my earliest experiences of the python "community" was seeing a poor perl developer at Europython in 2005 ripped to shreds by (someone I would expect nicer behaviour from) for merely suggesting "hey, the perl people actually have figured this out and have one obvious way to do it". Now, personally, given that CPAN (which I think borrowed the idea from CTAN - I think - I definitely heard of CTAN first) has worked very happily for the perl community for the past 10 years (the time I've used perl for), and given that the python community still has problems (pyinstall being the latest in the saga), maybe the time has come to recognise that perl maybe isn't "light years" ahead on this, but actually is a decade ahead at least. After all, the question: * What do I use to package my code *IS* a discussion point for python. For Perl, it's a FAQ with the answer being "CPAN". Sure, CPAN has had (and probably still has occasionally) problems, and it is far from perfect but at least the perl people DO have one obvious way to do it. Heck, in python its a discussion so old that people become INCREDIBLY ratty about it, and so on that note since I'll probably get flamed to hell and back for the above (given what happened at Europython a few years back), I'll back out quietly now. Regards, Michael. From barry at python.org Wed Sep 24 15:08:53 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 24 Sep 2008 09:08:53 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> Message-ID: <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 24, 2008, at 8:39 AM, Tarek Ziad? wrote: > - not compatible with Hg / Bzr / Git out of the box > > Notice that you can write a Manifest.in file to avoid it at this point Tarek, nice to see this hosted on Launchpad. Maintaining the code under a dvcs will be a huge win. I'm happy to provide any help with the use of Launchpad that I can. Let me also mention that Launchpad is a webservice scriptable in Python: https://launchpad.net/launchpadlib There are several plugins for integrating setuptools with the dvcs's above. I maintain the bzr one: https://launchpad.net/setuptoolsbzr and would be happy to help integrate it into the core. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSNo75XEjvBPtnXfVAQIzTwP+K9BpTRD9nalK7JfkE0CRO/ouX1uQMhTN xiZSP9C5M4UT7jsnOeo9ylZJwmgPhP/YEgnquTNlsdEWnYTGrcXVZV22utH0t+L7 uumlmpoMN1d8UVy4DBLNz75bpW5Etx3dohwLN6Nc5lddyMa4zHCHmMJNUezmtBHF iHJPkb6Uul0= =95Dn -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Wed Sep 24 15:20:05 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 15:20:05 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> Message-ID: <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> On Wed, Sep 24, 2008 at 3:08 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Sep 24, 2008, at 8:39 AM, Tarek Ziad? wrote: > > - not compatible with Hg / Bzr / Git out of the box >> >> Notice that you can write a Manifest.in file to avoid it at this point >> > > Tarek, nice to see this hosted on Launchpad. Maintaining the code under a > dvcs will be a huge win. I'm happy to provide any help with the use of > Launchpad that I can. Let me also mention that Launchpad is a webservice > scriptable in Python: > > https://launchpad.net/launchpadlib Great, thanks for the tips on Launchpad usage, and the support > > There are several plugins for integrating setuptools with the dvcs's above. > I maintain the bzr one: > > https://launchpad.net/setuptoolsbzr > > and would be happy to help integrate it into the core. After the 0.1 release of Distribute (which will be Friday) , we can integrate it for the next release, the idea of 0.2 is to integrated all pending bugfixes and obvious patches Cheers > > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSNo75XEjvBPtnXfVAQIzTwP+K9BpTRD9nalK7JfkE0CRO/ouX1uQMhTN > xiZSP9C5M4UT7jsnOeo9ylZJwmgPhP/YEgnquTNlsdEWnYTGrcXVZV22utH0t+L7 > uumlmpoMN1d8UVy4DBLNz75bpW5Etx3dohwLN6Nc5lddyMa4zHCHmMJNUezmtBHF > iHJPkb6Uul0= > =95Dn > -----END PGP SIGNATURE----- > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fadhley.salim at uk.calyon.com Wed Sep 24 15:24:15 2008 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Wed, 24 Sep 2008 14:24:15 +0100 Subject: [Distutils] Can we prevent setuptools from writing to stdout References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com><18650.10541.768378.823880@gargle.gargle.HOWL><5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com><94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com><2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> Is it possible to prevent setuptools from writing anything to stdout? For various inconvenient reasons our package must install silently - that means the installation process must not write to either stdout or stderr. Other than hacking distutils / setuptools to ensure that there's nothing which can write i've not yet been able to come up with a reliable means to control output. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Commission Bancaire in France and regulated by the Financial Services Authority for the conduct of business in the United Kingdom. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From barry at python.org Wed Sep 24 15:38:04 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 24 Sep 2008 09:38:04 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 24, 2008, at 9:20 AM, Tarek Ziad? wrote: > After the 0.1 release of Distribute (which will be Friday) , we can > integrate it for the next release, > the idea of 0.2 is to integrated all pending bugfixes and obvious > patches Sounds good. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSNpCvHEjvBPtnXfVAQK51wP+OHgcBDQqSRocJid3Vxdx3c4gFx5YlBv0 hf20siN3cmCdR+q05X2pX/UHeb2eXoMSb/pMm+Uic5s0U9VMgdKOu6B/AG5e952g q2xU867gafiA0zInWq2EXXrjsuldKa4963aPFHlyo29F0TvZS6yqgQeK86sfM4da 24MMk8dDSuw= =WHXo -----END PGP SIGNATURE----- From tarek.ziade at ingeniweb.com Wed Sep 24 15:33:25 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Wed, 24 Sep 2008 15:33:25 +0200 Subject: [Distutils] Can we prevent setuptools from writing to stdout In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> Message-ID: 2008/9/24 Fadhley Salim > Is it possible to prevent setuptools from writing anything to stdout? > > For various inconvenient reasons our package must install silently - that > means the installation process must not write to either stdout or stderr. > Other than hacking distutils / setuptools to ensure that there's nothing > which can write i've not yet been able to come up with a reliable means to > control output. > > Thanks > At distutils level you can set the verbosity and reduce the output, by setting it to 0 (distutils.log.set_verbosity) If your package does not provocate warnings, and if the commands used are all using this log facility (iirc setuptools does it everywhere?, you should not have any output. That said, this log module should be dropped in favor of the logging module in distutils Cheers Tarek > This email does not create a legal relationship between any member of the > Cr?dit Agricole group and the recipient or constitute investment advice. > The content of this email should not be copied or disclosed (in whole or > part) to any other person. It may contain information which is > confidential, privileged or otherwise protected from disclosure. If you > are not the intended recipient, you should notify us and delete it from > your system. Emails may be monitored, are not secure and may be amended, > destroyed or contain viruses and in communicating with us such conditions > are accepted. Any content which does not relate to business matters is not > endorsed by us. > > Calyon is authorised by the Commission Bancaire in France and regulated by > the Financial Services Authority for the conduct of business in the > United Kingdom. Calyon is incorporated in France with limited liability > and registered in England & Wales. Registration number: FC008194. > Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From fadhley.salim at uk.calyon.com Wed Sep 24 15:41:38 2008 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Wed, 24 Sep 2008 14:41:38 +0100 Subject: [Distutils] Can we prevent setuptools from writing to stdout References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483D91@MXCU10MX1.MSX.CIB> I assume that what you meant to says was that we should drop the logging module in distutils and replace it with the standard python-library logger in the logging module - if so, that would be great for us. I've always wondered why distutils has it's own implementation of logging. Thanks ________________________________ From: Tarek Ziade [mailto:tarek.ziade at ingeniweb.com] Sent: 24 September 2008 14:33 To: Salim, Fadhley (CALYON) Cc: distutils-sig Subject: Re: [Distutils] Can we prevent setuptools from writing to stdout 2008/9/24 Fadhley Salim Is it possible to prevent setuptools from writing anything to stdout? For various inconvenient reasons our package must install silently - that means the installation process must not write to either stdout or stderr. Other than hacking distutils / setuptools to ensure that there's nothing which can write i've not yet been able to come up with a reliable means to control output. Thanks At distutils level you can set the verbosity and reduce the output, by setting it to 0 (distutils.log.set_verbosity) If your package does not provocate warnings, and if the commands used are all using this log facility (iirc setuptools does it everywhere?, you should not have any output. That said, this log module should be dropped in favor of the logging module in distutils Cheers Tarek This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Commission Bancaire in France and regulated by the Financial Services Authority for the conduct of business in the United Kingdom. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Commission Bancaire in France and regulated by the Financial Services Authority for the conduct of business in the United Kingdom. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From tarek.ziade at ingeniweb.com Wed Sep 24 15:48:02 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Wed, 24 Sep 2008 15:48:02 +0200 Subject: [Distutils] Can we prevent setuptools from writing to stdout In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483D91@MXCU10MX1.MSX.CIB> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> <7F347D91614EBC48AA7540A2A76D3BB204483D91@MXCU10MX1.MSX.CIB> Message-ID: 2008/9/24 Fadhley Salim > I assume that what you meant to says was that we should drop the logging > module in distutils and replace it with the standard python-library logger > in the logging module - if so, that would be great for us. > > exactly. The logging module exists for that kind of needs. > I've always wondered why distutils has it's own implementation of logging. > It's because it did not evolve at this point I guess, I will propose a patch in the python bug tracker for that. Tarek > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jp.camguilhem at gmail.com Wed Sep 24 16:10:48 2008 From: jp.camguilhem at gmail.com (Jean-Philippe CAMGUILHEM) Date: Wed, 24 Sep 2008 16:10:48 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> Message-ID: On Wed, Sep 24, 2008 at 2:39 PM, Tarek Ziad? wrote: > > > On Wed, Sep 24, 2008 at 2:32 PM, Sebastien Douche > wrote: > >> On Wed, Sep 24, 2008 at 13:49, Matthias Klose >> wrote: >> > announcing a fork without any objective? what is this fork about? >> >> Python community *must* have a robust, stable and well featured >> infrastructure like Cpan, Gem or Apt. The actual situation is insane: >> > +1 > >> As far I see, the goal is to speed up the development of a great tool, >> not just a fork for fun :). >> > > exactly ! > +1 > > > >> >> >> PS: within 1 month, Ubuntu 8.10 & Debian 4.2 are out (default svn : >> 1.5). It's not acceptable to say "use trunk version, stupid!" >> > +1 Tarek Great job thanks ! I'm sure many improments ideas will be born on the launchpad project. how to not be agree with theses explanations ? : http://tarekziade.wordpress.com/2008/09/24/distribute-a-setuptools-fork/ @ ++ -------------- next part -------------- An HTML attachment was scrubbed... URL: From doko at cs.tu-berlin.de Wed Sep 24 16:16:39 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Wed, 24 Sep 2008 16:16:39 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> Message-ID: <18650.19399.50129.607737@gargle.gargle.HOWL> Sebastien Douche writes: > On Wed, Sep 24, 2008 at 13:49, Matthias Klose wrote: > > announcing a fork without any objective? what is this fork about? > > Python community *must* have a robust, stable and well featured > infrastructure like Cpan, Gem or Apt. The actual situation is insane: > > - Setuptools is not on Python core > - single point failure infrastructure (pypi.p.o) > - not compatible with Hg / Bzr / Git out of the box > - missing some features > > > As far I see, the goal is to speed up the development of a great tool, > not just a fork for fun :). > > > PS: within 1 month, Ubuntu 8.10 & Debian 4.2 are out (default svn : > 1.5). It's not acceptable to say "use trunk version, stupid!" For distributors like Debian, Fedora or Ubuntu, setuptools is still in a state where it's better ignored than used. It's fine to ship it, but using it in the distribution is a pain. Until today setuptools doesn't have the features needed by os distributors, and things like --single-version-externally-managed were only reluctantly added. Some things to change: - os distributors usually try to minimize the versions they include, trying to just ship one version. This single version is installed with --single-version-externally-managed, so that it can be imported without any pkg_resouces magic and fiddling with pth files. Unfortunately then it is not possible to use/import another version using pkg_resources. As discussed at PyCon such a setup is very common for os distributors. How will the fork handle such setups, and maybe incompatible changes? - setuptools has the narrow minded view of a python package being contained in a single directory, which doesn't fit well when you do have common locations for include or doc files. Does the fork accept patches to change such limitations and allowing FHS compliant packages? A branch with patches for the stable setuptools branch sounds fine. It makes it more easy to apply these fixes to the setuptools package as distributed by Debian and Ubuntu. But I doubt it will be good enough with the "compatibility approach" because changes like the ones mentioned above still seem to be left to the setuptools maintainer. Plus the name of the fork suggests that the all-coupled/all-integrated approach will live on, and distribute will remain a monolithic chunk of code. Matthias From gael.varoquaux at normalesup.org Wed Sep 24 15:41:59 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 24 Sep 2008 15:41:59 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> Message-ID: <20080924134159.GO1535@phare.normalesup.org> On Wed, Sep 24, 2008 at 09:38:04AM -0400, Barry Warsaw wrote: > On Sep 24, 2008, at 9:20 AM, Tarek Ziad? wrote: > > After the 0.1 release of Distribute (which will be Friday) , we can > > integrate it for the next release, > > the idea of 0.2 is to integrated all pending bugfixes and obvious > > patches > Sounds good. Yes, thanks Tarek for doing this. We know that Philip is very busy and does not have time to review and merge in that many patches for setuptools. Hopefully this new development will help by being a fast-moving project to explore new lines and offload some work from the main project. Ga?l From sdouche at gmail.com Wed Sep 24 17:12:17 2008 From: sdouche at gmail.com (Sebastien Douche) Date: Wed, 24 Sep 2008 17:12:17 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <18650.19399.50129.607737@gargle.gargle.HOWL> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> Message-ID: <5e1183fa0809240812u74e112d8u72b89a4ed7889d74@mail.gmail.com> On Wed, Sep 24, 2008 at 16:16, Matthias Klose wrote: >> PS: within 1 month, Ubuntu 8.10 & Debian 4.2 are out (default svn : >> 1.5). It's not acceptable to say "use trunk version, stupid!" > > For distributors like Debian, Fedora or Ubuntu, setuptools is still in > a state where it's better ignored than used. It's fine to ship it, but > using it in the distribution is a pain. Matthias, we speak about: http://bugs.python.org/setuptools/issue4 (Setuptools can't build a egg with svn 1.5) -- Seb From doko at cs.tu-berlin.de Wed Sep 24 18:48:33 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Wed, 24 Sep 2008 18:48:33 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48D8C701.6070707@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> Message-ID: <18650.28513.573244.313581@gargle.gargle.HOWL> Chris Withers writes: > (I'll be CC'ing the distutils sig in to these replies as this discussion > probably belongs there...) > > Nicolas Chauvat wrote: > >> The slides for my two talks can be found here: > >> > >> http://www.simplistix.co.uk/presentations > > > > "Python Package Management Sucks" > > > > Install debian and get back to productive tasks. > > This is an almost troll-like answer. > See page 35 of the presentation. I disagree. You could think of "Packages are Pythons Plugins" (taken from page 35) as a troll-like statement as well. Apparently this user is happy about Debian and Ubuntu how they distribute python modules and it's worth evaluating why. > If you're too lazy, here's a re-hash: > > - there are lots of different operating systems > > - even with Linux, there are many different package management > systems and a subset of all these operating systems (linux or other) do have the need to distribute python and a set of python modules and extensions. they cannot rely on a plugin system outside the (os) distribution. > - all the package management systems behave differently and expect > packages to be set up differently for them correct, but again they share common requirements. > - expecting package developers to shoulder this burden is unfair and > results in various bitrotting repackaged versions of python packages > rather than just one up-to-date version maintained by the entity > originating the python package some people prefer to name this "stable releases" instead of "bitrot". usually you can be assured that the set of python modules delivered with an (os) distribution is tested and works. I doubt you do achieve the same with an untested set of up-to-date versions. Speaking of extensions "maintained by the entity originating the python package": this much too often is a way of bitrot. is the shipped library up to date? does it have security fixes? how many duplicates are shipped in different extensions? does the library need to be shipped at all (because some os does ship it)? > - Adobe Photoshop Plugins, Firefox Add-ons, etc do not delegate their > management to an OS package manager. this is known trouble for os distributors, and your statement is generally wrong. firefox plugins are packaged in distributions and the plugin system is able to cope with packaged plugins. > Packages are Python's "plugins" and so should get the same type of > consistent, cross-platform package management targetted at the > application in question, which is Python in this case. No, as explained above. Considering an extension interfacing a library shipped with the os, you do want to use this library, not add another copy. and you do want to interface exactly this version. An upstream extension maintainer cannot provide this unless he builds this extension for every (os) distribution and maintains it during the os' lifecycle. Your view is very common (and legitimate) in the Python world (distributing an batteries-all-included application), but not everybody does need this (like os distributors, using the reliable-power-plug-where-available), and in some areas it starts to hurt. Your paper gives a nice overview of the shortcomings of each of the build/distribution systems. From an (os) distributors point of view I would add som things (also posted in [1]). - os distributors usually try to minimize the versions they include, trying to just ship one version. This single version is installed with --single-version-externally-managed, so that it can be imported without any pkg_resouces magic and fiddling with pth files. Unfortunately then it is not possible to use/import another version using pkg_resources. As discussed at PyCon such a setup is very common for os distributors. None of the tools does support this. - setuptools has the narrow minded view of a python package being contained in a single directory, which doesn't fit well when you do have common locations for include or doc files. A linux distribution is supposed to follow the File Hierarchy Standard (FHS). None of the tools does support this. - setuptools (sharing with distutils) doesn't allow splitting of a python package into several distribution binary packages, e.g. have a separate package with large docs, allowing installation on a server without having to install X, and so on. But maybe this is something an (os) distributor only cares about. You will see that most packages included in (os) distributions still use plain distutils, even if the setup.py does support a setuptools based install. That's simply because distutils doesn't get into the way packaging the python module with rpm or dpkg. E.g. namespace packages are a consequence how setuptools distributes and installs things. Why force this on everybody? A big win could be a modularized setuptools where you are able to only use the things you do want to use, e.g. - version specifications (not just the heuristics shipped with setuptools). - specification of dependencies. - resource management - a module system independent from any distribution specific stuff. - any other distribution specific stuff. The "Wouldn't it be nice if?" pacman (page 55) sounds like a nice idea, if it could just handle multiple repositories and one of them being the archive of the os distributor (iirc the java jsr277 proposed the use of multiple repositories, even in different formats). Matthias [1] http://mail.python.org/pipermail/distutils-sig/2008-September/010045.html From zooko at zooko.com Wed Sep 24 19:21:32 2008 From: zooko at zooko.com (zooko) Date: Wed, 24 Sep 2008 11:21:32 -0600 Subject: [Distutils] let's standardize dependency declaration without trying to agree on how to implement it In-Reply-To: <48D9DDE7.6010601@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <48D9DDE7.6010601@ar.media.kyoto-u.ac.jp> Message-ID: On Sep 24, 2008, at 0:27 AM, David Cournapeau wrote: > If you look at the ruby gem page, you have one link for a > specification; > I have not done it, and maybe I would realize I were wrong by > trying it, > but I got the impression I could generate gems myself from the > specification. Can I do that with eggs ? How about this: http://peak.telecommunity.com/DevCenter/EggFormats > There is also the problem that by making some things easy but > effectively "magic", when it breaks, you don't know how to fix. I agree that this is a problem. People interested in improving it should read Philip J. Eby's post "setuptools: past, present, future" from 2006: http://mail.python.org/pipermail/python-dev/2006-April/064145.html Since then we've made a great step forward by having the distutils in Python 2.5 and newer automatically produce .egg-info files. Then, we made another step forward when we persuaded Linux distributions like Debian and Red Hat to stop deleting those .egg- info files. ;-) In my opinion the next step forward at this layer of basic compatibility is to formalize setuptools's "requirements" syntax (mainly install_requires, but also setup_requires, test_requires, and extras_require) as a standard part of Python. Note that I am not saying anything about the implementation of how requirements get satisfied, which we've already failed to agree on for a Python standard, only that if developers want to write down "My package depends on package XYZ" in their package's meta-data, that they can do so in a single, standard syntax so that all of the new crop of packaging tools can read what they wrote. Of course, this standard syntax should be compatible with the most widely-used current implementation -- setuptools/easy_install. This is an opportunity to standardize some basic metadata, not to innovate and not to standardize anything harder and more implementation- specific than simple dependency declaration. Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From pje at telecommunity.com Wed Sep 24 19:28:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 13:28:20 -0400 Subject: [Distutils] Setup tools causes problems with orca finding its python packages In-Reply-To: <1222183475.24613.14.camel@localhost> References: <1222180342.11820.47.camel@localhost> <20080923144904.2CF343A40C2@sparrow.telecommunity.com> <1222183475.24613.14.camel@localhost> Message-ID: <20080924172714.9A8333A4127@sparrow.telecommunity.com> This should now be fixed in the trunk, branch, and the just-released 0.6c9 version. At 04:24 PM 9/23/2008 +0100, Michael Whapples wrote: >This seems to do it (I know that this is not the correct way of doing >things, but I have made the mistake): >As root install plasTeX (http://plastex.sf.net, also in pypi) from >source by using the setup.py script in it. This should add a plasTeX >directory to your site-packages directory. >Forget to uninstall it (IE. that source installation is still there when >you do the following). > >Running as root to perform the install, give the following command > >easy_install -Z plasTeX > >Now look in easy-install.pth in the site packages directory >(/usr/lib/python2.5/site-packages/easy-install.pth on my system), I >find /usr/lib/python2.5/site-packages listed there after the above >command. > >I don't believe this is specific to plasTeX but this is the one I >noticed it with. Also I know this isn't the correct way to do things, we >should remember to uninstall source installations first, but sometimes >things slip our minds. > >If plasTeX isn't installed in any form before installing with >easy_install this bug doesn't show, a subdirectory is created and the >subdirectory is added. > >Michael Whapples >On Tue, 2008-09-23 at 10:50 -0400, Phillip J. Eby wrote: > > At 03:32 PM 9/23/2008 +0100, Michael Whapples wrote: > > >I have noticed that easy_install has a easy-install.pth file which seems > > >to be adding the directories specified in it before any other python > > >package directories, including the environment variable PYTHONPATH. This > > >causes a problem with orca which uses a bash script to start itself, and > > >specifies where its python packages should be found by setting the > > >PYTHONPATH variable. This wouldn't cause problems except when a user has > > >more than one version of orca installed (eg. a stable version from the > > >distribution installed in the standard python site-packages directory > > >and one (SVN trunk version) in a custom directory). I found that > > >easy_install had added the standard python site-packages directory > > >(/usr/lib/python2.5/site-packages on my system) to easy-install.pth so > > >making it impossible to use my custom install of orca as the standard > > >location was first in sys.path. > > > > Eek. That sounds like a bug. Can you provide minimal steps to > > reproduce it? (i.e., getting a site directory to end up in > easy-install.pth) > > From pje at telecommunity.com Wed Sep 24 19:33:19 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 13:33:19 -0400 Subject: [Distutils] setuptools 0.6c9 released Message-ID: <20080924173212.D98E23A4128@sparrow.telecommunity.com> I've just cut a release of 0.6c9. Some outstanding bugs remain; see the tracker for details. A few (issues 16, 20, and 23) have candidate fixes in the trunk but have not been applied to the branch as yet. From pje at telecommunity.com Wed Sep 24 19:38:14 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 13:38:14 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.co m> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> Message-ID: <20080924173708.13DA93A4128@sparrow.telecommunity.com> At 01:32 PM 9/24/2008 +0200, Tarek Ziad? wrote: >Hi, > >I am launching a fork of setuptools. The name is "Distribute". (not >definitive) and will be community-driven > >This fork will remain 100% compatible with setuptools, and follow >closely setuptools evolutions. Unfortunately, the only way you can remain 100% compatible with code "in the field" is by calling your distribution "setuptools" also, or by having it also install a dummy setuptools egg or at least .egg-info. Otherwise, packages that declare a dependency to a specific setuptools version will cause it to be installed, thereby overwriting your package. There's also setup scripts that use ez_setup, which you could work around for easy_install scenarios with clever enough monkeypatching, but someone directly running a setup script would still break. If you're going to attempt this, I would consider very carefully how you will address these issues, as it seems to me that it would be very easy for someone to silently hose their system this way, and possibly quite difficult to find out what went wrong or how to fix it. From pje at telecommunity.com Wed Sep 24 19:39:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 13:39:01 -0400 Subject: [Distutils] Can we prevent setuptools from writing to stdout In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB > References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <94bdd2610809240539y7fbfb2b9uef1b22cc29270040@mail.gmail.com> <2FC790CB-212F-44F8-89B3-3DA9B59AE5DB@python.org> <94bdd2610809240620p1feb03acwc3c235b1204589cf@mail.gmail.com> <7F347D91614EBC48AA7540A2A76D3BB204483D90@MXCU10MX1.MSX.CIB> Message-ID: <20080924173755.3AC463A4128@sparrow.telecommunity.com> At 02:24 PM 9/24/2008 +0100, Fadhley Salim wrote: >Is it possible to prevent setuptools from writing anything to stdout? > >For various inconvenient reasons our package must install silently - >that means the installation process must not write to either stdout >or stderr. Other than hacking distutils / setuptools to ensure that >there's nothing which can write i've not yet been able to come up >with a reliable means to control output. Temporarily replace sys.stdout with another object, such as a StringIO. From pje at telecommunity.com Wed Sep 24 19:40:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 13:40:37 -0400 Subject: [Distutils] Importing non-default versions (was Re: Annoucing distribute project In-Reply-To: <18650.19399.50129.607737@gargle.gargle.HOWL> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> Message-ID: <20080924173933.19E1C3A4128@sparrow.telecommunity.com> At 04:16 PM 9/24/2008 +0200, Matthias Klose wrote: > - os distributors usually try to minimize the versions they include, > trying to just ship one version. This single version is installed > with --single-version-externally-managed, so that it can be > imported without any pkg_resouces magic and fiddling with pth > files. Unfortunately then it is not possible to use/import another > version using pkg_resources. Can we resolve this by simply documenting the __requires__ "feature"? From ziade.tarek at gmail.com Wed Sep 24 21:44:47 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 21:44:47 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080924173708.13DA93A4128@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> Message-ID: <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> On Wed, Sep 24, 2008 at 7:38 PM, Phillip J. Eby wrote: > At 01:32 PM 9/24/2008 +0200, Tarek Ziad? wrote: > >> Hi, >> >> I am launching a fork of setuptools. The name is "Distribute". (not >> definitive) and will be community-driven >> >> This fork will remain 100% compatible with setuptools, and follow closely >> setuptools evolutions. >> > > Unfortunately, the only way you can remain 100% compatible with code "in > the field" is by calling your distribution "setuptools" also, or by having > it also install a dummy setuptools egg or at least .egg-info. Otherwise, > packages that declare a dependency to a specific setuptools version will > cause it to be installed, thereby overwriting your package. There's also > setup scripts that use ez_setup, which you could work around for > easy_install scenarios with clever enough monkeypatching, but someone > directly running a setup script would still break. > > If you're going to attempt this, I would consider very carefully how you > will address these issues, as it seems to me that it would be very easy for > someone to silently hose their system this way, and possibly quite difficult > to find out what went wrong or how to fix it. Yup. I thought of different ways today, but trying to beat the main setuptools package installation is a non sense and will lead to problems. I think we will do like how you did to override distutils. It's OK for us to drop the 'setuptools' dependency in all our packages, and simply tell interested package maintainers to switch "setuptools" to "distribute" in their import line in setup.py files, and have a code base that is similar, besides the fact that it does a smart override of the commands, like you did, and a few other thing I still need to list. The only difference is that it will be run by a community, and I am pretty sure someone that has a bugfix or a feature available in Distribute wouldn't mind changing a simple import line and a dependency in his package and tell people to use it. I won't release the first version of Distribute friday since you have released 0.6c9, so the first release for Distribute will be in one month. A first sprint will probably be organized at the Plone Conference in two weeks on this topic. Best Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From marius at pov.lt Wed Sep 24 22:47:39 2008 From: marius at pov.lt (Marius Gedminas) Date: Wed, 24 Sep 2008 23:47:39 +0300 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> Message-ID: <20080924204739.GA29149@fridge.pov.lt> On Tue, Sep 23, 2008 at 09:24:00PM -0700, Kevin Teague wrote: > Or they can just use debian! Any debian developers out there up for > the task of packaging up the 1500+ odd packages released from the Zope > community? The SchoolTool guys made a tool and built .debs for all of Zope 3 that SchoolTool needs. The resulting packages are here: https://launchpad.net/~schooltool-owners/+archive Marius Gedminas -- Bumper sticker: Alcohol and calculus don't mix. Never drink and derive. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From ccomb at free.fr Wed Sep 24 22:54:36 2008 From: ccomb at free.fr (Christophe Combelles) Date: Wed, 24 Sep 2008 22:54:36 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> Message-ID: <48DAA90C.20504@free.fr> Tarek Ziad? a ?crit : > Hi, > > I am launching a fork of setuptools. The name is "Distribute". What about "distributing"? as with threading, processing, logging, ... string (mmh... no. Not string ;) > (not definitive) and will be community-driven Ah! this point is much more important than the name, that sounds good. > > This fork will remain 100% compatible with setuptools, and > follow closely setuptools evolutions. The best thing would be to have this kind of tool eventually integrated into Python! > > I won't release the first version of Distribute friday since you have > released 0.6c9, so the first release for Distribute will be in one 0.6c9 was urgently needed (thanks for the release...) because of svn 1.5 incompatibility, but there are a LOT of other improvements that are already wanted, such as support for other (d)vcs, an easy_uninstall, etc... I sincerely hope that things will speed-up with this fork. I'm volunteering to help on this effort. Christophe > month. A first sprint will probably be organized at the Plone Conference > in two weeks on this topic. > > Best Regards > Tarek > > > -- > Tarek Ziad? | Association AfPy | www.afpy.org > Blog FR | http://programmation-python.org > Blog EN | http://tarekziade.wordpress.com/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Wed Sep 24 23:34:47 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 17:34:47 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <48DAA90C.20504@free.fr> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> Message-ID: <20080924213342.612CC3A4127@sparrow.telecommunity.com> At 10:54 PM 9/24/2008 +0200, Christophe Combelles wrote: >there are a LOT of other improvements that are already wanted, such >as support for other (d)vcs, See: http://pypi.python.org/pypi?:action=browse&c=524 and: http://pypi.python.org/pypi?%3Aaction=search&term=setuptools&submit=search not to mention: http://bugs.python.org/setuptools/issue42 ...especially my comments on what the existing patch still needs in order to be accepted. >an easy_uninstall, etc... http://bugs.python.org/setuptools/issue21 is tracking this, but I have not reviewed the submitted patch in any detail. It would be much better to have a design proposal to review first. Note, by the way, that the reason most of the items listed as "feature" or "wish" on the tracker are in those statuses are because nobody has proposed a design for critique. Usually, it's just wishes for magic ponies (e.g. uninstall, post-install hooks, etc.) without any discussion of how to deal with the edge cases, interactions, or other negative consequences of said features. (magic pony manure?) > I sincerely hope that things will speed-up with this fork. I imagine they might speed up, but likely at the cost of stability. If anybody knew enough to be able to add these features in a safe way, then they knew enough to be able to contribute patches to setuptools (after first proposing how they would handle all the nasty edge cases). In principle, I have no problems with a fork. In practice, however, I would expect significant new features (e.g. uninstall) to be accompanied by a significant decrease in stability. In other words, I don't recommend mixing fork goals. If the goal is merely to have more-frequent releases of setuptools, it would be better to have a snapshot facility for same, and release dev-tagged eggs. Conversely, if the goal is to prototype new features, then it doesn't make a lot of sense to advertise it as a stable or up-to-date setuptools. So, my personal hope is that the persons doing the fork will make it clear to their users which it is: unstable feature prototype, or simple release snapshots? (Where the latter could just as easily be automated.) From zooko at zooko.com Wed Sep 24 23:43:39 2008 From: zooko at zooko.com (zooko) Date: Wed, 24 Sep 2008 15:43:39 -0600 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080924204739.GA29149@fridge.pov.lt> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <20080924204739.GA29149@fridge.pov.lt> Message-ID: On Sep 24, 2008, at 14:47 PM, Marius Gedminas wrote: > On Tue, Sep 23, 2008 at 09:24:00PM -0700, Kevin Teague wrote: >> Or they can just use debian! Any debian developers out there up for >> the task of packaging up the 1500+ odd packages released from the >> Zope >> community? > > The SchoolTool guys made a tool and built .debs for all of Zope 3 that > SchoolTool needs. The resulting packages are here: > https://launchpad.net/~schooltool-owners/+archive I used the stdeb tool on several Python packages that I maintain and it worked to produce .deb's from Python source distributions. http://stdeb.python-hosting.com Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From ziade.tarek at gmail.com Wed Sep 24 23:56:23 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Sep 2008 23:56:23 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080924213342.612CC3A4127@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> Message-ID: <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> On Wed, Sep 24, 2008 at 11:34 PM, Phillip J. Eby wrote: > In other words, I don't recommend mixing fork goals. If the goal is merely > to have more-frequent releases of setuptools, it would be better to have a > snapshot facility for same, and release dev-tagged eggs. Conversely, if the > goal is to prototype new features, then it doesn't make a lot of sense to > advertise it as a stable or up-to-date setuptools. > > So, my personal hope is that the persons doing the fork will make it clear > to their users which it is: unstable feature prototype, or simple release > snapshots? (Where the latter could just as easily be automated.) The point is that the stability you are claiming means that the whole community depends on your actions. The fact that you were busy lately made things even slower. That is my whole point. Even if you set up a snapshot facility, the only person that is able to make changes in setuptools is you and only you. Distribute will be driven by more people, and its trunk will be more unstable for sure. But it think we can do some valuable test-driven work all together, and our releases can be quite good. So I'll make it clear to our users: it will be a project with stable releases, and unstable features in branches, and a trunk to merge everything, like all the open source project out there. Best Regards Tarek > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ccomb at free.fr Thu Sep 25 00:09:56 2008 From: ccomb at free.fr (Christophe Combelles) Date: Thu, 25 Sep 2008 00:09:56 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080924213342.612CC3A4127@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> Message-ID: <48DABAB4.3060407@free.fr> Phillip J. Eby a ?crit : > At 10:54 PM 9/24/2008 +0200, Christophe Combelles wrote: >> there are a LOT of other improvements that are already wanted, such as >> support for other (d)vcs, > > See: > > http://pypi.python.org/pypi?:action=browse&c=524 > > and: > > > http://pypi.python.org/pypi?%3Aaction=search&term=setuptools&submit=search > > not to mention: > > http://bugs.python.org/setuptools/issue42 > > ...especially my comments on what the existing patch still needs in > order to be accepted. > > >> an easy_uninstall, etc... > > http://bugs.python.org/setuptools/issue21 is tracking this, but I have > not reviewed the submitted patch in any detail. It would be much better > to have a design proposal to review first. > > Note, by the way, that the reason most of the items listed as "feature" > or "wish" on the tracker are in those statuses are because nobody has > proposed a design for critique. Usually, it's just wishes for magic > ponies (e.g. uninstall, post-install hooks, etc.) without any discussion > of how to deal with the edge cases, interactions, or other negative > consequences of said features. (magic pony manure?) > > >> I sincerely hope that things will speed-up with this fork. > > I imagine they might speed up, but likely at the cost of stability. If > anybody knew enough to be able to add these features in a safe way, then > they knew enough to be able to contribute patches to setuptools (after > first proposing how they would handle all the nasty edge cases). > > In principle, I have no problems with a fork. In practice, however, I > would expect significant new features (e.g. uninstall) to be accompanied > by a significant decrease in stability. The decrease in stability may be significant, indeed but releasing more often means the code will be much more tested, then fixed much quicker. > > In other words, I don't recommend mixing fork goals. If the goal is > merely to have more-frequent releases of setuptools, it would be better > to have a snapshot facility for same, and release dev-tagged eggs. > Conversely, if the goal is to prototype new features, then it doesn't > make a lot of sense to advertise it as a stable or up-to-date setuptools. I'm not sure about the difference between a dev-tagged egg and an eternal 0.x candidate release. Was setuptools 0.6c8 really stable and up-to-date? > > So, my personal hope is that the persons doing the fork will make it > clear to their users which it is: unstable feature prototype, or simple > release snapshots? (Where the latter could just as easily be automated.) > > > From pje at telecommunity.com Thu Sep 25 00:36:03 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 24 Sep 2008 18:36:03 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.co m> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> Message-ID: <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> At 11:56 PM 9/24/2008 +0200, Tarek Ziad? wrote: >On Wed, Sep 24, 2008 at 11:34 PM, Phillip J. Eby ><pje at telecommunity.com> wrote: >In other words, I don't recommend mixing fork goals. If the goal is >merely to have more-frequent releases of setuptools, it would be >better to have a snapshot facility for same, and release dev-tagged >eggs. Conversely, if the goal is to prototype new features, then it >doesn't make a lot of sense to advertise it as a stable or >up-to-date setuptools. > >So, my personal hope is that the persons doing the fork will make it >clear to their users which it is: unstable feature prototype, or >simple release snapshots? (Where the latter could just as easily be >automated.) > >The point is that the stability you are claiming means that the >whole community depends on your actions. The fact that you were busy >lately made things even slower. Actually, it was the community that slowed down 0.6c9, in that I would have released it last month, except that certain contributors were not ready. As it is, I ended up giving up waiting for one of them to come through -- ironically, it was someone who used to complain a lot about delays. ;-) > That is my whole point. Even if you set up a snapshot facility, > the only person that is able to make changes in setuptools is you > and only you. Sigh. That's simply not true, as has been discussed here before -- more than once. If you check the commit logs, you'll see that other people HAVE made changes to setuptools, and are still quite able to do so. Jim Fulton is already a "blessed" committer, and I'm quite close to blessing P. Jenvey (if he wants it). And, when somebody else steps up with a series of quality patches (backed with pre-reviewed specs in the case of features with significant interactions), then that person can get blessed too. >So I'll make it clear to our users: it will be a project with stable >releases, and unstable features in branches, and a trunk to merge >everything, like all the open source project out there. Great. Just please make sure that you ask your contributors and users NOT to expect support from me, and to not use the setuptools bug tracker for non-setuptools issues. It's not reasonable to expect me to support the fork in addition to the original version. (I'm also somewhat concerned about what issue the OS vendors will blame me or "Python eggs" in general for next, even though I've got nothing to do with the fork.) From ziade.tarek at gmail.com Thu Sep 25 07:52:20 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 07:52:20 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> Message-ID: <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> On Thu, Sep 25, 2008 at 12:36 AM, Phillip J. Eby wrote: > At 11:56 PM 9/24/2008 +0200, Tarek Ziad? wrote: > >> On Wed, Sep 24, 2008 at 11:34 PM, Phillip J. Eby <> pje at telecommunity.com>pje at telecommunity.com> wrote: >> In other words, I don't recommend mixing fork goals. If the goal is >> merely to have more-frequent releases of setuptools, it would be better to >> have a snapshot facility for same, and release dev-tagged eggs. Conversely, >> if the goal is to prototype new features, then it doesn't make a lot of >> sense to advertise it as a stable or up-to-date setuptools. >> >> So, my personal hope is that the persons doing the fork will make it clear >> to their users which it is: unstable feature prototype, or simple release >> snapshots? (Where the latter could just as easily be automated.) >> >> The point is that the stability you are claiming means that the whole >> community depends on your actions. The fact that you were busy >> lately made things even slower. >> > > Actually, it was the community that slowed down 0.6c9, in that I would have > released it last month, except that certain contributors were not ready. As > it is, I ended up giving up waiting for one of them to come through -- > ironically, it was someone who used to complain a lot about delays. ;-) > > > That is my whole point. Even if you set up a snapshot facility, the only >> person that is able to make changes in setuptools is you and only you. >> > > Sigh. That's simply not true, as has been discussed here before -- more > than once. If you check the commit logs, you'll see that other people HAVE > made changes to setuptools, and are still quite able to do so. Jim Fulton > is already a "blessed" committer, and I'm quite close to blessing P. Jenvey > (if he wants it). > > And, when somebody else steps up with a series of quality patches (backed > with pre-reviewed specs in the case of features with significant > interactions), then that person can get blessed too. iirc Jim just worked on a particular point back then. Anyway, that sounds great indeed. But since setuptools is not on the top of your priority sometimes (you said it wasn't, and we could see it wasn't), are those people blessed to release it as well ? I mean, if we have a clear roadmap of the next releases of setuptools, with dates, and if you are unable to release it yourself, can we count on them to do so ? If so I'd be more than happy to stop that fork, and to continue happily to work in the bug tracker if I have a visible roadmap. > > > So I'll make it clear to our users: it will be a project with stable >> releases, and unstable features in branches, and a trunk to merge >> everything, like all the open source project out there. >> > > Great. Just please make sure that you ask your contributors and users NOT > to expect support from me, and to not use the setuptools bug tracker for > non-setuptools issues. It's not reasonable to expect me to support the fork > in addition to the original version. > > (I'm also somewhat concerned about what issue the OS vendors will blame me > or "Python eggs" in general for next, even though I've got nothing to do > with the fork.) I understand, and I would prefer not having to fork. But I don't want to be locked on this particular area because a package depends on some persons that are busy elsewhere. imho setuptools became a sensible package used by many developers; so unless it opens a bit I don't see how this is going to work. It really need to be boosted at this point. Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From doko at cs.tu-berlin.de Thu Sep 25 10:05:47 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Thu, 25 Sep 2008 10:05:47 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <20080924204739.GA29149@fridge.pov.lt> Message-ID: <18651.18011.880115.616165@gargle.gargle.HOWL> zooko writes: > > On Sep 24, 2008, at 14:47 PM, Marius Gedminas wrote: > > > On Tue, Sep 23, 2008 at 09:24:00PM -0700, Kevin Teague wrote: > >> Or they can just use debian! Any debian developers out there up for > >> the task of packaging up the 1500+ odd packages released from the > >> Zope > >> community? > > > > The SchoolTool guys made a tool and built .debs for all of Zope 3 that > > SchoolTool needs. The resulting packages are here: > > https://launchpad.net/~schooltool-owners/+archive yes, and we are evaluating how this is maintainable, seen from the packaging view. > I used the stdeb tool on several Python packages that I maintain and > it worked to produce .deb's from Python source distributions. > > http://stdeb.python-hosting.com see the todo list why this cannot be yet used for packages that we want to upload to debian/ubuntu. From jeff at taupro.com Thu Sep 25 11:14:10 2008 From: jeff at taupro.com (Jeff Rush) Date: Thu, 25 Sep 2008 04:14:10 -0500 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080924213342.612CC3A4127@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> Message-ID: <48DB5662.7050501@taupro.com> Phillip J. Eby wrote: > At 10:54 PM 9/24/2008 +0200, Christophe Combelles wrote: >> there are a LOT of other improvements that are already wanted, such as >> support for other (d)vcs, > > Note, by the way, that the reason most of the items listed as "feature" > or "wish" on the tracker are in those statuses are because nobody has > proposed a design for critique. Usually, it's just wishes for magic > ponies (e.g. uninstall, post-install hooks, etc.) without any discussion > of how to deal with the edge cases, interactions, or other negative > consequences of said features. (magic pony manure?) > >> I sincerely hope that things will speed-up with this fork. > > I imagine they might speed up, but likely at the cost of stability. If > anybody knew enough to be able to add these features in a safe way, then > they knew enough to be able to contribute patches to setuptools (after > first proposing how they would handle all the nasty edge cases). I agree, in that we must not add features without careful thought as to their impact and cross-platform support. Python is a conservative community and we've seen value in the role of an gatekeeper who sees the bigger picture, as Guido and others for other projects. Of course, a gatekeeper who is too-strict isn't good either - there must be balance. But to the distribute project, because of the nature of standardizing package technology, running this is a huge responsibility. Please do not start adding all features asked for without considering their tradeoffs. One issue with something like setuptools is that packaging APIs get immortalized in setup.py files that are not easy to go back and change, so the APIs must carry heavy backward compatibility baggage. You may want to consider using PEPs as a basis for discussing change requests. They're not just for the core Python language. -Jeff From david at ar.media.kyoto-u.ac.jp Thu Sep 25 14:31:45 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 25 Sep 2008 21:31:45 +0900 Subject: [Distutils] let's standardize dependency declaration without trying to agree on how to implement it In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <48D9DDE7.6010601@ar.media.kyoto-u.ac.jp> Message-ID: <48DB84B1.10406@ar.media.kyoto-u.ac.jp> zooko wrote: > On Sep 24, 2008, at 0:27 AM, David Cournapeau wrote: > > > How about this: > > http://peak.telecommunity.com/DevCenter/EggFormats I guess we don't have the same meaning when speaking about specification. A reference which states at the beginning "Note, however, that these are all internal implementation details and are therefore subject to change; stick to the published API if you don't want to be responsible for keeping your code from breaking when setuptools changes. You have been warned." Is not a specification in my mind. This is pervasive in the distutils/setuptools world BTW: everything is defined by implementation, you don't know what is implementation detail and what is the API (My first contact with distutils was to plug a scons command to use scons within distutils for the numpy project, and I was horrified by the code of distutils. The only way to understand what's going on is to run it; I can't say setuptools is much better in that departement, but since a setuptools requirement was to extend distutils, I don't blame setuptools for this) > I agree that this is a problem. People interested in improving it > should read Philip J. Eby's post "setuptools: past, present, future" > from 2006: > > http://mail.python.org/pipermail/python-dev/2006-April/064145.html > > Since then we've made a great step forward by having the distutils in > Python 2.5 and newer automatically produce .egg-info files. > > Then, we made another step forward when we persuaded Linux > distributions like Debian and Red Hat to stop deleting those .egg-info > files. ;-) I hope that with the fork, we will be able to deal better with the situation. To me, the biggest problem of setuptools is that it is a big mud of code, with things which are vastly different in purpose. There should be a submodule for building eggs, a submodule to deal with dependencies, a submodule for the distutils extensions, etc... I want to be able to import setuptools "build an egg" module without using setuptools at all otherwise, so that I can build an egg in e.g. scons. I want to be able to use setuptools capabilities for dependencies to get a list of all the dependencies without using setuptools at all otherwise. cheers, David From david at ar.media.kyoto-u.ac.jp Thu Sep 25 16:13:26 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 25 Sep 2008 23:13:26 +0900 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <18650.28513.573244.313581@gargle.gargle.HOWL> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <18650.28513.573244.313581@gargle.gargle.HOWL> Message-ID: <48DB9C86.6040909@ar.media.kyoto-u.ac.jp> Matthias Klose wrote: > Your paper gives a nice overview of the shortcomings of each of the > build/distribution systems. From an (os) distributors point of view I > would add som things (also posted in [1]). [snip] > I think it would be worthwhile to have one location to put those requirements. IMHO, taking into account OS packagers (Linux and other) in the development of a package tool for python is essential. Tarek, what is the best place to put this ? Somewhere on launchpad ? Or somewhere else ? cheers, David From ziade.tarek at gmail.com Thu Sep 25 16:52:00 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 16:52:00 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <48DB9C86.6040909@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <18650.28513.573244.313581@gargle.gargle.HOWL> <48DB9C86.6040909@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610809250752y6e8d69a3t9642a6fefb96fdbd@mail.gmail.com> On Thu, Sep 25, 2008 at 4:13 PM, David Cournapeau < david at ar.media.kyoto-u.ac.jp> wrote: > Matthias Klose wrote: > > Your paper gives a nice overview of the shortcomings of each of the > > build/distribution systems. From an (os) distributors point of view I > > would add som things (also posted in [1]). [snip] > > > > I think it would be worthwhile to have one location to put those > requirements. IMHO, taking into account OS packagers (Linux and other) > in the development of a package tool for python is essential. > > Tarek, what is the best place to put this ? Somewhere on launchpad ? Or > somewhere else ? > > cheers, > > David I think the best place would be in python.org wiki itself, As Jeff Rush mentioned earlier, the best way to think about all requirements would be to write a series of PEP. So I think this would be a great starting point in Python.org wiki, I am preparing write now a few pages so we can gather all the info, i'll give you the link when it is ready Tarek > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Thu Sep 25 17:01:23 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 17:01:23 +0200 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <94bdd2610809250752y6e8d69a3t9642a6fefb96fdbd@mail.gmail.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <18650.28513.573244.313581@gargle.gargle.HOWL> <48DB9C86.6040909@ar.media.kyoto-u.ac.jp> <94bdd2610809250752y6e8d69a3t9642a6fefb96fdbd@mail.gmail.com> Message-ID: <94bdd2610809250801x50ff09cevd6756eefb4534cbf@mail.gmail.com> On Thu, Sep 25, 2008 at 4:52 PM, Tarek Ziad? wrote: > > > On Thu, Sep 25, 2008 at 4:13 PM, David Cournapeau < > david at ar.media.kyoto-u.ac.jp> wrote: > >> Matthias Klose wrote: >> > Your paper gives a nice overview of the shortcomings of each of the >> > build/distribution systems. From an (os) distributors point of view I >> > would add som things (also posted in [1]). [snip] >> > >> >> I think it would be worthwhile to have one location to put those >> requirements. IMHO, taking into account OS packagers (Linux and other) >> in the development of a package tool for python is essential. >> >> Tarek, what is the best place to put this ? Somewhere on launchpad ? Or >> somewhere else ? >> >> cheers, >> >> David > > > I think the best place would be in python.org wiki itself, > > As Jeff Rush mentioned earlier, the best way to think about all > requirements would be to write > a series of PEP. > > So I think this would be a great starting point in Python.org wiki, > > I am preparing write now a few pages so we can gather all the info, i'll > give you the link when it is ready > > Tarek > Alright, I have just started it here: http://wiki.python.org/moin/Distribute I guess we can gather all documentation there. On my side I will write down a wrapup on what has been said, and what various people are expecting from such a project, and how it can be organized, etc. Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Thu Sep 25 17:37:07 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 26 Sep 2008 00:37:07 +0900 Subject: [Distutils] Annoucing distribute project In-Reply-To: <18650.19399.50129.607737@gargle.gargle.HOWL> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> Message-ID: <48DBB023.2030007@ar.media.kyoto-u.ac.jp> Matthias Klose wrote: > > - setuptools has the narrow minded view of a python package being > contained in a single directory, which doesn't fit well when you > do have common locations for include or doc files. Does the fork > accept patches to change such limitations and allowing FHS > compliant packages? Being self-contained is also a feature. People who package softwares outside distributions like this, as you are surely aware. Personally, I don't like that setuptools broke distutils install either (I prefer to manage my packages with stow, because setuptools broke too many times my setup for unknown reasons). There should be the possibility to do both kind of installs (self-contained, or FHS compliant), but this is not so much a setuptools issue as a distutils issue, isn't it ? Dealing with this in distutils will be no fun, though... cheers, David From ziade.tarek at gmail.com Thu Sep 25 18:29:42 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 18:29:42 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <48DBB023.2030007@ar.media.kyoto-u.ac.jp> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> <48DBB023.2030007@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610809250929x733efe36ldb5692f789c6e3fa@mail.gmail.com> On Thu, Sep 25, 2008 at 5:37 PM, David Cournapeau < david at ar.media.kyoto-u.ac.jp> wrote: > Matthias Klose wrote: > > > > - setuptools has the narrow minded view of a python package being > > contained in a single directory, which doesn't fit well when you > > do have common locations for include or doc files. Does the fork > > accept patches to change such limitations and allowing FHS > > compliant packages? > > Being self-contained is also a feature. People who package softwares > outside distributions like this, as you are surely aware. Personally, I > don't like that setuptools broke distutils install either (I prefer to > manage my packages with stow, because setuptools broke too many times my > setup for unknown reasons). > > There should be the possibility to do both kind of installs > (self-contained, or FHS compliant), but this is not so much a setuptools > issue as a distutils issue, isn't it ? Dealing with this in distutils > will be no fun, though... > Well, as long as things are clearly defined in the package, I guess FHS compliant package could be built with the same source tree. We could even install a distribution the FHS way or the self-contained way, as long as the tool knows what to put where. But that is already the case, a bit: For instance we have bdist_rpm, that builds rpms by mapping distutils metadata to rpm ones, The question is: starting with the current MetaData what would you miss to do a FHS installation ? Take a look at http://www.python.org/dev/peps/pep-0345/ > > cheers, > > David > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Thu Sep 25 18:31:40 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 25 Sep 2008 12:31:40 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.co m> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> Message-ID: <20080925163034.208613A4072@sparrow.telecommunity.com> At 07:52 AM 9/25/2008 +0200, Tarek Ziad? wrote: >Anyway, that sounds great indeed. But since setuptools is not on the >top of your priority sometimes (you said it wasn't, and we could see >it wasn't), are those people blessed to release it as well ? I don't see a problem with that, although the actual release process itself is nearly 100% automated. (See the 'release.sh' file.) Assuming that there is a releasable distribution in SVN, it takes maybe 15 minutes for me to cut a release and bump the version, if that long. (And it could probably be further automated.) By the way, you'll notice if you inspect the distutils-sig archives carefully, I'm *extremely* responsive when people post questions or report bugs that haven't been previously answered or fixed. I simply don't have a lot of time to drive new development. There are few people qualified to do the design vetting and testing that needs to happen for major new features to be added atop the existing code base, and that's 100% independent of any forking. If the persons existed who could actually do this, there is no technical reason why they couldn't have been doing it prior to the fork. (Which leads me to believe that they don't exist or have the time, and thus, the fork will not actually help anything.) Personally, I would rather see not a "fork", but development of an entirely new disutils replacement, and then not worry at all about backward compatibility at the setup.py level. Apart from the pkg_resources module, I would treat setuptools and distutils as legacy infrastructure, and explore better ways to build and manage distributions (including eggs). If I had the time, that's what I'd be doing. >I mean, if we have a clear roadmap of the next releases of >setuptools, with dates, and if you are unable to release it >yourself, can we count on them to do so ? If so I'd be more than >happy to stop that fork, and to continue happily to work in the bug >tracker if I have a visible roadmap. Feel free to create such a roadmap; as you can see, I do have time to answer a few emails and give feedback. And if somebody responsible wants to take over the job of deciding whether a release is "ready", I can certainly pull the trigger. The bottleneck is not me per se; it's having qualified parties. In the long run, having more tests would help a lot with that, as I've said before. In the short run, it's people with strong multi-platform, multi-distribution, multi-project, multi-Python-version experience that are sorely lacking in the patch contribution, vetting, and design departments. And right now, the only way for me to evaluate such contributors is on the basis of their submitted patches and design proposals... which are few and far between. Historically, most of the patches I get are poorly thought-out even on the level of what the patch does for that one person; i.e., there will be cases that that very person will have problems with, let alone what problems other people will have. This does not give me great hope for the viability of a fork whose expressed purpose is to accept more patches, faster. :) (This situation is due largely to the flexibility of distutils, by the way. The distutils accepts an utterly ridiculous number of possible source trees, for example. It also supports a ridiculous number of installation targets, formats, etc. On the other hand, this same flexibility also makes it virtually impossible for a competitor to "take over the market" unless it provides near-100% backward compatibility, as setuptools does.) From lists at zopyx.com Thu Sep 25 18:46:20 2008 From: lists at zopyx.com (Andreas Jung) Date: Thu, 25 Sep 2008 18:46:20 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080925163034.208613A4072@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> <20080925163034.208613A4072@sparrow.telecommunity.com> Message-ID: --On 25. September 2008 12:31:40 -0400 "Phillip J. Eby" wrote: > Personally, I would rather see not a "fork", but development of an > entirely new disutils replacement, and then not worry at all about > backward compatibility at the setup.py level. Apart from the > pkg_resources module, I would treat setuptools and distutils as legacy > infrastructure, and explore better ways to build and manage distributions > (including eggs). We can not restart with something completely different or new. Such an approach would be totally unacceptable due to the huge amount of existing modules having switched to setuptools. Setuptools or an improved codefork must be the tools for managing distibutions in Python. -aj -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: From david at ar.media.kyoto-u.ac.jp Thu Sep 25 18:39:12 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 26 Sep 2008 01:39:12 +0900 Subject: [Distutils] Annoucing distribute project In-Reply-To: <94bdd2610809250929x733efe36ldb5692f789c6e3fa@mail.gmail.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> <48DBB023.2030007@ar.media.kyoto-u.ac.jp> <94bdd2610809250929x733efe36ldb5692f789c6e3fa@mail.gmail.com> Message-ID: <48DBBEB0.5000507@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > > Well, as long as things are clearly defined in the package, I guess > FHS compliant > package could be built with the same source tree. > > We could even install a distribution the FHS way or the self-contained > way, > as long as the tool knows what to put where. Yes, in theory, it is easy: just set which kind of files are put where, like e.g. autotools (datadir, libdir, etc...). In practice, I am not sure it will so easy, because distutils is so badly "designed". It breaks in so many cases, because you never know what is part of the specification and what is not. Will you break something if you change a given directory ? You can't know, and testing on one platform is not enough because distutils authors thought it was a great idea to dynamically defined most attributes of most classes, with the consequence that some attributes exist in some cases only, on some platforms only. > But that is already the case, a bit: > For instance we have bdist_rpm, that builds rpms by mapping distutils > metadata > to rpm ones, You can't be general when talking about distutils. All the pain come from implementation details and undocumented features. I could spend one hour listing all the sillyness in distutils, none of them being present in any other build system I have ever encountered. Again, other people know distutils much better than me and may disagree with me, or maybe rightfully notice my lack of experience on that codebase, but from my own work on it - which while not complicated was not trivial either - I got the conviction that any fundamental change wrt distutils will be extremely painful to do if not impossible in a backward compatible way. I was hoping that a distutils replacement would come for Py3k, actually :) cheers, David From jim at zope.com Thu Sep 25 18:55:36 2008 From: jim at zope.com (Jim Fulton) Date: Thu, 25 Sep 2008 12:55:36 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> <20080925163034.208613A4072@sparrow.telecommunity.com> Message-ID: <27C55E67-E628-460F-8BCA-0D85DCFDC7AF@zope.com> On Sep 25, 2008, at 12:46 PM, Andreas Jung wrote: > > > --On 25. September 2008 12:31:40 -0400 "Phillip J. Eby" > wrote: > > >> Personally, I would rather see not a "fork", but development of an >> entirely new disutils replacement, and then not worry at all about >> backward compatibility at the setup.py level. Apart from the >> pkg_resources module, I would treat setuptools and distutils as >> legacy >> infrastructure, and explore better ways to build and manage >> distributions >> (including eggs). > > We can not restart with something completely different or new. Such > an approach would be totally unacceptable due to the huge amount of > existing modules having switched to setuptools. Setuptools or an > improved codefork must be the tools for managing distibutions in > Python. I would be willing to start with something new if and only if it was adopted as part of Python. Jim -- Jim Fulton Zope Corporation From ziade.tarek at gmail.com Thu Sep 25 18:58:09 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 18:58:09 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <20080925163034.208613A4072@sparrow.telecommunity.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> <20080925163034.208613A4072@sparrow.telecommunity.com> Message-ID: <94bdd2610809250958u23f4ac95hb3f319c692a9bec8@mail.gmail.com> On Thu, Sep 25, 2008 at 6:31 PM, Phillip J. Eby wrote: > At 07:52 AM 9/25/2008 +0200, Tarek Ziad? wrote: > >> Anyway, that sounds great indeed. But since setuptools is not on the top >> of your priority sometimes (you said it wasn't, and we could see it wasn't), >> are those people blessed to release it as well ? >> > > I don't see a problem with that, although the actual release process itself > is nearly 100% automated. (See the 'release.sh' file.) Assuming that there > is a releasable distribution in SVN, it takes maybe 15 minutes for me to cut > a release and bump the version, if that long. (And it could probably be > further automated.) > > By the way, you'll notice if you inspect the distutils-sig archives > carefully, I'm *extremely* responsive when people post questions or report > bugs that haven't been previously answered or fixed. I simply don't have a > lot of time to drive new development. > > There are few people qualified to do the design vetting and testing that > needs to happen for major new features to be added atop the existing code > base, and that's 100% independent of any forking. If the persons existed > who could actually do this, there is no technical reason why they couldn't > have been doing it prior to the fork. (Which leads me to believe that they > don't exist or have the time, and thus, the fork will not actually help > anything.) > > Personally, I would rather see not a "fork", but development of an entirely > new disutils replacement, and then not worry at all about backward > compatibility at the setup.py level. Apart from the pkg_resources module, I > would treat setuptools and distutils as legacy infrastructure, and explore > better ways to build and manage distributions (including eggs). > > If I had the time, that's what I'd be doing. > That makes a lot of sense, working on a distutils replacement would be more constructive you are right. But we would then need to work hard to migrate existing projects. This is quite scary. > > > > I mean, if we have a clear roadmap of the next releases of setuptools, >> with dates, and if you are unable to release it yourself, can we count on >> them to do so ? If so I'd be more than happy to stop that fork, and to >> continue happily to work in the bug tracker if I have a visible roadmap. >> > > Feel free to create such a roadmap; as you can see, I do have time to > answer a few emails and give feedback. And if somebody responsible wants to > take over the job of deciding whether a release is "ready", I can certainly > pull the trigger. Ok That sounds good, I will try to make that roadmap, with what we feel is important. > > > The bottleneck is not me per se; it's having qualified parties. In the > long run, having more tests would help a lot with that, as I've said before. > In the short run, it's people with strong multi-platform, > multi-distribution, multi-project, multi-Python-version experience that are > sorely lacking in the patch contribution, vetting, and design departments. I think those people exists, but they are lacking of visibility in setuptools projects, so they can't really contribute even if they would like to. We don't really know where setuptools is going at this point, I mean, there are mails like this one: http://mail.python.org/pipermail/python-dev/2006-April/064145.html but a few up-to-date web pages would help a lot on this. I can try to work on something besides the roadmap stuff, on the python wiki. That's what we have started to initiate in our 'fork' in fact. > > > And right now, the only way for me to evaluate such contributors is on the > basis of their submitted patches and design proposals... which are few and > far between. Historically, most of the patches I get are poorly thought-out > even on the level of what the patch does for that one person; i.e., there > will be cases that that very person will have problems with, let alone what > problems other people will have. This does not give me great hope for the > viability of a fork whose expressed purpose is to accept more patches, > faster. :) My "fork" is already starting to work if you think about it. > > > (This situation is due largely to the flexibility of distutils, by the way. > The distutils accepts an utterly ridiculous number of possible source > trees, for example. It also supports a ridiculous number of installation > targets, formats, etc. On the other hand, this same flexibility also makes > it virtually impossible for a competitor to "take over the market" unless it > provides near-100% backward compatibility, as setuptools does.) > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Thu Sep 25 19:16:30 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 19:16:30 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <27C55E67-E628-460F-8BCA-0D85DCFDC7AF@zope.com> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> <20080925163034.208613A4072@sparrow.telecommunity.com> <27C55E67-E628-460F-8BCA-0D85DCFDC7AF@zope.com> Message-ID: <94bdd2610809251016r100b551eqceb4e6e1e3c22e8@mail.gmail.com> On Thu, Sep 25, 2008 at 6:55 PM, Jim Fulton wrote: > > On Sep 25, 2008, at 12:46 PM, Andreas Jung wrote: > > >> >> --On 25. September 2008 12:31:40 -0400 "Phillip J. Eby" < >> pje at telecommunity.com> wrote: >> >> >> Personally, I would rather see not a "fork", but development of an >>> entirely new disutils replacement, and then not worry at all about >>> backward compatibility at the setup.py level. Apart from the >>> pkg_resources module, I would treat setuptools and distutils as legacy >>> infrastructure, and explore better ways to build and manage distributions >>> (including eggs). >>> >> >> We can not restart with something completely different or new. Such an >> approach would be totally unacceptable due to the huge amount of existing >> modules having switched to setuptools. Setuptools or an improved codefork >> must be the tools for managing distibutions in Python. >> > > I would be willing to start with something new if and only if it was > adopted as part of Python. Cool ! I think the best way to make sure it is adopted in Python is to include in the thaught of this new thing all people involved in distribution tools in the community. They ain't so many so I am pretty sure it can happen. It can't be rejected then. > > > Jim > > -- > Jim Fulton > Zope Corporation > > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Thu Sep 25 19:20:09 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 25 Sep 2008 13:20:09 -0400 Subject: [Distutils] Annoucing distribute project In-Reply-To: References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <20080924173708.13DA93A4128@sparrow.telecommunity.com> <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8@mail.gmail.com> <48DAA90C.20504@free.fr> <20080924213342.612CC3A4127@sparrow.telecommunity.com> <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90@mail.gmail.com> <20080924223457.2BD7E3A4127@sparrow.telecommunity.com> <94bdd2610809242252j131d9fbcyd3e629667b455d29@mail.gmail.com> <20080925163034.208613A4072@sparrow.telecommunity.com> Message-ID: <20080925171903.B907C3A4072@sparrow.telecommunity.com> At 06:46 PM 9/25/2008 +0200, Andreas Jung wrote: >We can not restart with something completely different or new. Such >an approach would be totally unacceptable due to the huge amount of >existing modules having switched to setuptools. Setuptools or an >improved codefork must be the tools for managing distibutions in Python. Yes and no. Creating a new approach will not cause setuptools to instantly disappear. And assuming the new approach supports only a subset of what setuptools/distutils does, then it should be relatively simple to generate a setuptools-compatible setup.py in distributions created by this other tool, thereby allowing the existing infrastructure to be used. On the flip side, there is no reason why the new tool(s) cannot *use* setuptools in order to cope with existing packages, as buildout, pyinstaller, and virtualenv already have shown. At that point, it would remain only to declare distutils and setuptools deprecated and legacy, and provide a clear migration path to the new tool. The biggest bottlenecks in setuptools development are backward compatibility, testing, and the distutils' excessive flexibility... not to mention the nature of the codebase itself. A new development would allow all of those issues to be addressed at the same time. From ziade.tarek at gmail.com Thu Sep 25 19:20:16 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 19:20:16 +0200 Subject: [Distutils] Annoucing distribute project In-Reply-To: <48DBBEB0.5000507@ar.media.kyoto-u.ac.jp> References: <94bdd2610809240432i307ed937rbfb1fd063cec915b@mail.gmail.com> <18650.10541.768378.823880@gargle.gargle.HOWL> <5e1183fa0809240532s31bc6c59k24075f9a483949e3@mail.gmail.com> <18650.19399.50129.607737@gargle.gargle.HOWL> <48DBB023.2030007@ar.media.kyoto-u.ac.jp> <94bdd2610809250929x733efe36ldb5692f789c6e3fa@mail.gmail.com> <48DBBEB0.5000507@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610809251020l17ab1b80kfef27d3c7664f582@mail.gmail.com> On Thu, Sep 25, 2008 at 6:39 PM, David Cournapeau < david at ar.media.kyoto-u.ac.jp> wrote: > Again, other people know distutils much better than me and may disagree > with me, or maybe rightfully notice my lack of experience on that > codebase, but from my own work on it - which while not complicated was > not trivial either - I got the conviction that any fundamental change > wrt distutils will be extremely painful to do if not impossible in a > backward compatible way. I was hoping that a distutils replacement would > come for Py3k, actually :) Well, everyone seem to agree on that: let's create a full, new replacement. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Sep 25 19:36:35 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Sep 2008 10:36:35 -0700 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DBC7FD.8000809@latte.ca> References: <48DBC7FD.8000809@latte.ca> Message-ID: I don't think anyone loves distutils, or even likes it. Unfortunately replacing it will be very tough, because distutils has arcane but important knowledge built into it about many platforms, and a new from-scratch system is unlikely to reach that level of coverage for years. There's also the issue that it's not enough to roll out the replacement with a new version of Python -- it would leave everyone using the old distutils in the lurch. It's a prime example of the network effect: until everyone owns one, nobody will want to own one. The best strategy I can think of would be to gradually rewrite the existing distutils, remaining compatible but also offering new APIs, and 5 years hence maybe the installed base will be close enough to 100% that developers writing setup scripts will start using the new features. That's how AJAX conquered the world. --Guido On Thu, Sep 25, 2008 at 10:18 AM, Blake Winton wrote: > Just as a question, is there _any_ chance of a new project to replace > distutils being accepted into Python? > > Thanks, > Blake. > > -------- Original Message -------- > Subject: Re: [Distutils] Annoucing distribute project > Date: Thu, 25 Sep 2008 12:55:36 -0400 > From: Jim Fulton > To: Andreas Jung > CC: distutils-sig at python.org > References: <94bdd2610809240432i307ed937rbfb1fd063cec915b at mail.gmail.com> > <20080924173708.13DA93A4128 at sparrow.telecommunity.com> > <94bdd2610809241244p13ad74dbkc13ee5b3ae3e14c8 at mail.gmail.com> > <48DAA90C.20504 at free.fr> > <20080924213342.612CC3A4127 at sparrow.telecommunity.com> > <94bdd2610809241456o20dade8bl4b46f3e0ed7aeb90 at mail.gmail.com> > <20080924223457.2BD7E3A4127 at sparrow.telecommunity.com> > <94bdd2610809242252j131d9fbcyd3e629667b455d29 at mail.gmail.com> > <20080925163034.208613A4072 at sparrow.telecommunity.com> > > > > On Sep 25, 2008, at 12:46 PM, Andreas Jung wrote: > >> >> >> --On 25. September 2008 12:31:40 -0400 "Phillip J. Eby" >> wrote: >> >> >>> Personally, I would rather see not a "fork", but development of an >>> entirely new disutils replacement, and then not worry at all about >>> backward compatibility at the setup.py level. Apart from the >>> pkg_resources module, I would treat setuptools and distutils as legacy >>> infrastructure, and explore better ways to build and manage >>> distributions >>> (including eggs). >> >> We can not restart with something completely different or new. Such an >> approach would be totally unacceptable due to the huge amount of existing >> modules having switched to setuptools. Setuptools or an improved codefork >> must be the tools for managing distibutions in Python. > > I would be willing to start with something new if and only if it was > adopted as part of Python. > > Jim > > -- > Jim Fulton > Zope Corporation > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From zooko at zooko.com Thu Sep 25 20:00:43 2008 From: zooko at zooko.com (zooko) Date: Thu, 25 Sep 2008 12:00:43 -0600 Subject: [Distutils] let's standardize dependency declaration without trying to agree on how to implement it In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <48D9DDE7.6010601@ar.media.kyoto-u.ac.jp> Message-ID: (following up to my own proposal with a case study) The Twisted Matrix project is a very big, widely used, well- engineered Python project. http://twistedmatrix.com It requires zope.interface to function. The Twisted hackers are mostly setuptools-haters, and are certainly not going to start using and depending on it, but they are willing to declare Twisted's dependency on zope.interface in a machine-readable way in order to facilitate correct installation of Twisted. The way they have currently accomplished this is by importing setuptools but attempting not to use it, and then if setuptools is present in sys.modules add the flag to setup(): "install_requires=['zope.interface']": http://twistedmatrix.com/trac/browser/trunk/setup.py The goal of all this from the perspective of Twisted developers is simply to add "This package requires zope.interface ." into their metadata in a way that setuptools, easy_install, pyinstall, distribute, stdeb, bbfreeze, vanguardistas.pydebdep, virtualenv, and other tools will understand. They do not want to use or depend on setuptools, nor do they want setuptools to have any other effects on their project than to emit that one simple fact of dependency metadata. What I am proposing is that in the next release of Python, all that Twisted developers need to do is put "install_requires= ['zope.interface']" into their invocation of distutils.setup(), and the appropriate metadata will be included in the resulting .egg-info in a way that all of the aforementioned tools will understand. This is a modest proposal -- it is backwards compatible, it is likely to be forwards-compatible with future Python packaging tools, and it does not, I hope, cause any problems for people who prefer not to use it. Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From ziade.tarek at gmail.com Thu Sep 25 22:59:17 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 25 Sep 2008 22:59:17 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> Message-ID: <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> On Thu, Sep 25, 2008 at 7:36 PM, Guido van Rossum wrote: > I don't think anyone loves distutils, or even likes it. Unfortunately > replacing it will be very tough, because distutils has arcane but > important knowledge built into it about many platforms, and a new > from-scratch system is unlikely to reach that level of coverage for > years. There's also the issue that it's not enough to roll out the > replacement with a new version of Python -- it would leave everyone > using the old distutils in the lurch. It's a prime example of the > network effect: until everyone owns one, nobody will want to own one. > The best strategy I can think of would be to gradually rewrite the > existing distutils, remaining compatible but also offering new APIs, > and 5 years hence maybe the installed base will be close enough to > 100% that developers writing setup scripts will start using the new > features. That's how AJAX conquered the world. It looks like you are describing setuptools in some ways: it was written on the top of distutils and provided enhancements and new features. But in the meantime distutils itself did not evolve. As far as I know, within the last year, one of the only new feature in distutils was the one I have proposed (multiple servers support in .pypirc). But it was really hard to get my change make it in because most Python core developers have a lack of interest for distutils. It took me 8 months iirc. Furthermore there are many things to change in this module to make it use modern Python before starting to add new features. (like logging usage but this is just an example) My point is that I am afraid not much will happen if something doesn't start outside Python core. Right now there's a momentum in the community, including framework gurus, that are willing to work on a new distutils package. They are not core developers but they are really good in distribution matters. Even Phillip Eby said that starting a new distutils could be a good pick in this thread earlier. Maybe a "distutils 2" project could start outside Python, using distutils and setuptools code as legacy infrastructure, and taking back pieces of it, Then it could be re-integrated in Python as a replacement for distutils when it is mature ? My 2 cents :) Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Sep 26 00:24:59 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Sep 2008 15:24:59 -0700 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> Message-ID: On Thu, Sep 25, 2008 at 1:59 PM, Tarek Ziad? wrote: > On Thu, Sep 25, 2008 at 7:36 PM, Guido van Rossum wrote: >> >> I don't think anyone loves distutils, or even likes it. Unfortunately >> replacing it will be very tough, because distutils has arcane but >> important knowledge built into it about many platforms, and a new >> from-scratch system is unlikely to reach that level of coverage for >> years. There's also the issue that it's not enough to roll out the >> replacement with a new version of Python -- it would leave everyone >> using the old distutils in the lurch. It's a prime example of the >> network effect: until everyone owns one, nobody will want to own one. >> The best strategy I can think of would be to gradually rewrite the >> existing distutils, remaining compatible but also offering new APIs, >> and 5 years hence maybe the installed base will be close enough to >> 100% that developers writing setup scripts will start using the new >> features. That's how AJAX conquered the world. > > It looks like you are describing setuptools in some ways: it was written > on the top of distutils and provided enhancements and new features. The mistake made was that setuptools didn't make it into the core distro. That in turn was because of concerns over its maintenance once incorporated into the core. (There's no need to go into more details.) > But in the meantime distutils itself did not evolve. Because nobody offered to evolve it gradually (setuptools was an attempt at a quantum leap, which I was trying to explain is the wrong approach). > As far as I know, within the last year, one of the only new feature in > distutils was the one > I have proposed (multiple servers support in .pypirc). But it was really > hard to get > my change make it in because most Python core developers have a lack of > interest for > distutils. It took me 8 months iirc. So you need to work yourself up to becoming a core developer. That's not impossible -- it just means contributing to other areas besides distutils. > Furthermore there are many things to change in this module to make it use > modern > Python before starting to add new features. (like logging usage but this is > just an > example) The best thing I can recommend is to start by contributing relatively small cleanups, that are easy to review and apply. > My point is that I am afraid not much will happen if something doesn't start > outside Python > core. My worry is that if something develops outside the core without careful thought about migration and compatibility it will be impossible to adapt into the core. A metaphor: You're not building a brand new bridge where there was no bridge before. You're trying to upgrade an existing bridge that people use for their daily commute. You can't even close a single lane during rush hour -- you have to plan carefully how the new bridge can grow organically right next to the old one and how traffic can be kept flowing continuously. Yes, it is much more expensive than just building a new bridge somewhere else. But it is the only way to do it. So you have to figure out how to do it. > Right now there's a momentum in the community, including framework gurus, > that are > willing to work on a new distutils package. They are not core developers but > they are really > good in distribution matters. Even Phillip Eby said that starting a new > distutils could be a good > pick in this thread earlier. I wasn't there. I'd like to refer to a post by Joel Spolsky about the problem with total rewrites: http://www.joelonsoftware.com/articles/fog0000000069.html > Maybe a "distutils 2" project could start outside Python, using distutils > and setuptools code as > legacy infrastructure, and taking back pieces of it, > > Then it could be re-integrated in Python as a replacement for distutils when > it is mature ? Only if much effort and study went into the planning of this re-integration. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From david at ar.media.kyoto-u.ac.jp Fri Sep 26 04:53:00 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 26 Sep 2008 11:53:00 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> Message-ID: <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> Guido van Rossum wrote: > I don't think anyone loves distutils, or even likes it. Unfortunately > replacing it will be very tough, because distutils has arcane but > important knowledge built into it about many platforms, and a new > from-scratch system is unlikely to reach that level of coverage for > years. Well, yes and no. Yes, there is some arcane knowledge in distutils, but the information is already there. Incidentally, a problem with distutils is that it is very difficult to reuse this knowlege in a programmatic way. For example, my first 'real' contact with distutils was to try extending it to build ctypes extensions (that is pure C dynamically loaded libraries loadable from ctypes). It should have been easy, since from a compile/build POV, that's the same than a C python extension. Unfortunately, the options are buried into the code, and worse, their location depend on the used platform/compiler (for example, distutils.sysconfig is not usable with Visual Studio). At the end, I gave up, doing it in a cross-platform way without copying all the knowledge from distutils was too difficult (for me at least). I was fed up with distutils complexity in numpy/scipy, so I started using scons within distutils to build all compiled code in numpy/scipy. I spent more time dealing with distutils idiosyncraties than writing the core new system. I am certainly not advocating using scons, but the fact is that in a couple of months, I, a not that knowledgable person about python and cross-platforms issues at that point, manage to build numpy and scipy without using distutils *at all* for C/C++/Fortran extensions. And numpy is certainly far from trivial to build (numpy distutils extensions is ~ 10 kloc), and deal with quite a variety of compilers and platforms. Maybe there is something to be learnt here: we could have a new small subproject to deal with building extensions, and plug it into distutils (e.g. have an extremely simplified scons; scons is overkill for the vast majority of python packages, and certainly not integrable in python core; several small projects in that space already exist). Something like rake; the arcane knowledge (for compiled extensions at least) could be moved gradually from distutils to this submodule. Would that fit into your vision or not ? cheers, David From uschmitt at mineway.de Fri Sep 26 11:10:53 2008 From: uschmitt at mineway.de (Uwe Schmitt) Date: Fri, 26 Sep 2008 11:10:53 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> References: <48DBC7FD.8000809@latte.ca> <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> Message-ID: <48DCA71D.3000100@mineway.de> Hi, is anybody aware of numscons ? http://conference.scipy.org/static/wiki/numscons.pdf I thinke they have some good ideas, maybe those ideas help. Greetings, Uwe -- Dr. rer. nat. Uwe Schmitt F&E Mathematik mineway GmbH Science Park 2 D-66123 Saarbr?cken Telefon: +49 (0)681 8390 5334 Telefax: +49 (0)681 830 4376 uschmitt at mineway.de www.mineway.de Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer Amtsgericht Saarbr?cken HRB 12339 From david at ar.media.kyoto-u.ac.jp Fri Sep 26 11:44:43 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 26 Sep 2008 18:44:43 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DCA71D.3000100@mineway.de> References: <48DBC7FD.8000809@latte.ca> <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> <48DCA71D.3000100@mineway.de> Message-ID: <48DCAF0B.1030603@ar.media.kyoto-u.ac.jp> Uwe Schmitt wrote: > Hi, > > is anybody aware of numscons ? > http://conference.scipy.org/static/wiki/numscons.pdf At least I am, since I am the main - and sole author at this point - of numscons :) Numscons was created to solve a specific problem: distutils is a nightmare for extensions which need to control the compilation process in a fine-grained manner. But numpy/scipy is quite special, and numscons would be overkill for most python packages. And it does not solve at all the problems people are talking about here (that is mostly deployment issues). The reasons why numscons was created are the same as why people want to get rid of distutils, though, that is distutils is very inflexible if you have special needs (and numpy/scipy needs are special; how many python packages need fortran ?), which is why I think it is somewhat relevant to this discussion. Numscons does show some points which are worth being pointed out IMHO: - it is possible to add a new command which handle a pretty big portion of distutils (the arcane part Guido was talking about I believe) in a sane manner, and it could be done gradually (scons is called from distutils, and scons is a distutils command: you do python setup.py scons to build your extensions, which means it is transparent for users at least). - It took me a few weeks to get simple python extensions built on most platforms (of which a big portion was to understand distutils and numpy.distutils). Of course, it would take more time to deal with all platforms python supports (more exactly all the corner cases), but I don't think it is that difficult. - A "miniscons" would be needed, but there are already many projects which do that, indicating that it should be relatively easy. The first rake prototype was 100 lines I believe; vellum is 300 lines. Something like vellum would be enough for most needs I believe. Of course, doing it for python itself means some issues which were non-existent in numpy's case become big problems (bootstrapping issues, for once). cheers, David From guido at python.org Fri Sep 26 16:10:36 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 26 Sep 2008 07:10:36 -0700 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> References: <48DBC7FD.8000809@latte.ca> <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> Message-ID: On Thu, Sep 25, 2008 at 7:53 PM, David Cournapeau wrote: > Guido van Rossum wrote: >> I don't think anyone loves distutils, or even likes it. Unfortunately >> replacing it will be very tough, because distutils has arcane but >> important knowledge built into it about many platforms, and a new >> from-scratch system is unlikely to reach that level of coverage for >> years. > > Well, yes and no. Yes, there is some arcane knowledge in distutils, but > the information is already there. Incidentally, a problem with distutils > is that it is very difficult to reuse this knowlege in a programmatic > way. For example, my first 'real' contact with distutils was to try > extending it to build ctypes extensions (that is pure C dynamically > loaded libraries loadable from ctypes). It should have been easy, since > from a compile/build POV, that's the same than a C python extension. > Unfortunately, the options are buried into the code, and worse, their > location depend on the used platform/compiler (for example, > distutils.sysconfig is not usable with Visual Studio). At the end, I > gave up, doing it in a cross-platform way without copying all the > knowledge from distutils was too difficult (for me at least). > > I was fed up with distutils complexity in numpy/scipy, so I started > using scons within distutils to build all compiled code in numpy/scipy. > I spent more time dealing with distutils idiosyncraties than writing the > core new system. I am certainly not advocating using scons, but the fact > is that in a couple of months, I, a not that knowledgable person about > python and cross-platforms issues at that point, manage to build numpy > and scipy without using distutils *at all* for C/C++/Fortran extensions. > And numpy is certainly far from trivial to build (numpy distutils > extensions is ~ 10 kloc), and deal with quite a variety of compilers and > platforms. > > Maybe there is something to be learnt here: we could have a new small > subproject to deal with building extensions, and plug it into distutils > (e.g. have an extremely simplified scons; scons is overkill for the vast > majority of python packages, and certainly not integrable in python > core; several small projects in that space already exist). Something > like rake; the arcane knowledge (for compiled extensions at least) could > be moved gradually from distutils to this submodule. Would that fit into > your vision or not ? Can you think of a specific way to evolve distutils to provide the API you would like to see, without (for the time being) losing backwards compatibility? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Sep 26 17:47:46 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 26 Sep 2008 11:47:46 -0400 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> Message-ID: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> At 03:24 PM 9/25/2008 -0700, Guido van Rossum wrote: > > Right now there's a momentum in the community, including framework gurus, > > that are > > willing to work on a new distutils package. They are not core > developers but > > they are really > > good in distribution matters. Even Phillip Eby said that starting a new > > distutils could be a good > > pick in this thread earlier. > >I wasn't there. I'd like to refer to a post by Joel Spolsky about the >problem with total rewrites: >http://www.joelonsoftware.com/articles/fog0000000069.html The economic factors are a bit different, here. Joel himself has previously pointed out that where Netscape failed, Mozilla won - i.e., the economics of open source can mean that it's sometimes easier to get volunteers for a new project than for fixing an old one, or at least for a project where dropping backward compatibility is allowed (e.g. Py3K). In the case of the distutils, the people who are capable of extending it are tired of doing so, and the people who have the energy and time are very unlikely to be able to work on it much without breaking something significant. Distutils is also far too flexible in some areas to be able to improve much while maintaining 100% backward compatibility -- it doesn't enforce One Obvious Way To Do It. What's more, there are very few people who've even said they like the distutils API or think it's a good fit for the application area. And, frankly, the domain knowledge embedded in the distutils are of fairly limited scope and kind: * Extension building, compile/link options and defines * Wildly-differing installation path schemes across platforms * Platform distribution formats like bdist_rpm, bdist_wininst, and bdist_msi Most other things the distutils do are either well-specified, obsolete (e.g. its internal logging and option parsing libs), or probably not worth keeping (e.g. bdist_dumb). > > Maybe a "distutils 2" project could start outside Python, using distutils > > and setuptools code as > > legacy infrastructure, and taking back pieces of it, > > > > Then it could be re-integrated in Python as a replacement for > distutils when > > it is mature ? > >Only if much effort and study went into the planning of this re-integration. That's why at least some of the discussion has been around requirements gathering and PEP-writing as a first step. My own inclination is that a scalable future for distutils means an improved sdist format, the end of setup.py as an command-line interface, and community-maintained platform-specific installation tools that process source or binary distributions. Most complaints about distutils (and setuptools, for that matter) are focused on installation policy&preference issues. Making it possible and practical for a variety of tools to flourish around a standardized format (ala WSGI) seems like the way to go. Notice that the existence of eggs has already allowed buildout, virtualenv, and pyinstall to appear. But eggs don't handle installing tests or documentation, and they have to be prebuilt by platform. An improved sdist format, on the other hand, with standardized layout and metadata would address all of those issues. The tools for building this format and APIs for inspecting it would be candidates for the stdlib, in much the same way that wsgiref was - the spec is/should be stable, and those parts that are compiler/install-layout specific will need to be maintained in the stdlib anyway, for Python's own build infrastructure. In that sense, "distutils 2" would not be so much a rewrite of the distutils, as the separation of them into tools for distributing, and tools for installing, where some of the tools for installing may be community-maintained. That's the general idea, anyway. (I'm going to be away from email for a few days, so I'll probably be out of this thread 'till Tuesday.) From pje at telecommunity.com Fri Sep 26 18:05:24 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 26 Sep 2008 12:05:24 -0400 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <20080926160418.ED8403A4072@sparrow.telecommunity.com> At 11:47 AM 9/26/2008 -0400, Phillip J. Eby wrote: >and those parts that are compiler/install-layout specific will need >to be maintained in the stdlib anyway, for Python's own build infrastructure. I just realized the above is not too clear: what I mean is that anything that's specific to *Python's* installation layout on a given platform should be in the stdlib. (Not the things that depend on how vendors want libraries to be installed or how users want to install them; that should be left to community-supplied installation tools, except for a single, simple "standard" install tool, analagous to wsgiref's SimpleHTTPServer -- able to do the basic stuff, but if you want fancy, you go roll your own or download someone else's.) From ziade.tarek at gmail.com Fri Sep 26 18:42:35 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 26 Sep 2008 18:42:35 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <94bdd2610809260942u64e755c5n9cd4145220f5c487@mail.gmail.com> On Fri, Sep 26, 2008 at 5:47 PM, Phillip J. Eby wrote: > > My own inclination is that a scalable future for distutils means an > improved sdist format, the end of setup.py as an command-line interface, and > community-maintained platform-specific installation tools that process > source or binary distributions. Most complaints about distutils (and > setuptools, for that matter) are focused on installation policy&preference > issues. Making it possible and practical for a variety of tools to flourish > around a standardized format (ala WSGI) seems like the way to go. > [cut] > In that sense, "distutils 2" would not be so much a rewrite of the > distutils, as the separation of them into tools for distributing, and tools > for installing, where some of the tools for installing may be > community-maintained. > There's something we could probably take out of distutils really easily : the register and upload commands. Those commands work together with a PyPI-like server, and deal with a .pypirc file. This code is really simple to take off distutils. What about putting it in a separate package or project ? (maybe "releaser" or something) We could have a separate project that deals with this idea of registering uploading a package to PyPI or elsewhere, with a set of scripts distributed with Python maybe. I can think of many features ? la CPAN, to query PyPI repositories through XML-RPC, like ppmtools, so users can upload but also browse what is available. This is probably a project where pyinstaller and/or easy_installer could live as well: a set of tools to browse, install, register and upload packages. Now maybe this is too ambitious, but at least, separate register and upload from distutils is definitely doable and not too risky. I could defenitely write a PEP on that particular part if it sounds good. Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Sep 26 18:55:28 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 26 Sep 2008 09:55:28 -0700 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: On Fri, Sep 26, 2008 at 8:47 AM, Phillip J. Eby wrote: > At 03:24 PM 9/25/2008 -0700, Guido van Rossum wrote: >> >> > Right now there's a momentum in the community, including framework >> > gurus, >> > that are >> > willing to work on a new distutils package. They are not core developers >> > but >> > they are really >> > good in distribution matters. Even Phillip Eby said that starting a new >> > distutils could be a good >> > pick in this thread earlier. >> >> I wasn't there. I'd like to refer to a post by Joel Spolsky about the >> problem with total rewrites: >> http://www.joelonsoftware.com/articles/fog0000000069.html > > The economic factors are a bit different, here. Joel himself has previously > pointed out that where Netscape failed, Mozilla won - i.e., the economics of > open source can mean that it's sometimes easier to get volunteers for a new > project than for fixing an old one, or at least for a project where dropping > backward compatibility is allowed (e.g. Py3K). Yeah, but *is* dropping backward compatibility an option here? > In the case of the distutils, the people who are capable of extending it are > tired of doing so, and the people who have the energy and time are very > unlikely to be able to work on it much without breaking something > significant. The same might be happen to distutils 2 in a few years. You are going to have to plan making it more maintainable. It might also help if there was a team of people working on the replacement instead of a single one -- reduces the risk of that one person getting tired or engaged or whatever. > Distutils is also far too flexible in some areas to be able to > improve much while maintaining 100% backward compatibility -- it doesn't > enforce One Obvious Way To Do It. That's a good criticism, and can be dealt with through careful deprecation. > What's more, there are very few people who've even said they like the > distutils API or think it's a good fit for the application area. And, > frankly, the domain knowledge embedded in the distutils are of fairly > limited scope and kind: > > * Extension building, compile/link options and defines > * Wildly-differing installation path schemes across platforms > * Platform distribution formats like bdist_rpm, bdist_wininst, and bdist_msi So maybe this part is worth salvaging? > Most other things the distutils do are either well-specified, obsolete (e.g. > its internal logging and option parsing libs), or probably not worth keeping > (e.g. bdist_dumb). It's fine to gradually replace these with better stuff. Deprecated backward compatible APIs can be offered for a few Python releases. >> > Maybe a "distutils 2" project could start outside Python, using >> > distutils >> > and setuptools code as >> > legacy infrastructure, and taking back pieces of it, >> > >> > Then it could be re-integrated in Python as a replacement for distutils >> > when >> > it is mature ? >> >> Only if much effort and study went into the planning of this >> re-integration. > > That's why at least some of the discussion has been around requirements > gathering and PEP-writing as a first step. That's good to hear. > My own inclination is that a scalable future for distutils means an improved > sdist format, the end of setup.py as an command-line interface, and > community-maintained platform-specific installation tools that process > source or binary distributions. Most complaints about distutils (and > setuptools, for that matter) are focused on installation policy&preference > issues. Making it possible and practical for a variety of tools to flourish > around a standardized format (ala WSGI) seems like the way to go. Given the success of WSGI (which I use every day) this sounds like a very good plan! > Notice that the existence of eggs has already allowed buildout, virtualenv, > and pyinstall to appear. But eggs don't handle installing tests or > documentation, and they have to be prebuilt by platform. An improved sdist > format, on the other hand, with standardized layout and metadata would > address all of those issues. Beware however of too large a scope -- the project may crumble under its ambition, or be mired in border conflicts with language-independent distribution management tools like RPM or the Debian/Ubuntu tools. > The tools for building this format and APIs for inspecting it would be > candidates for the stdlib, in much the same way that wsgiref was - the spec > is/should be stable, and those parts that are compiler/install-layout > specific will need to be maintained in the stdlib anyway, for Python's own > build infrastructure. Good thinking. This is also similar to the conclusion we came to with respect to NumPy -- NumPy itself is a separate package and will always remain so, but core Python contains some things that make this possible. > In that sense, "distutils 2" would not be so much a rewrite of the > distutils, as the separation of them into tools for distributing, and tools > for installing, where some of the tools for installing may be > community-maintained. > > That's the general idea, anyway. Sounds good. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From david at ar.media.kyoto-u.ac.jp Fri Sep 26 19:03:10 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 27 Sep 2008 02:03:10 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <48DD15CE.9020601@ar.media.kyoto-u.ac.jp> Phillip J. Eby wrote: > The economic factors are a bit different, here. Joel himself has > previously pointed out that where Netscape failed, Mozilla won - i.e., > the economics of open source can mean that it's sometimes easier to > get volunteers for a new project than for fixing an old one, or at > least for a project where dropping backward compatibility is allowed > (e.g. Py3K). Yes, I forgot to mention that earlier: I don't think Joel's point is applicable to distutils. Distutils is by almost all acount a small project (sloccount tells me Python-2.5.2/Lib/distutils is 9241 loc); one of Joel's point is that it is more costly to write from scratch than to refactor. But I think distutils is not "refactorable" (sorry for hurting native English speakers): a lot of things are fundamentally broken *by design* (option handling, running one command after the other, etc...), and cannot be changed in a backward compatible way. It is also small enough so that a single person can have a pretty good idea of the whole design. So it maybe less costly to do something from scratch, simply because starting from scratch would brings much more resources. For example, some people from OS distributors complain, rightfully, about distutils: my understanding is that they would put ressources for code which would (optionally) make python packages "OS compliant" (FHS compliant on linux, etc...). But not in the current state of affairs. Joel's point is valid when you have a fixed amount of resources, e.g. a company; in open source, the dynamics can be different [1] > What's more, there are very few people who've even said they like the > distutils API or think it's a good fit for the application area. And, > frankly, the domain knowledge embedded in the distutils are of fairly > limited scope and kind: > > * Extension building, compile/link options and defines This can be done without too much work: the hard work really is the options and the corner cases. Corner cases can be solved gradually (and there are not that many corner-cases; mostly VS stuff and some mac os X stuff in my experience). I have done it for numscons (scons did made this a bit easier, but I needed to read and understand almost all the distutils and numpy.distutils code to build C code on all supported platforms, including Visual Studio). I would be glad to help there, that's a part I now feel relatively confident with. > * Wildly-differing installation path schemes across platforms I did not touch this part too much. Would it be possible to refactor internally this part in distutils, such as we can provide an alternative implementation, which is not backward compatible, but would be specified from a set of requirements (FHS compliant option, etc...). By default, setup.py users would get the current default, but if they want, they would get a better model here. cheers, David [1] Again, talking about numpy/scipy: numpy.distutils itself was ~ 10000 loc, and it was much easier for me to start from scratch (but using numpy.distutils knowledge) than to solve numpy.distutils problems. At least in the case of numpy, people smarter than me tried to improve numpy.distutils, but never did it because everytime they did something, they broke something else. Now, it may just be that I have way more free-time than them. But the point remain that in open source, the dynamics can be different. From david at ar.media.kyoto-u.ac.jp Fri Sep 26 19:05:33 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 27 Sep 2008 02:05:33 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <48DC4E8C.2050009@ar.media.kyoto-u.ac.jp> Message-ID: <48DD165D.7050500@ar.media.kyoto-u.ac.jp> Guido van Rossum wrote: > > Can you think of a specific way to evolve distutils to provide the API > you would like to see, without (for the time being) losing backwards > compatibility? > Here is what I did with numscons: numscons is called through distutils scons command, which is called during the build dance. By default, the scons command does nothing, but you add scons scripts through a distutils command, and in that case only scons does something. Those hooks do not break anything in numpy.distutils. The best proof is that the default build system is still numpy.distutils, and numscons is not in numpy sources (only the hooks are). Some people build with numscons, other - the majority - use the current system. Same source tree, same setup.py files (except for C extensions of course; they have to be synchronised, but the common part between the two build could easily be factorized in a common_setup I think). That's the idea I was suggesting and may be applicable to python as a whole: - design a small self contained build system, which does nothing but is extensible (there is a lot of prior art: vellum, rake/rant, etc...), and "hook" it into distutils through a new distutils command. People smarter than me will enjoy this part, and will hopefully come up with something nice. - The new command would not nothing if you don't use the new features: you don't break any existing setup.py - Another thing would be to gradually move all the configuration from code into a separate module in a sane way: putting the flags in config files, not in the code, make an API to discover and reuse those settings, etc... Basically, all the arcane stuff. This would be independant of distutils, but would be callable from distutils (by the new command and the current distutils). A good way to design the API here would be to make sure it work in distutils and in e.g. scons (so a python extension builder in scons could be easily done importing this module only: all the compiler specific flags, etc...). It would be a distutils.sysconfig on steroids, which works for every platform - including windows with VS. So that every other python build tool, being vellum, scons, whatever, could access those. I believe this can be made general enough to be usable by almost any tool. At least I started something toward this direction for scons, and if it works for scons, it should work for a lot of tools. That would enable refactoring something like 1/3 of current distutils code, and provide a better build tool for people who use the new 'stuff'. I realize this is really vague at that point, but up to today, I never thought about what could be applied to python in general from my work numscons. If you think the above makes any sense, I will think more about this, cheers, David From ianb at colorstudy.com Fri Sep 26 21:34:08 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 26 Sep 2008 14:34:08 -0500 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <48DD3930.4060907@colorstudy.com> Phillip J. Eby wrote: > In the case of the distutils, the people who are capable of extending it > are tired of doing so, and the people who have the energy and time are > very unlikely to be able to work on it much without breaking something > significant. Distutils is also far too flexible in some areas to be > able to improve much while maintaining 100% backward compatibility -- it > doesn't enforce One Obvious Way To Do It. Basically volunteer projects like this just won't move forward when doing so is painful -- I think distutils is like this, and though you suffered through that with the development of setuptools I think part of this recent discussion is that it's still not really self-sustaining. So I think we need a bit higher bar of code quality (and brevity) than in a commercial project. > What's more, there are very few people who've even said they like the > distutils API or think it's a good fit for the application area. And, > frankly, the domain knowledge embedded in the distutils are of fairly > limited scope and kind: > > * Extension building, compile/link options and defines > * Wildly-differing installation path schemes across platforms > * Platform distribution formats like bdist_rpm, bdist_wininst, and > bdist_msi > > Most other things the distutils do are either well-specified, obsolete > (e.g. its internal logging and option parsing libs), or probably not > worth keeping (e.g. bdist_dumb). Yes -- what distutils really does just isn't that interesting or complicated. There's a lot of arcane knowledge built into the build process, but it's not well expressed or particularly useful, and for pure-Python packages there really isn't any arcane knowledge to be had. There's other features, but a lot of them aren't well enough implemented to be reliable, or they are adapted in eclectic ways by individual packages in a way that makes them unpredictable. The arcane knowledge is distutils is broken and unusable -- I don't think distutils reform would actually save much that was useful. >> > Maybe a "distutils 2" project could start outside Python, using >> distutils >> > and setuptools code as >> > legacy infrastructure, and taking back pieces of it, >> > >> > Then it could be re-integrated in Python as a replacement for >> distutils when >> > it is mature ? >> >> Only if much effort and study went into the planning of this >> re-integration. > > That's why at least some of the discussion has been around requirements > gathering and PEP-writing as a first step. There is a much better sense of what the scope of distutils should be now than when it was written, and the PEP process could make that clarity explicit. I suspect that when the scope is made clear that the implementation just won't be that complex. > My own inclination is that a scalable future for distutils means an > improved sdist format, the end of setup.py as an command-line interface, > and community-maintained platform-specific installation tools that > process source or binary distributions. Most complaints about distutils > (and setuptools, for that matter) are focused on installation > policy&preference issues. Making it possible and practical for a > variety of tools to flourish around a standardized format (ala WSGI) > seems like the way to go. I'd like to see an improvement of sdist, toward a more declarative and introspectable system. Just including EGG-INFO in sdist would be a big help, and allow the easy reading of the egg metadata without having to deal with the binary aspects of .egg (which remain very challenging to use reliably). In pyinstall I'm unpacking and running egg_info to get this data, but that's very awkward and that metadata should really just be there without having to invoke any code. I'm not sure I agree with removing setup.py, though I dislike how it functions currently -- that you kind of have to poke it from the side to get basic information out of it, instead of something more declarative. In some ways it's the distutils/setuptools interface that most people actually use -- pyinstall for instance only calls setup.py in subprocesses to install the package, and never touches it directly. So anything that acts fairly similar to the current setup.py's will be compatible. This is part of why I don't think the backward compatibility issue is as difficult as Guido suggests. Though... really if pyinstall could easily detect a new source format and only setup.py with the old source format, it wouldn't be hard to support both. There does need to be room for custom code, but mostly for the build/compilation process. > Notice that the existence of eggs has already allowed buildout, > virtualenv, and pyinstall to appear. But eggs don't handle installing > tests or documentation, and they have to be prebuilt by platform. An > improved sdist format, on the other hand, with standardized layout and > metadata would address all of those issues. > > The tools for building this format and APIs for inspecting it would be > candidates for the stdlib, in much the same way that wsgiref was - the > spec is/should be stable, and those parts that are > compiler/install-layout specific will need to be maintained in the > stdlib anyway, for Python's own build infrastructure. > > In that sense, "distutils 2" would not be so much a rewrite of the > distutils, as the separation of them into tools for distributing, and > tools for installing, where some of the tools for installing may be > community-maintained. I agree. For instance, I don't see a particular advantage to making pyinstall part of the core of Python -- it would increase adoption and add a certain review process, but its usefulness is not contingent on everyone using it. It's more important that there's a consensus (or... movement towards some consensus) about how people write and distribute packages. If some distutils 2 effort just addressed that, and avoided things like installation that mostly can be avoided (though installation needs to be co-developed, of course), it should keep the scope down so that it's not quite as hard to agree on things. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From nicolas.chauvat at logilab.fr Fri Sep 26 22:33:22 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Fri, 26 Sep 2008 22:33:22 +0200 Subject: [Distutils] [pyconuk] "Python Package Management Sucks" In-Reply-To: <48D8C701.6070707@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> Message-ID: <20080926203322.GD9951@volans.logilab.fr> On Tue, Sep 23, 2008 at 11:37:53AM +0100, Chris Withers wrote: >> Install debian and get back to productive tasks. > > This is an almost troll-like answer. True. But it worked, didn't it? Hopefully the thread will end up with some positive output. :) > See page 35 of the presentation. > > If you're too lazy, here's a re-hash: > > - there are lots of different operating systems Agreed. > - even with Linux, there are many different package management systems I would not link package management systems that strongly to operating systems (or kernels): http://www.debian.org/ports/index#nonlinux > - all the package management systems behave differently and expect > packages to be set up differently for them But there is the Linux Standard Base to bind them. > - expecting package developers to shoulder this burden is unfair and > results in various bitrotting repackaged versions of python packages > rather than just one up-to-date version maintained by the entity > originating the python package PyPI says it holds 4818 packages. I doubt all of these are actually worth being packaged. Please note that my point of view is not the one of the developer or single-user who wants to try out something easily and quickly, but the one of the developer that has to deploy his software many times in many places and maintain them in production for a long time. I understand easy_install and similar tools make it easy to try something out in one's home directory and I have nothing to say against that. I complain that some advocate this as the One True Way to distribute their code and end up with code that depends on it, thus complicating matters for the ones who have been happily using tools that manage entire systems for years. > - Adobe Photoshop Plugins, Firefox Add-ons, etc do not delegate their > management to an OS package manager. Packages are Python's "plugins" and > so should get the same type of consistent, cross-platform package > management targetted at the application in question, which is Python in > this case. I strongly disagree with this. I guess this is why you may like a Python-specific package management system whereas I never will. To me, there is no such thing as a clear boundary between a "Python subsystem" and the rest of a computing system. A have only one (complex) system, in my case Debian, and want only one tool to manage it. -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From nicolas.chauvat at logilab.fr Fri Sep 26 22:35:59 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Fri, 26 Sep 2008 22:35:59 +0200 Subject: [Distutils] [pyconuk] "Python Package Management Sucks" In-Reply-To: <94bdd2610809230438m70869babh6e8a3791d5b661fb@mail.gmail.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <94bdd2610809230438m70869babh6e8a3791d5b661fb@mail.gmail.com> Message-ID: <20080926203559.GE9951@volans.logilab.fr> Hi Tarek, On Tue, Sep 23, 2008 at 01:38:00PM +0200, Tarek Ziad? wrote: > I just wanted to react on that: I have created a setuptools-enabled package > for Pylint (logilab.pylintinstaller, see http://pypi.python.org/pypi/ > logilab.pylintinstaller) with the bless of Logilab, so people under Windows > could use the tool with a one-liner installation process... Thanks again for doing it. We provide Debian packages for the free software we make and we will support anyone wanting to package our free software for other targets than Debian. -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From nicolas.chauvat at logilab.fr Fri Sep 26 23:09:40 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Fri, 26 Sep 2008 23:09:40 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> Message-ID: <20080926210940.GH9951@volans.logilab.fr> Hi Stephen, On Tue, Sep 23, 2008 at 11:59:18AM +0100, Stephen Pascoe wrote: > I'm interested in hearing what you find so annoying about the egg format > because for me it's the one part of the setuptools system that I would keep. I suppose the best answer is to point you to this thread: http://teams.debian.net/lurker/message/20070904.152810.4f84c924.fr.html There is also useful information at: http://wiki.debian.org/DebianPythonFAQ and http://www.debian.org/doc/packaging-manuals/python-policy/ but I would not bet on the latter being up to date. Baseline is "no problem with providing egg-info metadata, but pretty please Python developers, do not code *for* distutils/setuptools/etc. Just find a way to provide useful dependency/meta information then let your users choose how they install your code on *their* system". -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From nicolas.chauvat at logilab.fr Fri Sep 26 23:41:16 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Fri, 26 Sep 2008 23:41:16 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48D8C927.4060609@simplistix.co.uk> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> Message-ID: <20080926214116.GI9951@volans.logilab.fr> Hi, [sorry for slowly drifting away from pure distutils-related topics] On Tue, Sep 23, 2008 at 11:47:03AM +0100, Chris Withers wrote: >> My main tool is Python, but I have many other tools on my system. I >> do not want to have as many package management utilities as >> "subsystems". > > Then I suggest you volunteer to maintain the debian packages for every > single python package. Do you really think every single Python package in PyPI deserves to be packaged for every distribution? I don't. How do I make a difference? When I need something I download it. When I find it really useful and plan on using it I package it. Many others are behaving in the same way and the result is "apt-cache search python". > If you have projects this large, then you likely want to roll your own > OS packages anyway. I am not sure what you mean by "OS packages". Do you mean "roll your own distribution" as in "roll your own thing based on Debian and adapt packages to your needs as Ubuntu is doing"? >> [Please note that for an experienced Debian developer, making the >> initial package of a Python module can be a matter of half an hour to >> a couple hours and releasing a new version a matter of minutes.] > > ...and for someone not using Debian or not an experienced Debian > developer? Despite being a fan of Debian, I'm well aware of just how > "friendly" a community it can be to the new user... Do you expect someone who is not proficient at programming to quickly design and develop a piece of software? What makes you think it would be different for integrating pieces of software into a consistent system? As I said on the pyconuk list, packaging software requires some work and currently there is no way around it. Tools get better over time, but automation is out of reach. As usual "user != developer". For someone not using Debian: just be happy with whatever tool you choose to use. For someone not an experienced Debian developer: just wait for someone to do the work you want to benefit from, or learn to do it yourself and get it done. In my company's case, we worked for years on setting up an efficient environment for software development and system administration and we are now able to move python code from a mercurial repository to a production system running Debian in a matter of minutes. The tools we built to support this process have been free software from the very beginning and can be found on our website http://www.logilab.org/project/logilab-devtools -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From floris.bruynooghe at gmail.com Sat Sep 27 00:06:56 2008 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Fri, 26 Sep 2008 23:06:56 +0100 Subject: [Distutils] [pyconuk] "Python Package Management Sucks" In-Reply-To: <20080926203322.GD9951@volans.logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <20080926203322.GD9951@volans.logilab.fr> Message-ID: <20080926220656.GA12586@laurie.devork> On Fri, Sep 26, 2008 at 10:33:22PM +0200, Nicolas Chauvat wrote: > On Tue, Sep 23, 2008 at 11:37:53AM +0100, Chris Withers wrote: > > - all the package management systems behave differently and expect > > packages to be set up differently for them > > But there is the Linux Standard Base to bind them. Hmm, I've never read the packaging sections of the LSB (and maybe I should before making a comment like this), but I've always heard complaints that it is very RPM-centric and ignores DEBs etc. So maybe even the LSB is not quite up to the task of binding them all yet... Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From ben+python at benfinney.id.au Sat Sep 27 01:01:26 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 27 Sep 2008 09:01:26 +1000 Subject: [Distutils] [pyconuk] "Python Package Management Sucks" References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <20080926203322.GD9951@volans.logilab.fr> <20080926220656.GA12586@laurie.devork> Message-ID: <87r676v7eh.fsf@benfinney.id.au> Floris Bruynooghe writes: > Hmm, I've never read the packaging sections of the LSB (and maybe I > should before making a comment like this) > but I've always heard complaints that it is very RPM-centric and > ignores DEBs etc. The LSB core specification does recommend (and specify the format of) RPM packages for distribution; but it doesn't require the system to use RPMs natively, only that such packages should be installable. Note: Supplying an RPM format package is encouraged because it makes systems easier to manage. This specification does not require the implementation to use RPM as the package manager; it only specifies the format of the package file. So, a Debian system is LSB-compliant if it enables the installation of externally-produced LSB-compliant RPM packages. This is easily done by making an appropriate tool available; I think 'alien' is the one used on a Debian system. The LSB folks have (IMHO as an outside observer) worked quite well with the Debian project and vice versa. A Debian system now has various 'lsb-*' packages available that make it simple to bring a Debian system into compliance with the various specifications of the LSB (note: I don't know what gaps in compliance may remain even with those packages installed). -- \ ?Everything is futile.? ?Marvin of Borg | `\ | _o__) | Ben Finney From greg.ewing at canterbury.ac.nz Sat Sep 27 03:20:15 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 27 Sep 2008 13:20:15 +1200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <48DD8A4F.9030201@canterbury.ac.nz> Guido van Rossum wrote: > Yeah, but *is* dropping backward compatibility an option here? I'm not sure the concept of backward compatibility makes sense here. The only kind of distutils replacement I would be interested in would have a completely different API, completely different structure and completely different implementation. Anything less would fail to fix the problems we want to fix. Given that, what would it even *mean* for the new tool to be backward compatible with distutils? > The same might be happen to distutils 2 in a few years. You are going > to have to plan making it more maintainable. A modular structure with an easy-to-follow implementation would certainly be part of the plan, I hope. -- Greg From greg.ewing at canterbury.ac.nz Sat Sep 27 03:27:22 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 27 Sep 2008 13:27:22 +1200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> Message-ID: <48DD8BFA.7010406@canterbury.ac.nz> Guido van Rossum wrote: > You're not building a brand new bridge where there was no bridge > before. You're trying to upgrade an existing bridge that people use > for their daily commute. But that hasn't been decided. Whether to upgrade the existing bridge or build a new one is precisely what we're debating. And it seems to me that the easiest way to avoid disrupting existing traffic *is* to build a new bridge alongside and then scrap the old one. If the existing bridge isn't too badly run down and is basically still the kind of bridge you want, upgrading it may make sense. But that doesn't seem to be the case with the distutils bridge. -- Greg From greg.ewing at canterbury.ac.nz Sat Sep 27 03:05:24 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 27 Sep 2008 13:05:24 +1200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <20080926154640.5B9B83A4072@sparrow.telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <48DD86D4.50704@canterbury.ac.nz> Phillip J. Eby wrote: > the domain knowledge embedded in the distutils are of fairly > limited scope and kind: > > * Extension building, compile/link options and defines > * Wildly-differing installation path schemes across platforms > * Platform distribution formats like bdist_rpm, bdist_wininst, and > bdist_msi Seems to me that if there were a well-defined API for plugging in platform-dependent modules, it shouldn't be too hard to find people willing to contribute modules for the platforms they're familiar with. It's not really that the knowledge is particularly arcane, just that there's no single person who knows it all. This makes it very unlikely that one perso could come up with an entire viable distutils replacement just by scratching their own itches. We need a communal itch-scratching session. -- Greg From ivazqueznet at gmail.com Fri Sep 26 20:50:08 2008 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Fri, 26 Sep 2008 14:50:08 -0400 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> Message-ID: <1222455008.3327.18.camel@ignacio.lan> On Fri, 2008-09-26 at 09:55 -0700, Guido van Rossum wrote: > On Fri, Sep 26, 2008 at 8:47 AM, Phillip J. Eby wrote: > > Notice that the existence of eggs has already allowed buildout, virtualenv, > > and pyinstall to appear. But eggs don't handle installing tests or > > documentation, and they have to be prebuilt by platform. An improved sdist > > format, on the other hand, with standardized layout and metadata would > > address all of those issues. > > Beware however of too large a scope -- the project may crumble under > its ambition, or be mired in border conflicts with > language-independent distribution management tools like RPM or the > Debian/Ubuntu tools. Would it be worth bringing in people from those groups to help guide its growth? -- Ignacio Vazquez-Abrams -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From david at ar.media.kyoto-u.ac.jp Sat Sep 27 10:18:36 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 27 Sep 2008 17:18:36 +0900 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080926214116.GI9951@volans.logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> Message-ID: <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> Nicolas Chauvat wrote: > > Do you really think every single Python package in PyPI deserves to be > packaged for every distribution? I don't. How do I make a difference? > When I need something I download it. When I find it really useful and > plan on using it I package it. Many others are behaving in the same > way and the result is "apt-cache search python". This is narrow-minded. I understand your POV, I really do (I use a debian-based system myself, and hate the way software installation works on any other system), but what you are saying cannot work in a general manner. For example, installing from sources on most other systems is frowned upon, and rightfully so, because it is even more complicated to do than on a decent linux system. It is painful because either the OS makes it terribly difficult (windows), or because you have antique/not well supported toolsets (old Solaris, etc...). If you mainly use only one OS, you just can't understand the pain. I am not saying that python plugins must be THE deployment system, but that it has to be one system, because plugins systems are as pervasive on other OS as .deb are on debian. So we should think about what kind of things python core can provide to help other tools to either build "native" packages or eggs, and not having a big pile of code which mix everything. As Matthias Klose mentioned earlier, a lot of those formats share common requirements. We should talk about those instead of saying my package is bigger than yours. > > As usual "user != developer". For someone not using Debian: just be > happy with whatever tool you choose to use. For someone not an > experienced Debian developer: just wait for someone to do the work you > want to benefit from, or learn to do it yourself and get it done. And how do you distribute new versions of your package ? You wait for debian to package it correctly ? For fast-moving packages, debian are not the ultimate solution, far from it. I mean, it is not like OpenSuse build service, Ubuntu ppa systems came from nowhere. There is a need for softwares developers to distribute themselves the software for newer versions, and in that case, the native system (at least used "officialy") simply is not appropriate. cheers, David From tarek.ziade at ingeniweb.com Sat Sep 27 11:52:39 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Sat, 27 Sep 2008 11:52:39 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> Message-ID: 2008/9/27 David Cournapeau > Nicolas Chauvat wrote: > > > > Do you really think every single Python package in PyPI deserves to be > > packaged for every distribution? I don't. How do I make a difference? > > When I need something I download it. When I find it really useful and > > plan on using it I package it. Many others are behaving in the same > > way and the result is "apt-cache search python". > > This is narrow-minded. I understand your POV, I really do (I use a > debian-based system myself, and hate the way software installation works > on any other system), but what you are saying cannot work in a general > manner. IMHO we could have a command-line search feature ? la CPAN at PyPI like what apt provides. $ pypi-search "foo" Tarek -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From tarek.ziade at ingeniweb.com Sat Sep 27 12:01:39 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Sat, 27 Sep 2008 12:01:39 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> Message-ID: 2008/9/27 Tarek Ziade > > > 2008/9/27 David Cournapeau > >> Nicolas Chauvat wrote: >> > >> > Do you really think every single Python package in PyPI deserves to be >> > packaged for every distribution? I don't. How do I make a difference? >> > When I need something I download it. When I find it really useful and >> > plan on using it I package it. Many others are behaving in the same >> > way and the result is "apt-cache search python". >> >> This is narrow-minded. I understand your POV, I really do (I use a >> debian-based system myself, and hate the way software installation works >> on any other system), but what you are saying cannot work in a general >> manner. > > > > IMHO we could have a command-line search feature ? la CPAN at PyPI like > what apt provides. > > $ pypi-search "foo" > > It exists, its called yolk, and it rocks :D > > > Tarek > > -- > Tarek Ziad? - Directeur Technique > INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 > Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage > 92210 Saint Cloud - France > Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 > http://www.ingeniweb.com - une soci?t? du groupe Alter Way > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Sat Sep 27 13:48:53 2008 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 27 Sep 2008 07:48:53 -0400 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DD8A4F.9030201@canterbury.ac.nz> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD8A4F.9030201@canterbury.ac.nz> Message-ID: <48DE1DA5.208@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Greg Ewing wrote: > Guido van Rossum wrote: > >> Yeah, but *is* dropping backward compatibility an option here? > > I'm not sure the concept of backward compatibility > makes sense here. The only kind of distutils replacement > I would be interested in would have a completely different > API, completely different structure and completely > different implementation. Anything less would fail to > fix the problems we want to fix. > > Given that, what would it even *mean* for the new tool > to be backward compatible with distutils? I think a tool which was willing to fake being distutils enough to extract information from existing 'setup()' calls would probably be enough, myself, so that: $ python .py setup.py would create the equivalent of PKG_INFO / EGG_INFO on disk, where it could then be used to drive the new installer. >> The same might be happen to distutils 2 in a few years. You are going >> to have to plan making it more maintainable. > > A modular structure with an easy-to-follow implementation > would certainly be part of the plan, I hope. And good tests and even better docs. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI3h2l+gerLs4ltQ4RAo1hAKCyxaoTUvr+IlbY0R8ZGZmwCvS4igCfbEVp 5LuQS4q13ciDMqzMvaw3YUA= =BxQ/ -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Sat Sep 27 15:16:02 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 27 Sep 2008 15:16:02 +0200 Subject: [Distutils] Distribute, what's next Message-ID: <94bdd2610809270616p10f8a183p7a991ef8d1f9cb7a@mail.gmail.com> Hello, I am trying to build a page now in Python.org wiki, that synthetizes the current state of packaging in Python, the problems, and what can be done. I'll work on it this week-end, and try to synthetize everything that has been said in the latest threads (and elsewhere). My goal is to gather in this single page everything in a high-level view, so hopefully, so maybe we can find the "path" But I am very Zope-oriented, so If some other people want to help me, I'll be hanging in #distutils on freenode (paris time) http://wiki.python.org/moin/Distribute ++ Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Sat Sep 27 15:19:56 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 27 Sep 2008 22:19:56 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DD8A4F.9030201@canterbury.ac.nz> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD8A4F.9030201@canterbury.ac.nz> Message-ID: <48DE32FC.2040908@ar.media.kyoto-u.ac.jp> Greg Ewing wrote: > I'm not sure the concept of backward compatibility > makes sense here. The only kind of distutils replacement > I would be interested in would have a completely different > API, completely different structure and completely > different implementation. Anything less would fail to > fix the problems we want to fix. I agree with your points (a new system would have a totally different design), but I believe this does not mean we have to break everything and force people to start from scratch. For example, in the case of numscons, I got all the scons features to build C code (automatic dependencies handling, emitters, configuration, etc...), while keeping the basic same setup.py files. Being 100 % compatible is not possible, because distutils is implementation defined; but many projects are not that complicated, and basic compatibility would be enough, no ? Maybe I don't realize the complexity (I really don't know anything about deployment issues for zope, web frameworks like django and co), but do many projects use all the arcane stuff ? By adding a new dummy command to distutils, which would hook to a brand new system (the one you are talking about), we could maybe enable a path forward ? Things like option handling, distribution: those things cannot be done in a backward compatible way, because it is broken by design. But a correct build system, with clean API, separated code and data: this can be done such as it would be usable from setup.py. Then, later, we could implement the installation/deployment part, and this part would not be backward compatible (and we could use something different than setup.py). cheers, David From tarek.ziade at ingeniweb.com Sat Sep 27 15:44:04 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Sat, 27 Sep 2008 15:44:04 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DE32FC.2040908@ar.media.kyoto-u.ac.jp> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD8A4F.9030201@canterbury.ac.nz> <48DE32FC.2040908@ar.media.kyoto-u.ac.jp> Message-ID: 2008/9/27 David Cournapeau > By adding a new dummy command to distutils, which would hook to a brand > new system (the one you are talking about), we could maybe enable a path > forward ? Things like option handling, distribution: those things cannot > be done in a backward compatible way, because it is broken by design. > But a correct build system, with clean API, separated code and data: > this can be done such as it would be usable from setup.py. Then, later, > we could implement the installation/deployment part, and this part would > not be backward compatible (and we could use something different than > setup.py). > This could be possible as long as we work on the current metadata definition. What would they miss for your dummy command to build everything even if it is another new application under the hood ? I mean, there's the code part, and there's the description part. Can an OS vendor for example is able to pick what he needs in a source distribution by looking at those metadata ? (me don't know) Imho, beyond the coding strategy, that is very important . Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From zooko at zooko.com Sat Sep 27 17:42:52 2008 From: zooko at zooko.com (zooko) Date: Sat, 27 Sep 2008 09:42:52 -0600 Subject: [Distutils] backwards compatibility for packaging tools Message-ID: When thinking about compatibility, keep in mind the distinction between two use cases: 1. You are using a tool (e.g. distutils or setuptools) to package your work. You might consider switching to a new tool, either one included in the Python Standard Library or a separately-shipped one, to build, package, and distribute your software. 2. You are re-using other people's work that they have packaged and distributed. You might consider switching to a new tool (either Python Standard or separate) to re-use their work. Backwards compatibility in the first case is not overwhelmingly important. Some people will be willing to entirely rewrite their setup.py scripts to use a new tool, other people will be willing to use a new tool only if they can keep using their old setup.py scripts, and still other people will continue to package and distribute their software with Python 2.5 and the distutils that came with it for the forseeable future (let's say, for the next 5 years). We can't prevent them from doing that, but also we don't need to persuade them to change tools in order to benefit from their work. Compatibility in the second case is overwhelmingly important. If you offer me a new tool for re-using other people's source code, and this new tool does *not* give me access to the thousands and thousands of Python packages that are already out there and the dozens of hundreds of new ones that are appearing every month, then this new tool is completely uninteresting to me. So we should focus on documenting and standardizing the metadata that allows code re-use -- the interface between the author of one package and the author of another package -- not the interface between the author of a package and the tool that he uses to build and distribute his own package. We've already got a pretty good start on this -- the distutils in Python 2.5 emits PKG-INFO and .egg-info in a format that is understood by all of the current crop of packaging tools (easy_install, pyinstall, distribute, yolk, bbfreeze, etc. etc. etc.). Also of course setuptools produces .egg-info metadata in a way that those tools can understand. We also got past the problem that we had for awhile that Linux distributions like Fedora and Debian were deleting those .egg-info files. They don't do that anymore. The next piece that is missing, from my experience in packaging Tahoe and its numerous Python dependencies [1], is for more tools to start emitting "requires" metadata in a compatible way. We already have a de jure standard for how spell "I require zope.interface" -- PEPs 314 and 345. However, I have never seen metadata of this format in the wild, and for all I know there are no tools that actually produce or consume this metadata and no packages that are actually labelled with this metadata. We also have a de facto standard that is widely used by a large and growing library of packages and is supported by a large and growing set of tools -- the way that setuptools spells "I require zope.interface" in its .egg-info files. So, I have a simple and urgent request: Extend distutils so that when the author of package passes install_requires=['zope.interface'] to distutils.core.setup(), then it emits an entry named "requires" with body "zope.interface" in the resulting .egg-info. That's all. It won't hurt, and it will probably help quite a lot to facilitate interoperation of future Python packaging tools. Regards, Zooko [1] http://mail.python.org/pipermail/distutils-sig/2008-September/ 010081.html --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From greg.ewing at canterbury.ac.nz Sun Sep 28 01:25:41 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 28 Sep 2008 11:25:41 +1200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DE1DA5.208@palladion.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD8A4F.9030201@canterbury.ac.nz> <48DE1DA5.208@palladion.com> Message-ID: <48DEC0F5.3090207@canterbury.ac.nz> Tres Seaver wrote: > I think a tool which was willing to fake being distutils enough to > extract information from existing 'setup()' calls would probably be > enough, myself, so that: > > $ python .py setup.py > > would create the equivalent of PKG_INFO / EGG_INFO on disk, where it > could then be used to drive the new installer. It should be possible to emulate the behaviour of distutils to some extent using the new tool, so it can be invoked by making a setup() call. I don't think it could be done by statically introspecting a setup.py file, since in many cases the only way to find out what a setup.py does is to execute it. -- Greg From greg.ewing at canterbury.ac.nz Sun Sep 28 01:51:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 28 Sep 2008 11:51:00 +1200 Subject: [Distutils] backwards compatibility for packaging tools In-Reply-To: References: Message-ID: <48DEC6E4.8020608@canterbury.ac.nz> zooko wrote: > So we should focus on documenting and standardizing the metadata that > allows code re-use -- the interface between the author of one package > and the author of another package -- not the interface between the > author of a package and the tool that he uses to build and distribute > his own package. That makes sense to me. -- Greg From ianb at colorstudy.com Sun Sep 28 05:05:09 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 27 Sep 2008 22:05:09 -0500 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DE1DA5.208@palladion.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD8A4F.9030201@canterbury.ac.nz> <48DE1DA5.208@palladion.com> Message-ID: <48DEF465.5040400@colorstudy.com> Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Greg Ewing wrote: >> Guido van Rossum wrote: >> >>> Yeah, but *is* dropping backward compatibility an option here? >> I'm not sure the concept of backward compatibility >> makes sense here. The only kind of distutils replacement >> I would be interested in would have a completely different >> API, completely different structure and completely >> different implementation. Anything less would fail to >> fix the problems we want to fix. >> >> Given that, what would it even *mean* for the new tool >> to be backward compatible with distutils? > > I think a tool which was willing to fake being distutils enough to > extract information from existing 'setup()' calls would probably be > enough, myself, so that: > > $ python .py setup.py > > would create the equivalent of PKG_INFO / EGG_INFO on disk, where it > could then be used to drive the new installer. pyinstall does exactly this, like: python -c "import setuptools; __file__='setup.py'; execfile('setup.py')" egg_info well, it's slightly more involved, because the Package.egg-info/ directory is written in the base of the package, which might be ./Package.egg-info, or ./src/Package.egg-info -- to avoid this I add --egg-base=pyinstall-egg-info so that the egg info always goes in the same location. Anyway, setuptools already fakes distutils in exactly the way you describe. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ianb at colorstudy.com Sun Sep 28 05:07:25 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 27 Sep 2008 22:07:25 -0500 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DD86D4.50704@canterbury.ac.nz> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD86D4.50704@canterbury.ac.nz> Message-ID: <48DEF4ED.50802@colorstudy.com> Greg Ewing wrote: > Phillip J. Eby wrote: > >> the domain knowledge embedded in the distutils are of fairly limited >> scope and kind: >> >> * Extension building, compile/link options and defines >> * Wildly-differing installation path schemes across platforms >> * Platform distribution formats like bdist_rpm, bdist_wininst, and >> bdist_msi > > Seems to me that if there were a well-defined API for > plugging in platform-dependent modules, it shouldn't be > too hard to find people willing to contribute modules > for the platforms they're familiar with. For the most part I don't think you'd even need plugins, these platform-dependent tools could just be independent, maybe working more like: make_wininst SomePackage/ They'd read the metadata about the package, and just do their work however they wanted. Plugins get complicated compared to stand-alone tools. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From david at ar.media.kyoto-u.ac.jp Sun Sep 28 07:27:01 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 28 Sep 2008 14:27:01 +0900 Subject: [Distutils] backwards compatibility for packaging tools In-Reply-To: References: Message-ID: <48DF15A5.3010103@ar.media.kyoto-u.ac.jp> zooko wrote: > When thinking about compatibility, keep in mind the distinction > between two use cases: > > 1. You are using a tool (e.g. distutils or setuptools) to package > your work. You might consider switching to a new tool, either one > included in the Python Standard Library or a separately-shipped one, > to build, package, and distribute your software. > > 2. You are re-using other people's work that they have packaged and > distributed. You might consider switching to a new tool (either > Python Standard or separate) to re-use their work. > > Backwards compatibility in the first case is not overwhelmingly > important. Some people will be willing to entirely rewrite their > setup.py scripts to use a new tool, other people will be willing to > use a new tool only if they can keep using their old setup.py scripts, > and still other people will continue to package and distribute their > software with Python 2.5 and the distutils that came with it for the > forseeable future (let's say, for the next 5 years). We can't prevent > them from doing that, but also we don't need to persuade them to > change tools in order to benefit from their work. I don't think it is accurate. I have no reason to doubt it is true in your case and many other people's case, but I think you have to be careful when saying 2 is "obviously" more important than 1. My impression is that 1 is important for people who invested a lot in distutils, have a lot of distutils-based extensions. It depends on the complexity of the setup.py and their build tools around. Again: numpy.distutils is as big a distutils. For numpy's case, a new well design would make most of those obsolete, but is it true for any package which has heavily invested in distutils ? >From your description, I understand that you care about dependencies, this kind of things. I know *I* don't care about that for the most part, and still I would welcome a new distutils. For me, something with a sane build tool (built around a DAG, not a sequence of commands) is what matters the most. Something which enables easier packaging by OS vendors matters. So I would kindly ask to be careful when taking into account what matters and what not. cheers, David From ziade.tarek at gmail.com Sun Sep 28 14:55:11 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 28 Sep 2008 14:55:11 +0200 Subject: [Distutils] PEP for distutils Message-ID: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> Hello Let's write PEPs ! I started to write a new PEP (well a wiki page in fact...) that describes a new package called "pypi" that would be dedicated to package registering and uploading mechanisms. It would also provide enhancements like a proper password hash, or deepers metadata controls http://wiki.python.org/moin/A_new_pypi_module Any opinions about this PEP ? I tried to include all the problems people are having with register and upload. I linked it here in the page where I am trying to summarize the current state of distutils http://wiki.python.org/moin/Distribute (work in progress) Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Sun Sep 28 15:08:40 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 28 Sep 2008 09:08:40 -0400 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DD3930.4060907@colorstudy.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD3930.4060907@colorstudy.com> Message-ID: <7.0.1.0.0.20080928084330.02533980@telecommunity.com> At 03:34 PM 9/26/2008, Ian Bicking wrote: >I'm not sure I agree with removing setup.py, though I dislike how it >functions currently -- that you kind of have to poke it from the >side to get basic information out of it, instead of something more >declarative. In some ways it's the distutils/setuptools interface >that most people actually use -- pyinstall for instance only calls >setup.py in subprocesses to install the package, and never touches >it directly. So anything that acts fairly similar to the current >setup.py's will be compatible. This is part of why I don't think >the backward compatibility issue is as difficult as Guido suggests. > >Though... really if pyinstall could easily detect a new source >format and only setup.py with the old source format, it wouldn't be >hard to support both. There does need to be room for custom code, >but mostly for the build/compilation process. The main reason to drop setup.py as the interface is to prevent people writing their own crazy command line "parsing" (e.g. probing sys.argv directly) and from trying to install stuff or do other things unconditionally in code that shouldn't be doing anything but configuration. Having a 'configure.py' or some such would be a clear break with the past in this respect. That is, a configure.py script would be there if and only if you needed dynamic metadata or build-spec generation, and it shouldn't be designed to run as a user-run script, but instead be invoked in a restricted way by build and install tools. It should also be required to be sandboxable, in the sense that it should write to no files outside the build area. It should also be well-understood that it is not to unconditionally import dependencies, nor assume that failure to import a dependency means the dependency isn't present. (And if you don't have any dynamic metadata requirements, the package metadata and build specs should be able to be defined in non-executable configuration files, with no need for a configure.py.) IOW, the main reason (IMO) to drop backward compatibility is precisely to get rid of setup.py and thus all of the crazy hairy things people do with it that they've got no business doing (because they can't accomplish what they need any other way, without first becoming an expert at distutils internals). Meanwhile, legitimate *build* extensions can and should be restrained to *building*, rather than installing. We can define a build directory layout for installers to use for actual installation, including various platform-specific elements (such as icons/menu items, registry stuff, post-install scripts, etc.) which it's then the responsibility of individual installer programs to handle. I guess that means that we would need to spec out: 1. Source tree layout 2. Build configuration files/build processing 3. configure.py 4. sdist format & metadata 5. build tree layout (for build code to build and installers to install from) But the basic idea is that the stdlib supplies the build-and-distribute framework, package authors can have build extensions, but only install programs can actually install or uninstall anything. The build layout should be flexible enough to allow things like dumping Debian or RPM or MSI metadata in there, for "install" tools (which might technically be platform bdist tools) to pull out. This is a rather important separation of duties, because right now bdist tools are responsible for build management and tricking "install" commands in order to do their work. This makes it too hard for people to write bdist tools. (And you can't even *think* about custom install tools with the distutils now.) These elements would make it possible for the distutils to be divided up into more maintainable slices, that are owned by a much larger group of people, with much lower probability of burnout. Given a stable build layout, install tools aren't likely to change much. Given a stable source and build layout, build tools aren't likely to change much. The hairiest bits (extension/compiler stuff) will have to be in stdlib anyway, and are fairly isolated within the overall process. (I also think we should encourage the model pioneered in buildout and setuptools of keeping build extensions as separate packages from the things they're used to build, but I could be convinced that making it *possible* (but not necessarily easy) to include a build extension within a package that's using it, could *perhaps* be a reasonable idea.) From tarek.ziade at ingeniweb.com Sun Sep 28 15:46:51 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Sun, 28 Sep 2008 15:46:51 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <48DEF4ED.50802@colorstudy.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD86D4.50704@canterbury.ac.nz> <48DEF4ED.50802@colorstudy.com> Message-ID: 2008/9/28 Ian Bicking > Greg Ewing wrote: > >> Phillip J. Eby wrote: >> >> the domain knowledge embedded in the distutils are of fairly limited >>> scope and kind: >>> >>> * Extension building, compile/link options and defines >>> * Wildly-differing installation path schemes across platforms >>> * Platform distribution formats like bdist_rpm, bdist_wininst, and >>> bdist_msi >>> >> >> Seems to me that if there were a well-defined API for >> plugging in platform-dependent modules, it shouldn't be >> too hard to find people willing to contribute modules >> for the platforms they're familiar with. >> > > For the most part I don't think you'd even need plugins, these > platform-dependent tools could just be independent, maybe working more like: > > make_wininst SomePackage/ > > They'd read the metadata about the package, and just do their work however > they wanted. Plugins get complicated compared to stand-alone tools. +1 this is exactly the same thing for uploading or registering a package at PyPI. We don't need to use command plugins for that as long as the medata can be read to do the job. > > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Sun Sep 28 16:58:02 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 28 Sep 2008 16:58:02 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD86D4.50704@canterbury.ac.nz> <48DEF4ED.50802@colorstudy.com> Message-ID: <94bdd2610809280758y290eee26o544eef2b1b27fcff@mail.gmail.com> On Sun, Sep 28, 2008 at 3:46 PM, Tarek Ziade wrote: > > > 2008/9/28 Ian Bicking > >> Greg Ewing wrote: >> >>> Phillip J. Eby wrote: >>> >>> the domain knowledge embedded in the distutils are of fairly limited >>>> scope and kind: >>>> >>>> * Extension building, compile/link options and defines >>>> * Wildly-differing installation path schemes across platforms >>>> * Platform distribution formats like bdist_rpm, bdist_wininst, and >>>> bdist_msi >>>> >>> >>> Seems to me that if there were a well-defined API for >>> plugging in platform-dependent modules, it shouldn't be >>> too hard to find people willing to contribute modules >>> for the platforms they're familiar with. >>> >> >> For the most part I don't think you'd even need plugins, these >> platform-dependent tools could just be independent, maybe working more like: >> >> make_wininst SomePackage/ >> >> They'd read the metadata about the package, and just do their work however >> they wanted. Plugins get complicated compared to stand-alone tools. > > > > +1 > > this is exactly the same thing for uploading or registering a package at > PyPI. We don't need to use command plugins for that as long as > the medata can be read to do the job. > What about gently splitting setup.py in two ? Let's define in distutils a PackageInfo class, that provides the metadata, with default values when applicable. Let's put this class definition in a file called package_info.py or even at the root in __init__. Setup.py could use it instead of asking these metadata as arguments for setupt() >From there, any tool could use these metadata without calling the distutils commands. I mean, it's a really simple change and it could be a first step to split the concerns in distutils Tarek > > > > > >> >> >> >> -- >> Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > > > -- > Tarek Ziad? - Directeur Technique > INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 > Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage > 92210 Saint Cloud - France > Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 > http://www.ingeniweb.com - une soci?t? du groupe Alter Way > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.chauvat at logilab.fr Sun Sep 28 19:30:11 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Sun, 28 Sep 2008 19:30:11 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> Message-ID: <20080928173011.GA4852@volans.logilab.fr> Hi, On Sat, Sep 27, 2008 at 05:18:36PM +0900, David Cournapeau wrote: > > Do you really think every single Python package in PyPI deserves to be > > packaged for every distribution? I don't. How do I make a difference? > > When I need something I download it. When I find it really useful and > > plan on using it I package it. Many others are behaving in the same > > way and the result is "apt-cache search python". > > For example, installing from sources on most other systems is frowned > upon, and rightfully so, because it is even more complicated to do than > on a decent linux system. It is painful because either the OS makes it > terribly difficult (windows), or because you have antique/not well > supported toolsets (old Solaris, etc...). If you mainly use only one OS, > you just can't understand the pain. I have used many operating systems and I still do from time to time. I frown upon anything that has to be done more than once by hand, including installing things from source. > I am not saying that python plugins must be THE deployment system, but > that it has to be one system, because plugins systems are as pervasive > on other OS as .deb are on debian. So we should think about what kind of > things python core can provide to help other tools to either build > "native" packages or eggs, and not having a big pile of code which mix > everything. As Matthias Klose mentioned earlier, a lot of those formats > share common requirements. We should talk about those instead of saying > my package is bigger than yours. Sure, the package system I use is bigger than yours (if you are not using Debian), but that's not my main point and insisting on it would turn into an endless flame war. Can we focus on something else? You call me narrow minded, but I pretend to understand why people came up with distutils/setuptools/eggs etc. I have been there. I felt the need for a tool to easily manage systems and install dependencies. I started writing one myself. Then I discovered Debian and stopped using the other tools I had. Problem solved. For me. Now that more and more people are using Python and computers, more and more people feel the same need. Not everyone can solve his problem by dropping what he has and adopting Debian. I am well aware of that. I am not trying to convince people to adopt Debian, I am trying to explain to people who probably have not used it or not developed a lot of packages for a large system: * how I do it very efficiently, * why it suits my needs, * why other people trying to make "python plugins systems" are making my work more difficult when it becomes the only/main distribution channel. I repeat. I am not trying to force other people to use Debian, I am trying to get other people not to force me to use tools I do not need (distutils, etc) for I have good ones already (debian packages). > > As usual "user != developer". For someone not using Debian: just be > > happy with whatever tool you choose to use. For someone not an > > experienced Debian developer: just wait for someone to do the work you > > want to benefit from, or learn to do it yourself and get it done. > > And how do you distribute new versions of your package ? You wait for > debian to package it correctly ? For fast-moving packages, debian are > not the ultimate solution, far from it. As I said in an other email, I now get python code from a mercurial repository to a Debian repository then to a production system running Debian in a matter of minutes. That's fast enough for me. > I mean, it is not like OpenSuse build service, Ubuntu ppa systems > came from nowhere. There is a need for softwares developers to > distribute themselves the software for newer versions, and in that > case, the native system (at least used "officialy") simply is not > appropriate. http://ftp.logilab.org/debian/sid/ is not official. Anyone wanting to make one's own repository can do it. Anyone wanting to use some repository that's "non-official" as a source of packages for one's system can do it. I have talked a lot. I think I'll stop drowning this list with my messages. Here is my conclusion. People can do whatever they think is good for them! Just do not use easy_install/whatever as the only distribution channel. If you use it, put a standard tarball with an up-to-date README detailing installation and dependencies on the same download page. Please. -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From ziade.tarek at gmail.com Mon Sep 29 00:32:24 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 29 Sep 2008 00:32:24 +0200 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <7.0.1.0.0.20080928084330.02533980@telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD3930.4060907@colorstudy.com> <7.0.1.0.0.20080928084330.02533980@telecommunity.com> Message-ID: <94bdd2610809281532t47ecebb4u8763969e4993702e@mail.gmail.com> On Sun, Sep 28, 2008 at 3:08 PM, Phillip J. Eby wrote: > > Meanwhile, legitimate *build* extensions can and should be restrained to > *building*, rather than installing. We can define a build directory layout > for installers to use for actual installation, including various > platform-specific elements (such as icons/menu items, registry stuff, > post-install scripts, etc.) which it's then the responsibility of individual > installer programs to handle. > > I guess that means that we would need to spec out: > > 1. Source tree layout > 2. Build configuration files/build processing > 3. configure.py > 4. sdist format & metadata > 5. build tree layout (for build code to build and installers to install > from) And what about documentation ? Do we want to have the documentation in the build directory, or this could be a specific element defined in the package ? For instance, the long_description metadata is the string where we all push our documentation so it can be seen at PyPI, but we could have a better way to point the package documentation to PyPI or to other platforms. I mean, look at this long_description here: http://pypi.python.org/pypi/zc.buildout/ It's long for sure, and it is a merge of several files. We could probably do better to describe the doc. Secondly, What would you have in configure.py ? Is this where you define the configuration for your package or point an .ini or .cfg file ? When a package uses a configuration file like an ini-based file, I'd say the best place for it is under /etc for Debian for instance, and probably inside the package itself for platform like Windows, So don't we miss a list of such files ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Mon Sep 29 13:46:15 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 29 Sep 2008 20:46:15 +0900 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080928173011.GA4852@volans.logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> Message-ID: <48E0C007.70702@ar.media.kyoto-u.ac.jp> Nicolas Chauvat wrote: > > Sure, the package system I use is bigger than yours (if you are not > using Debian), but that's not my main point and insisting on it would > turn into an endless flame war. Can we focus on something else? Sure, that's what I am interested in :) > > You call me narrow minded, but I pretend to understand why people came > up with distutils/setuptools/eggs etc. I have been there. I felt the > need for a tool to easily manage systems and install dependencies. I > started writing one myself. Then I discovered Debian and stopped using > the other tools I had. Problem solved. For me. The problem is that debian packages are not always the solution (even on debian systems). Two big problems are: - installation as non root - developers deploying their own software on a custom debian repository does not scale at all. We have to think about those user-case. > I repeat. I am not trying to force other people to use Debian, I am > trying to get other people not to force me to use tools I do not need > (distutils, etc) for I have good ones already (debian packages). This part I don't understand: distutils and debian dpkg-related tools do have some overlap, but they are not a replacement from each other. In particular, distutils manages the details of python C extensions, etc... Where distutils failed big time IMHO is that it made it more difficult for you (or for me for that matter), not easier. Autotools did help packagers; a distutils successor should be able to help without getting in the way. For example, by providing simple discoverable meta-data. Wouldn't it help a debian packager to have a simple description of the meta-data for the dependencies ? Wouldn't it help if it was easy to set data_dir, doc_dir, etc... according to the FHS ? Autotools "packages" are relatively easy to package; I don't see why we could not achieve the same for python packages. cheers, David From david at ar.media.kyoto-u.ac.jp Mon Sep 29 13:51:21 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 29 Sep 2008 20:51:21 +0900 Subject: [Distutils] [Fwd: Re: Annoucing distribute project] In-Reply-To: <7.0.1.0.0.20080928084330.02533980@telecommunity.com> References: <48DBC7FD.8000809@latte.ca> <94bdd2610809251359l50b1ab46s24c9a7683455ef9@mail.gmail.com> <20080926154640.5B9B83A4072@sparrow.telecommunity.com> <48DD3930.4060907@colorstudy.com> <7.0.1.0.0.20080928084330.02533980@telecommunity.com> Message-ID: <48E0C139.7090403@ar.media.kyoto-u.ac.jp> Phillip J. Eby wrote: > 1. Source tree layout > 2. Build configuration files/build processing > 3. configure.py > 4. sdist format & metadata > 5. build tree layout (for build code to build and installers to > install from) Could you explain more why you want do spec out tree layout, because I don't understand. To me, it should be possible to set where different things go from a description (ala autoconf: data, shared, etc... except we would have to take into account non-unix OS of course), but forcing a particular build tree layout and source tree ? What does this give us ? > These elements would make it possible for the distutils to be divided > up into more maintainable slices, that are owned by a much larger > group of people, with much lower probability of burnout. Given a > stable build layout, install tools aren't likely to change much. > Given a stable source and build layout, build tools aren't likely to > change much. The hairiest bits (extension/compiler stuff) will have > to be in stdlib anyway, and are fairly isolated within the overall > process. Yes, this is a critical point. Installation, deployment and build are linked, in the sense the former needs information from the later, but they have very different requirements otherwise. cheers, David From ziade.tarek at gmail.com Mon Sep 29 14:09:15 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 29 Sep 2008 14:09:15 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48E0C007.70702@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> On Mon, Sep 29, 2008 at 1:46 PM, David Cournapeau wrote: > Nicolas Chauvat wrote: >> >> Sure, the package system I use is bigger than yours (if you are not >> using Debian), but that's not my main point and insisting on it would >> turn into an endless flame war. Can we focus on something else? > > Sure, that's what I am interested in :) > >> >> You call me narrow minded, but I pretend to understand why people came >> up with distutils/setuptools/eggs etc. I have been there. I felt the >> need for a tool to easily manage systems and install dependencies. I >> started writing one myself. Then I discovered Debian and stopped using >> the other tools I had. Problem solved. For me. > > The problem is that debian packages are not always the solution (even on > debian systems). Two big problems are: > - installation as non root > - developers deploying their own software on a custom debian > repository does not scale at all. > > We have to think about those user-case. >> I repeat. I am not trying to force other people to use Debian, I am >> trying to get other people not to force me to use tools I do not need >> (distutils, etc) for I have good ones already (debian packages). > > This part I don't understand: distutils and debian dpkg-related tools do > have some overlap, but they are not a replacement from each other. In > particular, distutils manages the details of python C extensions, etc... > > Where distutils failed big time IMHO is that it made it more difficult > for you (or for me for that matter), not easier. Autotools did help > packagers; a distutils successor should be able to help without getting > in the way. For example, by providing simple discoverable meta-data. > Wouldn't it help a debian packager to have a simple description of the > meta-data for the dependencies ? Wouldn't it help if it was easy to set > data_dir, doc_dir, etc... according to the FHS ? Autotools "packages" > are relatively easy to package; I don't see why we could not achieve the > same for python packages. That is exactly what was brought in the other thread in distutils-SIG, providing the package metadata in a simple way for os-vendors, without having to deal with things like setup.py Then having third party applications that knows how to use them to install things in debian, or whatever the system is. Now, the question is, what would debian miss in here to install: http://www.python.org/dev/peps/pep-0345/ If you can come up with a list of missing elements, we could probably start to work on a PEP together. Tarek > > cheers, > > David > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From nicolas.chauvat at logilab.fr Mon Sep 29 17:07:37 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Mon, 29 Sep 2008 17:07:37 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <48E0C007.70702@ar.media.kyoto-u.ac.jp> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> Message-ID: <20080929150737.GB8032@volans.logilab.fr> On Mon, Sep 29, 2008 at 08:46:15PM +0900, David Cournapeau wrote: > The problem is that debian packages are not always the solution (even on > debian systems). Two big problems are: > - installation as non root True, this is a common use case. Unfortunately, this use case is common because users get stuck on a system with no way to get things installed on it. No admin rights, no way to get the contracted sysadmin to install things, etc. So far I have not found a good solution to this problem when I had to face it. Tried several things including a "user-side gentoo". Did not work well. I see why easy_install could be a solution. > - developers deploying their own software on a custom debian > repository does not scale at all. Why do you think that? > Where distutils failed big time IMHO is that it made it more difficult > for you (or for me for that matter), not easier. Autotools did help > packagers; a distutils successor should be able to help without getting > in the way. Yes. > For example, by providing simple discoverable meta-data. Wouldn't > it help a debian packager to have a simple description of the > meta-data for the dependencies ? Wouldn't it help if it was easy to > set data_dir, doc_dir, etc... according to the FHS ? Autotools > "packages" are relatively easy to package; I don't see why we could > not achieve the same for python packages. Good. Let's do it. Do you agree with Tarek that writing a PEP is a good approach? -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From david at ar.media.kyoto-u.ac.jp Mon Sep 29 17:29:31 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 30 Sep 2008 00:29:31 +0900 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080929150737.GB8032@volans.logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <20080929150737.GB8032@volans.logilab.fr> Message-ID: <48E0F45B.40104@ar.media.kyoto-u.ac.jp> Nicolas Chauvat wrote: >> - developers deploying their own software on a custom debian >> repository does not scale at all. > > Why do you think that? With N vendors packaging P packages, it is not hard to imagine that many package will overlap, and since they are independently built, it will fail pretty quickly for people who use several of those repositories. Package distribution is very centralized with rpm/deb. That's a big shortcoming of the current software distribution on Linux IMHO. > > Good. Let's do it. Do you agree with Tarek that writing a PEP is a > good approach? I think there is already enough material from people familiar with the problems in the last few days emails on this ML to write something semi-formalized, as a basis for further discussion, yes. cheers, David From nicolas.chauvat at logilab.fr Tue Sep 30 10:42:53 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Tue, 30 Sep 2008 10:42:53 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> Message-ID: <20080930084253.GF4494@volans.logilab.fr> Hi, On Mon, Sep 29, 2008 at 02:09:15PM +0200, Tarek Ziad? wrote: > That is exactly what was brought in the other thread in distutils-SIG, > providing the package metadata in a simple way for os-vendors, without > having to deal with things like setup.py > > Then having third party applications that knows how to use them > to install things in debian, or whatever the system is. > > Now, the question is, what would debian miss in here to install: > > http://www.python.org/dev/peps/pep-0345/ > > If you can come up with a list of missing elements, we could > probably start to work on a PEP together. I started a thread on debian-python to ask for help: http://lists.debian.org/debian-python/2008/09/msg00025.html Here the answer from Josselin Mouette who is the author of python-support, a tool that dramatically eases the packaging of Python code for Debian. --------------------------------------------------------------------- Le lundi 29 septembre 2008 ? 15:12 +0200, Nicolas Chauvat a ?crit : > Here is where we stand today: >http://mail.python.org/pipermail/distutils-sig/2008-September/010126.html This looks like a step in the right direction if we want to generate inter-module dependencies. Most things defined in the PEP will not be useful for packaging, except for making something like a dh_make_python almost trivial to write. The one thing that we'd almost certainly use is the Requires and Provides fields. However you should be careful with the notion of version. It is nice to have a lot of flexibility in specifying versioned dependencies, but the more stuff the norm allows, the more complicated it will be to translate this into inter-package dependencies. For example, if you require a minimal version of 1.4, you can translate this to a package version of 1.4; it is a bit hackish but will work if you handle epochs correctly. But if the package you depend on has a Provides: blah (1.4), you have no way to map that to a dependency, because you can't know what other versions of the package will provide. In all cases, it will be necessary to manually add shlibs-like information to the packages; they could be partly autogenerated like symbol files, but you need a mapping between provided modules and the first version of the package that provides it. --------------------------------------------------------------------- Good to see this is moving forward :) -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From ziade.tarek at gmail.com Tue Sep 30 14:05:08 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 14:05:08 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080930084253.GF4494@volans.logilab.fr> References: <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> Message-ID: <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> On Tue, Sep 30, 2008 at 10:42 AM, Nicolas Chauvat wrote: > For example, if you require a minimal version of 1.4, you can > translate this to a package version of 1.4; it is a bit hackish but > will work if you handle epochs correctly. But if the package you > depend on has a Provides: blah (1.4), you have no way to map that to a > dependency, because you can't know what other versions of the package > will provide. I am not sure to fully understand, could you provide a real-word example ? > > In all cases, it will be necessary to manually add shlibs-like > information to the packages; they could be partly autogenerated like > symbol files, but you need a mapping between provided modules and the > first version of the package that provides it. Is this related ? http://lists.debian.org/debian-dpkg/2006/01/msg00118.html From ziade.tarek at gmail.com Tue Sep 30 14:10:23 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 14:10:23 +0200 Subject: [Distutils] Distutils Sprint Message-ID: <94bdd2610809300510v46ed46dbxe2c95ffe9590b458@mail.gmail.com> Hello In order to continue the effort started here. We are organizing a distutils "PEP sprint" with people that works differently to deliver Python applications. http://wiki.python.org/moin/DistributeSprint_#1 Some Python developers from the Debian world and Scons specialist will join. I'll bring the zc.buildout/setuptools point of view. I would like to come up with enough material to write a meta-PEP and a series of PEP. These PEPs will then be submitted here for further discussions. Please join ! Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From zooko at zooko.com Tue Sep 30 14:38:06 2008 From: zooko at zooko.com (zooko) Date: Tue, 30 Sep 2008 06:38:06 -0600 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> Message-ID: <2991F570-F093-4B9C-8147-1C9D8F603FB4@zooko.com> On Sep 29, 2008, at 6:09 AM, Tarek Ziad? wrote: > Now, the question is, what would debian miss in here to install: > > http://www.python.org/dev/peps/pep-0345/ It really seems to me that PEP-345's specification of dependency metadata is the wrong starting point. There are not, to my knowledge, any Python packages in existence which use this form of dependency metadata, and there are not, to my knowledge, any Python tools which are capable of producing or consuming it. In contrast, there are a large number of packages already in existence that declare their dependencies in their EGG-INFO/ depends.txt. There are many tools -- I don't even know how many -- which already know how to produce and consume that dependency metadata. In fact, one such tool has a patch that I contributed myself to use that dependency metadata to automatically produce the Debian "Depends:" information [1]. I learned yesterday that there is a tool by David Malcolm to do likewise for Fedora RPM packages. We would gain power by continuing to use the format that is already implemented and deployed, instead of asking everyone to switch to a different format. So it seems like the next step is to write a PEP that supercedes the parts of PEP-345 which are about dependency metadata and instead says that the standard way to encode Python dependency metadata is in the EGG-INFO/requires.txt file. Regards, Zooko [1] https://code.launchpad.net/~astraw/stdeb/autofind-depends --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From ziade.tarek at gmail.com Tue Sep 30 15:18:43 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 15:18:43 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <2991F570-F093-4B9C-8147-1C9D8F603FB4@zooko.com> References: <48D00408.50705@simplistix.co.uk> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <2991F570-F093-4B9C-8147-1C9D8F603FB4@zooko.com> Message-ID: <94bdd2610809300618w410ccf37sb0926e58f7f785c@mail.gmail.com> On Tue, Sep 30, 2008 at 2:38 PM, zooko wrote: > On Sep 29, 2008, at 6:09 AM, Tarek Ziad? wrote: > >> Now, the question is, what would debian miss in here to install: >> >> http://www.python.org/dev/peps/pep-0345/ > > It really seems to me that PEP-345's specification of dependency metadata is > the wrong starting point. > > There are not, to my knowledge, any Python packages in existence which use > this form of dependency metadata, and there are not, to my knowledge, any > Python tools which are capable of producing or consuming it. > > In contrast, there are a large number of packages already in existence that > declare their dependencies in their EGG-INFO/depends.txt. There are many > tools -- I don't even know how many -- which already know how to produce and > consume that dependency metadata. > > In fact, one such tool has a patch that I contributed myself to use that > dependency metadata to automatically produce the Debian "Depends:" > information [1]. I learned yesterday that there is a tool by David Malcolm > to do likewise for Fedora RPM packages. > > We would gain power by continuing to use the format that is already > implemented and deployed, instead of asking everyone to switch to a > different format. The point is not to switch to a different format but to make sure: - we are able to read it without a setup.py magic call - we do have everything needed in these metadata for OS-vendors to work with the package otherwise propose some extensions > > So it seems like the next step is to write a PEP that supercedes the parts > of PEP-345 which are about dependency metadata and instead says that the > standard way to encode Python dependency metadata is in the > EGG-INFO/requires.txt file. > I would go further and say that we shouldn't have to run a command to generate the EGG-INFO of PKG-INFO or wathever, they should be avalaible in the package, directly, in a flat file. maybe in a "package_info.py" file I don't know or a .cfg file. But we shouldn't depend on a setup.py command call to read them or on a directory built by a command. That is one simple evolution I'd like to propose in the PEP I am working on. Regards, > Regards, > > Zooko > > [1] https://code.launchpad.net/~astraw/stdeb/autofind-depends > --- > http://allmydata.org -- Tahoe, the Least-Authority Filesystem > http://allmydata.com -- back up all your files for $5/month -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ziade.tarek at gmail.com Tue Sep 30 15:49:22 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 15:49:22 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <1222780673.4166.55.camel@shizuru> References: <20080917171851.GA27261@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> Message-ID: <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> On Tue, Sep 30, 2008 at 3:17 PM, Josselin Mouette wrote: > Le mardi 30 septembre 2008 ? 14:05 +0200, Tarek Ziad? a ?crit : >> On Tue, Sep 30, 2008 at 10:42 AM, Nicolas Chauvat >> wrote: >> > For example, if you require a minimal version of 1.4, you can >> > translate this to a package version of 1.4; it is a bit hackish but >> > will work if you handle epochs correctly. But if the package you >> > depend on has a Provides: blah (1.4), you have no way to map that to a >> > dependency, because you can't know what other versions of the package >> > will provide. >> >> I am not sure to fully understand, could you provide a real-word example ? > > Let's say you have module bar, contained in the package python-bar. The > last version is 1.4. After that version, it is decided to distribute it > in the same tarball as module bar. It is therefore moved to the package > python-foo, which is at version 1.2. In this case, you can specify in > the metadata : > Provides: foo > Provides: bar (1.4) > This is the typical use case for versioned provides. > > Let's say application baz requires module bar with minimal version 1.3, > it will have as dependency: > Requires: bar >= 1.3 > This way it will be happy to find the versioned provides if module foo > is installed, and everyone is happy. Well, except that, if you try to > build a package of baz, there is no way to express correctly that you > depend on python-bar (>= 1.3) or python-foo (>= 1.2). > > This is why I'd prefer to have versioned provides simply not part of the > specification. > The "Obsoletes" info could be used maybe. But the main problem I can see is that in any case several versions of the same module can be needed to build one application. That is what tools like zc.buildout or virtualenv exists : they are building an isolated environment where they install the packages so a given Python application can run. In other words the problem we have today with an OS-based installation is that you cannot really have two versions of the same package installed, that would make happy two Python applications. The setuptools project has partly improved this by providing a way to install several version of the same package in Python and give a way to select which one is active. >From your point of view, how could we solve it at Debian level ? to kind of isolate a group of packages that fit the needs of one given application ? (btw A recent change it Python has allowed us to define per-user site-packages http://mail.python.org/pipermail/python-dev/2008-January/076108.html) >> Is this related ? http://lists.debian.org/debian-dpkg/2006/01/msg00118.html > > Yes. The thread you point to did not let to something being actually > implemented, because at that moment, we lacked the necessary metadata. > Since then, setuptools appeared, but it does not provide it in a sane > way and it is not universal. Which is why I'm interested into the > metadata format that's discussed here. > > From this metadata, we will be able to generate some files that express > what is provided and the required version. Something like: > foo 1.0-1 > bar 1.2~beta3 > > This way, if another package requires foo (> 1.1) and bar (without a > version requirement), we can convert this dependency into: > python-foo (>= 1.1), python-foo (>= 1.2~beta3) > which can then be factorized, of course. > Interesting.. That would mean you would do version conflict resolution at the OS level, That makes me think about the previous point: how two applications that use conflicting versions that are not comptabile with each other (you have to choose one of them) can cohabit ? Cheers > Cheers, > -- > .''`. > : :' : We are debian.org. Lower your prices, surrender your code. > `. `' We will add your hardware and software distinctiveness to > `- our own. Resistance is futile. > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From joss at debian.org Tue Sep 30 15:17:53 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 15:17:53 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> Message-ID: <1222780673.4166.55.camel@shizuru> Le mardi 30 septembre 2008 ? 14:05 +0200, Tarek Ziad? a ?crit : > On Tue, Sep 30, 2008 at 10:42 AM, Nicolas Chauvat > wrote: > > For example, if you require a minimal version of 1.4, you can > > translate this to a package version of 1.4; it is a bit hackish but > > will work if you handle epochs correctly. But if the package you > > depend on has a Provides: blah (1.4), you have no way to map that to a > > dependency, because you can't know what other versions of the package > > will provide. > > I am not sure to fully understand, could you provide a real-word example ? Let?s say you have module bar, contained in the package python-bar. The last version is 1.4. After that version, it is decided to distribute it in the same tarball as module bar. It is therefore moved to the package python-foo, which is at version 1.2. In this case, you can specify in the metadata : Provides: foo Provides: bar (1.4) This is the typical use case for versioned provides. Let?s say application baz requires module bar with minimal version 1.3, it will have as dependency: Requires: bar >= 1.3 This way it will be happy to find the versioned provides if module foo is installed, and everyone is happy. Well, except that, if you try to build a package of baz, there is no way to express correctly that you depend on python-bar (>= 1.3) or python-foo (>= 1.2). This is why I?d prefer to have versioned provides simply not part of the specification. Another thing that can cause issues is exact dependencies. If you require strictly version 1.1 of foo, there is no good way of translating it into a package dependency. All the following will have serious drawbacks when facing the real world: python-foo (>= 1.1), python-foo (<< 1.1.~) python-foo (>= 1.1), python-foo (<< 1.2) python-foo (= 1.1-1) If you allow to specify requires and provides in a sophisticated way, people will use it and we will run into unmanageable situations when converting them to packages. If a module provides an API at version 1.2, it will have to still provide it at version 1.3, otherwise the module should remain private and never be installed in a public python module directory. Just like we rename C libraries when their ABI changes, we need to reach a situation where we can make the same assumptions about python modules. > > In all cases, it will be necessary to manually add shlibs-like > > information to the packages; they could be partly autogenerated like > > symbol files, but you need a mapping between provided modules and the > > first version of the package that provides it. > > Is this related ? http://lists.debian.org/debian-dpkg/2006/01/msg00118.html Yes. The thread you point to did not let to something being actually implemented, because at that moment, we lacked the necessary metadata. Since then, setuptools appeared, but it does not provide it in a sane way and it is not universal. Which is why I?m interested into the metadata format that?s discussed here. From this metadata, we will be able to generate some files that express what is provided and the required version. Something like: foo 1.0-1 bar 1.2~beta3 This way, if another package requires foo (> 1.1) and bar (without a version requirement), we can convert this dependency into: python-foo (>= 1.1), python-foo (>= 1.2~beta3) which can then be factorized, of course. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From joss at debian.org Tue Sep 30 16:27:48 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 16:27:48 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <20080926214116.GI9951@volans.logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> Message-ID: <1222784868.4166.88.camel@shizuru> Le mardi 30 septembre 2008 ? 15:49 +0200, Tarek Ziad? a ?crit : > The "Obsoletes" info could be used maybe. But the main problem I can > see is that > in any case several versions of the same module can be needed to build > one application. This is indeed a problem, and when it happens, it needs fixing instead of trying to work with it. > That is what tools like zc.buildout or virtualenv exists : they are building > an isolated environment where they install the packages so a given > Python application > can run. > > In other words the problem we have today with an OS-based installation is that > you cannot really have two versions of the same package installed, > that would make happy > two Python applications. And this is not a problem, but something that is desired. No, the problem we have today is that some developers are providing modules without API stability, which means you cannot simply depend on a module, you need a specific version. Again, when a C library changes its ABI, we do not allow it to keep the same name. It?s as simple as that. > The setuptools project has partly improved this by providing a way to > install several > version of the same package in Python and give a way to select which > one is active. This is not an improvement, it is a nightmare for the sysadmin. You cannot install things as simple (and as critical) as security updates if you allow several versions to be installed together. > From your point of view, how could we solve it at Debian level ? to > kind of isolate a group > of packages that fit the needs of one given application ? I think we need to enforce even more the habit to move unstable and private-use modules to private directories. It is not viable to add them to public directories. This is something that is done punctually in some Debian packages, but it should become mandatory for all cases where there is no API stability. A tool that eases installation and use of modules in private directories would certainly encourage developers to do so and improve the situation in this matter. > (btw A recent change it Python has allowed us to define per-user site-packages > http://mail.python.org/pipermail/python-dev/2008-January/076108.html) This is definitely a nice improvement for those on multi-user systems without administrative rights, and for those who wish to install a more recent version of a specific module. However, I don?t think we should rely on it as the normal way of installing python modules. And especially, we should not rely on on-demand download/installation of modules like setuptools does. > Interesting.. That would mean you would do version conflict resolution > at the OS level, That makes me think about the previous point: how > two applications that use conflicting versions that are not comptabile > with each other (you have to choose one of them) can cohabit ? Two conflicting versions must not use the same module namespace. The real, fundamental issue, that generates even more brokenness when you accept it and work around it, is here. It is a nightmare for the developer (who can?t rely on a defined API after "import foo"), a nightmare for the distributor (who has to use broken-by-design selection methods), and a nightmare for the system administrator (who cannot easily track what is installed on the system). Forbid that strictly, and you?ll see that methods that work today for a Linux distribution (where we already forbid it) will work just as nicely for all other distribution mechanisms. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From chris at simplistix.co.uk Tue Sep 30 17:10:34 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:10:34 +0100 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080924204739.GA29149@fridge.pov.lt> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <1222170824.8878.442.camel@balin> <274693FD-58C6-46F3-82F1-64FDE2906E55@bud.ca> <20080924204739.GA29149@fridge.pov.lt> Message-ID: <48E2416A.6030606@simplistix.co.uk> Marius Gedminas wrote: > On Tue, Sep 23, 2008 at 09:24:00PM -0700, Kevin Teague wrote: >> Or they can just use debian! Any debian developers out there up for >> the task of packaging up the 1500+ odd packages released from the Zope >> community? > > The SchoolTool guys made a tool and built .debs for all of Zope 3 that > SchoolTool needs. The resulting packages are here: > https://launchpad.net/~schooltool-owners/+archive Yes, but how many of these have made it into an official debian release? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 17:11:49 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:11:49 +0100 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <20080926210940.GH9951@volans.logilab.fr> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <20080919174433.GS8000@logilab.fr> <48D8C927.4060609@simplistix.co.uk> <4a2dae60809230359v1adf1717t66faad371dde30f9@mail.gmail.com> <20080926210940.GH9951@volans.logilab.fr> Message-ID: <48E241B5.9070501@simplistix.co.uk> Nicolas Chauvat wrote: > Baseline is "no problem with providing egg-info metadata, but pretty > please Python developers, do not code *for* distutils/setuptools/etc. > Just find a way to provide useful dependency/meta information then let > your users choose how they install your code on *their* system". Right, now this I agree with, and it seems a lot of other people do too... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Sep 30 17:20:49 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 17:20:49 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <1222784868.4166.88.camel@shizuru> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> Message-ID: <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> On Tue, Sep 30, 2008 at 4:27 PM, Josselin Mouette wrote: >> In other words the problem we have today with an OS-based installation is that >> you cannot really have two versions of the same package installed, >> that would make happy >> two Python applications. > > And this is not a problem, but something that is desired. > > No, the problem we have today is that some developers are providing > modules without API stability, which means you cannot simply depend on a > module, you need a specific version. > > Again, when a C library changes its ABI, we do not allow it to keep the > same name. It's as simple as that. I see, so there's no deprecation processes for a package ? I mean, if you change a public API of your package , you *have* to change its name ? My convention is to : - keep the the old API and the new API in the new version, let's say "2.0" - mark the old API as deprecated (we have this "warning'" module in Python to do so) - remove the old API in the next release, like "2.1" But I don't want to change the package name. And the development cycles in a python package are really short compared to OS systems, in fact we can have quite a few releases before a package is really stable. > >> The setuptools project has partly improved this by providing a way to >> install several >> version of the same package in Python and give a way to select which >> one is active. > > This is not an improvement, it is a nightmare for the sysadmin. You > cannot install things as simple (and as critical) as security updates if > you allow several versions to be installed together. > mmm... unless the version is "part of the name" in a way.... [cut] >> Interesting.. That would mean you would do version conflict resolution >> at the OS level, That makes me think about the previous point: how >> two applications that use conflicting versions that are not comptabile >> with each other (you have to choose one of them) can cohabit ? > > Two conflicting versions must not use the same module namespace. The > real, fundamental issue, that generates even more brokenness when you > accept it and work around it, is here. It is a nightmare for the > developer (who can't rely on a defined API after "import foo"), a > nightmare for the distributor (who has to use broken-by-design selection > methods), and a nightmare for the system administrator (who cannot > easily track what is installed on the system). Forbid that strictly, and > you'll see that methods that work today for a Linux distribution (where > we already forbid it) will work just as nicely for all other > distribution mechanisms. I have an idea: what about having a "known good set" (KGS) like what Zope has built on its side. a Known Good Set is a set of python package versions, that are known to provide the good execution context for a given version of Python. Maybe the Python community could maintain a known good set of python packages at PyPI, with a real work on its integrity, like any OS-vendor does I believe. And maybe this KGS could be used by Debian as the reference of package versions. -> if a package is listed in this KGS, it defines the version, for a given version of Python then, application developers in Python could work with this KGS, in their code. And if they can't get a package added in the official KGS, they will have to be on their own, inside their application, and maintain their own modules. > > Cheers, > -- > .''`. > : :' : We are debian.org. Lower your prices, surrender your code. > `. `' We will add your hardware and software distinctiveness to > `- our own. Resistance is futile. > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From chris at simplistix.co.uk Tue Sep 30 17:21:43 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:21:43 +0100 Subject: [Distutils] python v. perl - cpan v. pypi In-Reply-To: <200809241331.28077.ms@cerenity.org> References: <48D00408.50705@simplistix.co.uk> <48D970BB.7070504@snaplogic.org> <5D757E75-C78F-4A5A-9F49-46ED2E4C5806@zope.com> <200809241331.28077.ms@cerenity.org> Message-ID: <48E24407.3070705@simplistix.co.uk> Michael wrote: > Now, with python there's the general ethos: > There should be one-- and preferably only one --obvious way to do it. > > And with perl there's the general ethos: > There's more than one way to do it > > Anyone who's written extensive amounts of code in both languages will know > that the latter ethos does cause major problems in practice. > > However for packaging, with python the rule is > * "There's more than one way to do it" > > And for perl the rule is: > * Use CPAN > > I've always found this difference amusing, Me too... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Sep 30 17:25:19 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 17:25:19 +0200 Subject: [Distutils] Distutils Sprint In-Reply-To: <48E23F71.8050000@simplistix.co.uk> References: <94bdd2610809300510v46ed46dbxe2c95ffe9590b458@mail.gmail.com> <48E23F71.8050000@simplistix.co.uk> Message-ID: <94bdd2610809300825q277059f7q54ca768c97bd2abb@mail.gmail.com> On Tue, Sep 30, 2008 at 5:02 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> http://wiki.python.org/moin/DistributeSprint_#1 > > The dates on this make no sense... I fixed the typo thanks. Please propose some other dates then over there, if you wish Keep in mind that people from Paris, Tokyo and maybe SF bay are interested so the right moment in the day is hard to find :) > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From chris at simplistix.co.uk Tue Sep 30 17:02:09 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:02:09 +0100 Subject: [Distutils] Distutils Sprint In-Reply-To: <94bdd2610809300510v46ed46dbxe2c95ffe9590b458@mail.gmail.com> References: <94bdd2610809300510v46ed46dbxe2c95ffe9590b458@mail.gmail.com> Message-ID: <48E23F71.8050000@simplistix.co.uk> Tarek Ziad? wrote: > http://wiki.python.org/moin/DistributeSprint_#1 The dates on this make no sense... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 17:01:01 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:01:01 +0100 Subject: [Distutils] PEP for distutils In-Reply-To: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> Message-ID: <48E23F2D.3000802@simplistix.co.uk> Tarek Ziad? wrote: > I started to write a new PEP (well a wiki page in fact...) that > describes a new package called "pypi" that would be dedicated to package > registering and uploading mechanisms. > It would also provide enhancements like a proper password hash, or > deepers metadata controls > > http://wiki.python.org/moin/A_new_pypi_module > > Any opinions about this PEP ? I tried to include all the problems people > are having with register and upload. I think that catalog-sig would be interested in this. That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 17:36:05 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:36:05 +0100 Subject: [Distutils] "just use debian" In-Reply-To: <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> Message-ID: <48E24765.1020908@simplistix.co.uk> Tarek Ziad? wrote: >>> In other words the problem we have today with an OS-based installation is that >>> you cannot really have two versions of the same package installed, >>> that would make happy >>> two Python applications. Right, which is why dependencies can often be best matched by a project-based tool like buildout rather than having to have one python setup support all use cases. >> No, the problem we have today is that some developers are providing >> modules without API stability, which means you cannot simply depend on a >> module, you need a specific version. This problem is never going away, it's the nature of software. >> Again, when a C library changes its ABI, we do not allow it to keep the >> same name. It's as simple as that. That's insane, and I bet without trying to hard, I could find examples of violation of this supposed practice. > My convention is to : > - keep the the old API and the new API in the new version, let's say "2.0" > - mark the old API as deprecated (we have this "warning'" module in > Python to do so) > - remove the old API in the next release, like "2.1" Right. > But I don't want to change the package name. Right. >>> The setuptools project has partly improved this by providing a way to >>> install several >>> version of the same package in Python and give a way to select which >>> one is active. >> This is not an improvement, it is a nightmare for the sysadmin. Absolutely. This multi-version rubbish is totally and utterly insanely wrong. > I have an idea: what about having a "known good set" (KGS) like what > Zope has built on its > side. > > a Known Good Set is a set of python package versions, that are known to provide > the good execution context for a given version of Python. Given how poorly maintained Zope's "KGS" is, I think this is a pipe dream. Besides, accurately specified dependency information, including versions, within a package should suffice. It would be handy if you could also specify python version compatibility in this, something that setuptools does not currently support AFAIK. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From gael.varoquaux at normalesup.org Tue Sep 30 17:38:22 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 30 Sep 2008 17:38:22 +0200 Subject: [Distutils] PEP for distutils In-Reply-To: <48E23F2D.3000802@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> Message-ID: <20080930153822.GJ26878@phare.normalesup.org> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: > That said, I didn't see any indication of what I consider to be a critical > failure in PyPI: No dependency metadata prior to downloading the package. +1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible. There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install. Ga?l From ianb at colorstudy.com Tue Sep 30 17:41:11 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 10:41:11 -0500 Subject: [Distutils] PEP for distutils In-Reply-To: <20080930153822.GJ26878@phare.normalesup.org> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> Message-ID: <48E24897.8010800@colorstudy.com> Gael Varoquaux wrote: > On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >> That said, I didn't see any indication of what I consider to be a critical >> failure in PyPI: No dependency metadata prior to downloading the package. > > +1. I want to be able do list all the packages an easy_install run will > download without running it. Something like the "-s" option of apt-get. > In addition, I want this information to be available programmatically (ie > with a good api, not something that expects to be called from the command > line) to be able to use it to build dependency graphs, generate conflicts > list, or simply tell me that I have requested something that is > impossible. > > There is nothing that I hate more than easy_install failing after having > half-installed a package because of a missing dependency. This is one of > the reasons I am never too happy when I have to run easy_install. FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Tue Sep 30 17:43:51 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 17:43:51 +0200 Subject: [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <94bdd2610809300843r516851aasdca03f7ce60878f9@mail.gmail.com> On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking wrote: > Gael Varoquaux wrote: >> >> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >>> >>> That said, I didn't see any indication of what I consider to be a >>> critical failure in PyPI: No dependency metadata prior to downloading the >>> package. >> >> +1. I want to be able do list all the packages an easy_install run will >> download without running it. Something like the "-s" option of apt-get. >> In addition, I want this information to be available programmatically (ie >> with a good api, not something that expects to be called from the command >> line) to be able to use it to build dependency graphs, generate conflicts >> list, or simply tell me that I have requested something that is >> impossible. >> >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to get > the metadata. Yes, so having them at PyPI would be a good idea indeed, I am adding that to that small PEP > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From joss at debian.org Tue Sep 30 17:50:11 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 17:50:11 +0200 Subject: [Distutils] [pyconuk] "just use debian" In-Reply-To: <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> Message-ID: <1222789811.4166.114.camel@shizuru> Le mardi 30 septembre 2008 ? 17:20 +0200, Tarek Ziad? a ?crit : > > Again, when a C library changes its ABI, we do not allow it to keep the > > same name. It's as simple as that. > > I see, so there's no deprecation processes for a package ? Not per se. It is the job of the package manager to propose removing deprecated packages when they are no longer available in the repository. > I mean, if you change a public API of your package , you *have* to > change its name ? Yes, this is the requirement for C libraries, and we try to enforce it as well for other languages. > My convention is to : > - keep the the old API and the new API in the new version, let's say "2.0" > - mark the old API as deprecated (we have this "warning'" module in > Python to do so) > - remove the old API in the next release, like "2.1" > > But I don't want to change the package name. > > And the development cycles in a python package are really short > compared to OS systems, in fact > we can have quite a few releases before a package is really stable. I don?t think the requirements are different from those of C library developers. There are, of course, special cases for libraries that are in development; generally we take a snapshot, give it a specific soname and enforce the ABI compatibility in the Debian package. The other possibility is to distribute the library only in a private directory. Nothing in this process is specific to C; the technical details are different for python modules, but we should be able to handle it in a similar way. > > This is not an improvement, it is a nightmare for the sysadmin. You > > cannot install things as simple (and as critical) as security updates if > > you allow several versions to be installed together. > > mmm... unless the version is "part of the name" in a way.... Yes, this is what C libraries do with the SONAME, for which the convention is to postfix it with a number, which changes when the ABI is changed in an incompatible way. I don?t know whether it would be possible to do similar things with python modules, but it is certainly something to look at. > > Two conflicting versions must not use the same module namespace. > > I have an idea: what about having a "known good set" (KGS) like what > Zope has built on its > side. > > a Known Good Set is a set of python package versions, that are known to provide > the good execution context for a given version of Python. > > Maybe the Python community could maintain a known good set of python > packages at PyPI, with a real work on its integrity, like any > OS-vendor does I believe. Having a body that enforces API stability for a number of packages would probably prevent such issues from happening in those packages. However, that means relying too much and this body, and experience proves it will quickly lag behind. Furthermore, the need to add packages that are not in the KGS to distributions will arise sooner or later. > And maybe this KGS could be used by Debian as the reference of package versions. We will always need, for some cases, more recent packages or packages that are not in the KGS. > -> if a package is listed in this KGS, it defines the version, for a > given version of Python You don?t have to define it so strictly. There is no reason why a new version couldn?t be accepted in the KGS for an existing python version, if it has been checked that it will not break existing applications using this module. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From tarek.ziade at ingeniweb.com Tue Sep 30 17:51:38 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Tue, 30 Sep 2008 17:51:38 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E24765.1020908@simplistix.co.uk> References: <20080917171851.GA27261@logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> Message-ID: 2008/9/30 Chris Withers > Tarek Ziad? wrote: > >> In other words the problem we have today with an OS-based installation is >>>> that >>>> you cannot really have two versions of the same package installed, >>>> that would make happy >>>> two Python applications. >>>> >>> > Right, which is why dependencies can often be best matched by a > project-based tool like buildout rather than having to have one python setup > support all use cases. > > No, the problem we have today is that some developers are providing >>> modules without API stability, which means you cannot simply depend on a >>> module, you need a specific version. >>> >> > This problem is never going away, it's the nature of software. > > Again, when a C library changes its ABI, we do not allow it to keep the >>> same name. It's as simple as that. >>> >> > That's insane, and I bet without trying to hard, I could find examples of > violation of this supposed practice. > > My convention is to : >> - keep the the old API and the new API in the new version, let's say >> "2.0" >> - mark the old API as deprecated (we have this "warning'" module in >> Python to do so) >> - remove the old API in the next release, like "2.1" >> > > Right. > > But I don't want to change the package name. >> > > Right. > > The setuptools project has partly improved this by providing a way to >>>> install several >>>> version of the same package in Python and give a way to select which >>>> one is active. >>>> >>> This is not an improvement, it is a nightmare for the sysadmin. >>> >> > Absolutely. This multi-version rubbish is totally and utterly insanely > wrong. > > I have an idea: what about having a "known good set" (KGS) like what >> Zope has built on its >> side. >> >> a Known Good Set is a set of python package versions, that are known to >> provide >> the good execution context for a given version of Python. >> > > Given how poorly maintained Zope's "KGS" is, I think this is a pipe dream. > > Besides, accurately specified dependency information, including versions, > within a package should suffice. It would be handy if you could also specify > python version compatibility in this, something that setuptools does not > currently support AFAIK. you can use the Requires-Python metadata though. For KGS I agree that this is a big work, but there's the need to work at a higher level that in your package that is what zc.buildout brought in a way, but at the application level, and with no respect to the OS-level in a way. Si we should find a way to generalize this at Python level imho: being able to develop your package in a known environment. and being able to give that info to the OS. Python frameworks are exploding in a myriad of packags : a Python instalation needs to handle up to a hundreds of public packages now to run a plone site for example > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way -------------- next part -------------- An HTML attachment was scrubbed... URL: From jp.camguilhem at gmail.com Tue Sep 30 17:53:22 2008 From: jp.camguilhem at gmail.com (Jean-Philippe CAMGUILHEM) Date: Tue, 30 Sep 2008 17:53:22 +0200 Subject: [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking wrote: > Gael Varoquaux wrote: > >> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >> >>> That said, I didn't see any indication of what I consider to be a >>> critical failure in PyPI: No dependency metadata prior to downloading the >>> package. >>> >> >> +1. I want to be able do list all the packages an easy_install run will >> download without running it. Something like the "-s" option of apt-get. >> In addition, I want this information to be available programmatically (ie >> with a good api, not something that expects to be called from the command >> line) to be able to use it to build dependency graphs, generate conflicts >> list, or simply tell me that I have requested something that is >> impossible. >> >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. >> > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to get > the metadata. is a "simple catalog "db storage for metadata like /usr/ports/ on freebsd a bad idea ? the idea is to not download all packages to get the metadata, but just query the catalog/db Cheers, Jean-Philippe > > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at simplistix.co.uk Tue Sep 30 17:55:06 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:55:06 +0100 Subject: [Distutils] "just use debian" In-Reply-To: References: <20080917171851.GA27261@logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> Message-ID: <48E24BDA.2050104@simplistix.co.uk> Tarek Ziade wrote: > For KGS I agree that this is a big work, but there's the need to work at > a higher level that in your package Why? You really need to explain to me why the dependency information in each of the packages isn't enough? > Python frameworks are exploding in a myriad of packags : a Python > instalation needs to handle up to a hundreds of public packages now to > run a plone site for example Yes, Plone and Zope both got the wrong end of the stick by making myriads of eggs rather than a few big ones... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 17:56:24 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:56:24 +0100 Subject: [Distutils] "Python Package Management Sucks" In-Reply-To: <18650.28513.573244.313581@gargle.gargle.HOWL> References: <48D00408.50705@simplistix.co.uk> <20080917171851.GA27261@logilab.fr> <48D8C701.6070707@simplistix.co.uk> <18650.28513.573244.313581@gargle.gargle.HOWL> Message-ID: <48E24C28.8020304@simplistix.co.uk> Matthias Klose wrote: >>> Install debian and get back to productive tasks. >> This is an almost troll-like answer. >> See page 35 of the presentation. > > I disagree. You could think of "Packages are Pythons Plugins" (taken > from page 35) as a troll-like statement as well. You're welcome to your (incorrect) opinion ;-) Debian packages could just as easilly be seen as Debian's pluggins. > and a subset of all these operating systems (linux or other) do have > the need to distribute python and a set of python modules and > extensions. they cannot rely on a plugin system outside the (os) > distribution. OK, you guys have persuaded me of this at least... >> - all the package management systems behave differently and expect >> packages to be set up differently for them > > correct, but again they share common requirements. ...but all have different implementations. > some people prefer to name this "stable releases" instead of > "bitrot". I'll call bullshit on this one. The most common problem I have as a happy Debian user and advocate when I go to try and get help for a packaged application (I use packages because I perhaps mistakenly assume this is the best way to get security-fixed softare), such as postfix, postgres, and Zope if I was foolish enough to take that path, is "why are toy using that ancient and buggy version of the software?!" shortly before pointing out how all the issues I'm facing are solved in newer (stable) releases. The problem is that first the application needs to be tested and released by its community, then Debian needs to re-package, patch, generally mess around with it, etc before it eventually gets a "Debian release". It's bad enough with apps with huge support bases like portgres, imagine trying to do this "properly" for the 4000-odd packages on PyPI... > Speaking of extensions "maintained by the entity originating the > python package": this much too often is a way of bitrot. is the > shipped library up to date? does it have security fixes? how many > duplicates are shipped in different extensions? does the library need > to be shipped at all (because some os does ship it)? So what do you propose doing one projectA depends on version 1.0 of libC and projectB depends on version 2.0 of libC? > this is known trouble for os distributors, and your statement is > generally wrong. firefox plugins are packaged in distributions and the > plugin system is able to cope with packaged plugins. I guess since my desktop OS is still windows, this is not something I've had to fight with ;-) >> Packages are Python's "plugins" and so should get the same type of >> consistent, cross-platform package management targetted at the >> application in question, which is Python in this case. > > No, as explained above. While I'll buy the argument that python packaging tools should make life easier for production of os-specific packages, I still don't think you're correct ;-) > Considering an extension interfacing a library > shipped with the os, you do want to use this library, not add another > copy. libxml2 seems to be agood example to use here... I guess on debian I'd need to likely install libxml2-dev before I could install the lxml package... ...what about MacOS X? ...what about Windows? > An upstream > extension maintainer cannot provide this unless he builds this > extension for every (os) distribution and maintains it during the os' > lifecycle. ...or just says in the docs "hey, you need libxml2 for this, unless you're on Windows, in which case the binary includes it". > - os distributors usually try to minimize the versions they include, > trying to just ship one version. ...which is fair enough for the "system python", but many of us have a collection of apps, some of which require Python 2.4, some Python 2.5, and on top of each of those, different versions of different packages for each app. In my case, I do source (alt-)installs of python rather than trusting the broken stuff that ships with Debian and buildout to make sure I get the right versions of the right packages for each project. > - setuptools has the narrow minded view of a python package being > contained in a single directory, which doesn't fit well when you > do have common locations for include or doc files. Python packages have no idea of "docs" or "includes", which is certainly a deficiency. > way packaging the python module with rpm or dpkg. E.g. namespace > packages are a consequence how setuptools distributes and installs > things. Why force this on everybody? being able to break a large lump (say zope.*) into seperate distributions is a good idea, which setuptools implements very badly using namespace packages... > A big win could be a modularized setuptools where you are able to only > use the things you do want to use, e.g. > > - version specifications (not just the heuristics shipped with > setuptools). not sure what you mean by this. > - specification of dependencies. > > - resource management ? > - a module system independent from any distribution specific stuff. ? > - any other distribution specific stuff. ? > The "Wouldn't it be nice if?" pacman (page 55) sounds like a nice > idea, if it could just handle multiple repositories and one of them > being the archive of the os distributor (iirc the java jsr277 proposed > the use of multiple repositories, even in different formats). Where does it say it doesn't support multiple repositories? ;-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From joss at debian.org Tue Sep 30 18:01:32 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 18:01:32 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E24765.1020908@simplistix.co.uk> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> Message-ID: <1222790492.4166.127.camel@shizuru> Le mardi 30 septembre 2008 ? 16:36 +0100, Chris Withers a ?crit : > >> No, the problem we have today is that some developers are providing > >> modules without API stability, which means you cannot simply depend on a > >> module, you need a specific version. > > This problem is never going away, it's the nature of software. It doesn?t have to go away per se, but we need proper ways to deal with incompatible changes in the interfaces. > >> Again, when a C library changes its ABI, we do not allow it to keep the > >> same name. It's as simple as that. > > That's insane, and I bet without trying to hard, I could find examples > of violation of this supposed practice. Of course, Python developers don?t have the monopoly on misunderstanding maintainability requirements. It evens happens more often in C, where the ABI can change without any incompatibility in the API. When this happens without a soname change, we either change the soname ourselves (diverging from upstream) or change the package name, making it impossible to install two conflicting versions at once. In Python libraries, this is not possible without changing the code, since the file name and the module name are the same. If a Python module changes its API incompatibly, we are forced to update all reverse dependencies and add versioned conflicts, without being able to ensure none is forgotten, and without enforcing the change for third-party packages. [snip] > Besides, accurately specified dependency information, including > versions, within a package should suffice. It will suffice, but we will not be able to manage it in distributions if you allow too many weird things to be specified in these dependencies. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From ziade.tarek at gmail.com Tue Sep 30 18:01:49 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 18:01:49 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E24BDA.2050104@simplistix.co.uk> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> Message-ID: <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> On Tue, Sep 30, 2008 at 5:55 PM, Chris Withers wrote: > Tarek Ziade wrote: >> >> For KGS I agree that this is a big work, but there's the need to work at a >> higher level that in your package > > Why? You really need to explain to me why the dependency information in each > of the packages isn't enough? > Because you can keep up with the dependencies changes, removed, or introduced by a package you depend on. How do you decide that the version 1.2 of bar is the one you should use, when you use the foo package that can work with any version of bar ? You can define the version of foo, but can't describe all the versions of the packages foo uses. You'd end up building your own KGS in a way.. So a general list of versions can help >> Python frameworks are exploding in a myriad of packags : a Python >> instalation needs to handle up to a hundreds of public packages now to run >> a plone site for example > > Yes, Plone and Zope both got the wrong end of the stick by making myriads of > eggs rather than a few big ones... I think it is a good opportunity to re-uses things. Right now I can work on projects that use packages from pylons AND plone AND zope 3. Bigger eggs wouldn't let you reuse thing like you can now imho > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From chris at simplistix.co.uk Tue Sep 30 18:06:23 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 17:06:23 +0100 Subject: [Distutils] more thoughts on python package management Message-ID: <48E24E7F.2040903@simplistix.co.uk> Hi All, I've been trying to catch up on all the packaging discussions but couldn't find the right place to reply so thought I'd just do so seperately... Probably the biggest thing that strikes me now is that distutils/setuptools/distribute/pacman/whatever should aim to do much less... In fact, I get the feeling what we really need is a way for package maintainers to provide the following metadata: - where the docs are - where the tests are and how they're run - how anything not-python should be built - what the dependencies are (maybe even what the non-python dependencies are!) - what version of the package this is This should be in a build-tool independent fashion such that any build tools, but especially those of operating system maintainers, can run over the same metadata and build their packages. The only other critical thing for me is that *all* of the above metadata should be available post-install. With the above in place, we free up the evolution of build tools and let the OS-specific packaging tools play nicely. I think a good aim would also be to have some "one-way-to-do-it" python tools too for: - installing a package - uploading a package to PyPI - getting a package from PyPI ...without any silly big plugin system in the way distutils currently works. What do other people feel? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 18:08:31 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 17:08:31 +0100 Subject: [Distutils] "just use debian" In-Reply-To: <1222790492.4166.127.camel@shizuru> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <1222790492.4166.127.camel@shizuru> Message-ID: <48E24EFF.3010707@simplistix.co.uk> Josselin Mouette wrote: > Le mardi 30 septembre 2008 ? 16:36 +0100, Chris Withers a ?crit : >>>> No, the problem we have today is that some developers are providing >>>> modules without API stability, which means you cannot simply depend on a >>>> module, you need a specific version. >> This problem is never going away, it's the nature of software. > > It doesn?t have to go away per se, but we need proper ways to deal with > incompatible changes in the interfaces. Well, the generally accepted way seems to be to increase the major version number... > In Python libraries, this is not possible without changing the code, > since the file name and the module name are the same. The distribution name and package/module name do not have to be the same... > It will suffice, but we will not be able to manage it in distributions > if you allow too many weird things to be specified in these > dependencies. Explain "too many weird things"... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 18:12:39 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 17:12:39 +0100 Subject: [Distutils] "just use debian" In-Reply-To: <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> Message-ID: <48E24FF7.9000300@simplistix.co.uk> Tarek Ziad? wrote: >> Tarek Ziade wrote: >>> For KGS I agree that this is a big work, but there's the need to work at a >>> higher level that in your package >> Why? You really need to explain to me why the dependency information in each >> of the packages isn't enough? > > Because you can keep up with the dependencies changes, removed, or introduced > by a package you depend on. Why can this not be expressed in the dependency information in the package? > How do you decide that the version 1.2 of bar is the one you should use, > when you use the foo package that can work with any version of bar ? If you are using no other packages that have a dependency on bar that has a specific version requirement, then the answer is that you can use any version of bar you desire. > You can define the version of foo, but can't describe all the versions of > the packages foo uses. Why? > You'd end up building your own KGS in a way.. ...you mean like the [versions] section with buildout? Yes, I agree us paranoid people may want to do that, but we really shouldn't need to provided the packages each correctly define their dependencies... > So a general list of versions can help I would be happy to wager that this would never successfully be maintained. > Bigger eggs wouldn't let you reuse thing like you can now imho That doesn't explain the majority of eggs that end up dragging a whole load of eggs down themselves. In this case, they should all be packaged as one egg... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 30 18:13:18 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 17:13:18 +0100 Subject: [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <48E2501E.90202@simplistix.co.uk> Ian Bicking wrote: > FWIW, pyinstall can collect all the packages before installing any of > them. You do have to download all packages, though, as that's the only > way to get the metadata. ...yes, and this is why PyPI should change! Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From joss at debian.org Tue Sep 30 18:17:42 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 18:17:42 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E24EFF.3010707@simplistix.co.uk> References: <20080917171851.GA27261@logilab.fr> <48DDEC5C.7090601@ar.media.kyoto-u.ac.jp> <20080928173011.GA4852@volans.logilab.fr> <48E0C007.70702@ar.media.kyoto-u.ac.jp> <94bdd2610809290509l2ab7359ep7f4edfcd23eae4ee@mail.gmail.com> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <1222790492.4166.127.camel@shizuru> <48E24EFF.3010707@simplistix.co.uk> Message-ID: <1222791462.4166.134.camel@shizuru> Le mardi 30 septembre 2008 ? 17:08 +0100, Chris Withers a ?crit : > Josselin Mouette wrote: > > It doesn?t have to go away per se, but we need proper ways to deal with > > incompatible changes in the interfaces. > > Well, the generally accepted way seems to be to increase the major > version number... This information is not accessible directly at import time. If you want to rely on it to check the API compatibility, you?ll end up doing the horrible things pygtk and gst-python did. And believe me, that will not be helpful. > > In Python libraries, this is not possible without changing the code, > > since the file name and the module name are the same. > > The distribution name and package/module name do not have to be the same... Indeed, but if we change the package name without changing the file name, we have to make both packages conflict with each other. This works for distribution packages, but it doesn?t help for third-party addons, and it can make things complicated if two packages need different APIs. > > It will suffice, but we will not be able to manage it in distributions > > if you allow too many weird things to be specified in these > > dependencies. > > Explain "too many weird things"... I showed already two examples: versioned provides and exact dependencies. That?s just after thinking about it for 5 minutes; if we want it to really work, we need to thoroughly think of what exact kind of information we are able to use. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From amk at amk.ca Tue Sep 30 18:25:59 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 30 Sep 2008 12:25:59 -0400 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <20080930162559.GA11804@amk-desktop.matrixgroup.net> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: > FWIW, pyinstall can collect all the packages before installing any of > them. You do have to download all packages, though, as that's the only > way to get the metadata. Does the DOAP output for a package not contain enough metadata? --amk From ianb at colorstudy.com Tue Sep 30 18:37:01 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 11:37:01 -0500 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> Message-ID: <48E255AD.7020408@colorstudy.com> A.M. Kuchling wrote: > On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> FWIW, pyinstall can collect all the packages before installing any of >> them. You do have to download all packages, though, as that's the only >> way to get the metadata. > > Does the DOAP output for a package not contain enough metadata? No. It probably could hold that information, but currently PyPI doesn't keep any record of requirements, and so the DOAP file it generates doesn't indicate requirements either. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ianb at colorstudy.com Tue Sep 30 18:37:13 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 11:37:13 -0500 Subject: [Distutils] "just use debian" In-Reply-To: <48E24FF7.9000300@simplistix.co.uk> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> Message-ID: <48E255B9.50806@colorstudy.com> Chris Withers wrote: > Tarek Ziad? wrote: >>> Tarek Ziade wrote: >>>> For KGS I agree that this is a big work, but there's the need to >>>> work at a >>>> higher level that in your package >>> Why? You really need to explain to me why the dependency information >>> in each >>> of the packages isn't enough? >> >> Because you can keep up with the dependencies changes, removed, or >> introduced >> by a package you depend on. > > Why can this not be expressed in the dependency information in the package? I tried this briefly for a while when Setuptools first came out, and I found it completely unmaintainable. Say I have a package that represents an application. We'll call it FooBlog. I release version 1.0. It uses the Turplango web framework (1.5 at the time of release) and the Storchalmy ORM (0.4), and Turplango uses HardJSON (1.2.1). I want my version 1.0 to keep working. So, I figure I'll add the dependencies: Turplango==1.5 Storchalmy==0.4 Then HardJSON 2.0 is released, and Turplango only required HardJSON>=1.2, so new installations start installing HardJSON 2.0. But my app happens not to be compatible with that library, and so it's broken. OK... so, I could add HardJSON==1.2.1 in my requirements. But then a small bug fix, HardJSON 1.2.2 comes out, that fixes a security bug. Turplango releases version 1.5.1 that requires HardJSON>=1.2.2. I now have have to update FooBlog to require both Turplango==1.5.1 and HardJSON==1.2.2. Later on, I decide that Turplango 1.6 fixes some important bugs, and I want to try it with my app. I can install Turplango 1.6, but I can't start my app because I'll get a version conflict. So to even experiment with a new version of the app, I have to check out FooBlog, update setup.py, reinstall (setup.py develop) the package, and then I can start using it. But if I've made other hard requirements of packages like HardJSON, I'll have to update all those too. So... that's the kind of thing I encountered with just a couple dependencies, but in practice it was much worse because there were a lot more than 3 libraries involved. I now think it is best to only use version requirements to express known conflicts. For future versions of packages you can't really know if they will cause conflicts until they are released. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Tue Sep 30 19:13:03 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 19:13:03 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E255B9.50806@colorstudy.com> References: <20080917171851.GA27261@logilab.fr> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> Message-ID: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> On Tue, Sep 30, 2008 at 6:37 PM, Ian Bicking wrote: > Chris Withers wrote: >> >> Tarek Ziad? wrote: >>>> >>>> Tarek Ziade wrote: >>>>> >>>>> For KGS I agree that this is a big work, but there's the need to work >>>>> at a >>>>> higher level that in your package >>>> >>>> Why? You really need to explain to me why the dependency information in >>>> each >>>> of the packages isn't enough? >>> >>> Because you can keep up with the dependencies changes, removed, or >>> introduced >>> by a package you depend on. >> >> Why can this not be expressed in the dependency information in the >> package? > > I tried this briefly for a while when Setuptools first came out, and I found > it completely unmaintainable. > > Say I have a package that represents an application. We'll call it FooBlog. > I release version 1.0. It uses the Turplango web framework (1.5 at the > time of release) and the Storchalmy ORM (0.4), and Turplango uses HardJSON > (1.2.1). > > I want my version 1.0 to keep working. So, I figure I'll add the > dependencies: > > Turplango==1.5 > Storchalmy==0.4 > > Then HardJSON 2.0 is released, and Turplango only required HardJSON>=1.2, so > new installations start installing HardJSON 2.0. But my app happens not to > be compatible with that library, and so it's broken. OK... so, I could add > HardJSON==1.2.1 in my requirements. > > But then a small bug fix, HardJSON 1.2.2 comes out, that fixes a security > bug. Turplango releases version 1.5.1 that requires HardJSON>=1.2.2. I now > have have to update FooBlog to require both Turplango==1.5.1 and > HardJSON==1.2.2. > > Later on, I decide that Turplango 1.6 fixes some important bugs, and I want > to try it with my app. I can install Turplango 1.6, but I can't start my > app because I'll get a version conflict. So to even experiment with a new > version of the app, I have to check out FooBlog, update setup.py, reinstall > (setup.py develop) the package, and then I can start using it. But if I've > made other hard requirements of packages like HardJSON, I'll have to update > all those too. > > So... that's the kind of thing I encountered with just a couple > dependencies, but in practice it was much worse because there were a lot > more than 3 libraries involved. I now think it is best to only use version > requirements to express known conflicts. For future versions of packages > you can't really know if they will cause conflicts until they are released. Exactly, you can't control everything from your package unless you work in an isolated environement like virtualenv or zc.buildout provides, so I can't see any solution unless someone is taking care of it at a higher level :( maybe PyPI though, can automate this, when a package is uploaded, by browsing all dependency and finding relevant conflict ? PyPI "knows" all the packages out there. At least display those conflicts somehow ? or warn about them. (I am pushing this to catalog-sig as well, sorry for the cross-post. I do thing though, these mailing lists should merge) > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ianb at colorstudy.com Tue Sep 30 19:25:17 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 12:25:17 -0500 Subject: [Distutils] "just use debian" In-Reply-To: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> Message-ID: <48E260FD.9010309@colorstudy.com> Tarek Ziad? wrote: >> So... that's the kind of thing I encountered with just a couple >> dependencies, but in practice it was much worse because there were a lot >> more than 3 libraries involved. I now think it is best to only use version >> requirements to express known conflicts. For future versions of packages >> you can't really know if they will cause conflicts until they are released. > > Exactly, you can't control everything from your package unless you > work in an isolated environement like virtualenv or zc.buildout > provides, so I can't see any solution unless someone is taking care of > it at a higher level :( > > maybe PyPI though, can automate this, when a package is uploaded, by > browsing all dependency and > finding relevant conflict ? PyPI "knows" all the packages out there. > > At least display those conflicts somehow ? or warn about them. Yes, keeping this version information separate from packages would help, I think. If you find out more information about a conflict it shouldn't require a new release -- new releases take a while to do, and have cascading effects. This kind of metadata isn't so much about the package, as about how the package relates to other packages. If we could somewhat safely have collaborative conflict information that would be nice, though there's different kinds of conflicts so it might be infeasible. It's all too common for a person to just poke around with version stuff until something works, but in a way that is only accurate for the context of their application, and if they submit that information upstream they could easily break other people's setups unnecessarily. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From gael.varoquaux at normalesup.org Tue Sep 30 19:38:26 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 30 Sep 2008 19:38:26 +0200 Subject: [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <20080930173826.GL26878@phare.normalesup.org> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to > get the metadata. Yes, I have seen that. I was very happy to witness the release of this tool. Thank you. Ga?l From pje at telecommunity.com Tue Sep 30 19:51:30 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 30 Sep 2008 13:51:30 -0400 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> Message-ID: <20080930175201.106863A409C@sparrow.telecommunity.com> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: > > FWIW, pyinstall can collect all the packages before installing any of > > them. You do have to download all packages, though, as that's the only > > way to get the metadata. > >Does the DOAP output for a package not contain enough metadata? Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.) From cakebread at gmail.com Tue Sep 30 20:21:54 2008 From: cakebread at gmail.com (Rob Cakebread) Date: Tue, 30 Sep 2008 11:21:54 -0700 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <20080930175201.106863A409C@sparrow.telecommunity.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> <20080930175201.106863A409C@sparrow.telecommunity.com> Message-ID: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby wrote: > At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >> >> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> > FWIW, pyinstall can collect all the packages before installing any of >> > them. You do have to download all packages, though, as that's the only >> > way to get the metadata. >> >> Does the DOAP output for a package not contain enough metadata? > > Nope. And it can't possibly do so, unless it contains dependency data for > every possible variation of the package. For example, a package might > dynamically declare dependency on ctypes, depending on whether you're > installing it for Python 2.4 or Python 2.5. (Dependencies can also be > platform-specific and build-option-specific, as well as > Python-version-specific.) > Not to mention the DOAP vocabulary lacks a way to describe dependency information. This is planned but it has to be well thought out because of all the variations Philip mentions. The good news is much of this dependency info is already in existence in Linux distributions. Take a Gentoo ebuild, for example. It has separate run-time, build-time and test dependency info, dependencies based on enabled features, and dependencies based on the version of Python used. Ebuilds also have metadata mapping the PyPI name to the Gentoo package name, so it'll be easy enough to create a database with all this info. I'm working on this now at http://doapspace.org/ where you can find DOAP for Python packages with a bit more metadata than the DOAP supplied on PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME ) From nicolas.chauvat at logilab.fr Tue Sep 30 20:58:36 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Tue, 30 Sep 2008 20:58:36 +0200 Subject: [Distutils] more thoughts on python package management Message-ID: <20080930185836.GA13231@volans.logilab.fr> > What do other people feel? Open Standards. Standardizing data format rather than tools. Well defined public PyPI API... of course I agree with you! -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances From joss at debian.org Tue Sep 30 21:02:41 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 21:02:41 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E255B9.50806@colorstudy.com> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> Message-ID: <1222801361.32659.1.camel@shizuru> Le mardi 30 septembre 2008 ? 11:37 -0500, Ian Bicking a ?crit : > Say I have a package that represents an application. [snip] > Then HardJSON 2.0 is released, and Turplango only required > HardJSON>=1.2, so new installations start installing HardJSON 2.0. But > my app happens not to be compatible with that library, and so it's > broken. OK... No, please stop here. That?s not OK. If a new version of HardJSON breaks your application, it is friggin? broken. If that new version is not compatible, it should be called HardJSON2, and nothing will break. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From martin at v.loewis.de Tue Sep 30 22:08:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Sep 2008 22:08:25 +0200 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <48E23F2D.3000802@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> Message-ID: <48E28739.7080901@v.loewis.de> > That said, I didn't see any indication of what I consider to be a > critical failure in PyPI: No dependency metadata prior to downloading > the package. Contributions are welcome. The source code of PyPI is available publically, and I'm willing to accept patches. I won't have time to work on this in the next 12 months myself. Regards, Martin From kiorky at cryptelium.net Tue Sep 30 22:11:34 2008 From: kiorky at cryptelium.net (kiorky) Date: Tue, 30 Sep 2008 22:11:34 +0200 Subject: [Distutils] more thoughts on python package management In-Reply-To: <48E24E7F.2040903@simplistix.co.uk> References: <48E24E7F.2040903@simplistix.co.uk> Message-ID: <48E287F6.9010605@cryptelium.net> Chris Withers a ?crit : > > Hi All, > > > > I've been trying to catch up on all the packaging discussions but > > couldn't find the right place to reply so thought I'd just do so > > seperately... Maybe, we have all our own definition ... > > > > Probably the biggest thing that strikes me now is that > > distutils/setuptools/distribute/pacman/whatever should aim to do much > > less... I don't totally agree, we must provide a tool to build from start to end our package even if we let means (eg via complete metadatas) to do the job in another way, by other tools. > > > > In fact, I get the feeling what we really need is a way for package > > maintainers to provide the following metadata: > > > > - where the docs are > > > > - where the tests are and how they're run > > > > - how anything not-python should be built > > > > - what the dependencies are > > (maybe even what the non-python dependencies are!) > > > > - what version of the package this is > > > > This should be in a build-tool independent fashion such that any build > > tools, but especially those of operating system maintainers, can run > > over the same metadata and build their packages. +1 for more explicit metadata on python packages, But i don't think our target is only the target "OS" and its relative packages manager. I really can't see how to get running a lot of projects with inter-related incompatibilities on the same system without some sort of isolation So we cannot say we will be able to install whatever we want with everything else at the same time, system wide. And selecting versions by hand, in each project, can be painful. I'd prefer to have a "project" or "environment specific" approach like buildout, or virtualenv, or the combination of these tools, even maybe a wrap-up tool like minitage [1]. > > > > The only other critical thing for me is that *all* of the above metadata > > should be available post-install. > > I 'd prefer to know what i am installing before installing it. For me, the metadata are predictable things, which must be there before installing any part of a project. > > With the above in place, we free up the evolution of build tools and let > > the OS-specific packaging tools play nicely. > > > > I think a good aim would also be to have some "one-way-to-do-it" python > > tools too for: > > > > - installing a package > > > > - uploading a package to PyPI > > > > - getting a package from PyPI > > > > +1, but see under > > ...without any silly big plugin system in the way distutils currently > > works. > > > > What do other people feel? > > > > cheers, > > > > Chris > > IMHO, a good management suite for python packages is a collection of tools, plugins and API bindings which: - Can do all its job in total isolation (installation in /prefix even where our only privilege is a filesystem write access) There two levels i can see there: - Isolation from the OS - Isolation in the "target environment" (/package1/the-stuff and /package2/the-stuff) - Is OS independant (maybe the most hard thing to do) - Is failure tolerant: - Handle offline mode - Know how to go backward - Use a sandbox before (un)installing - Let us means to add, remove and override the packages 's repositories and even the packages themselves. - Can use a download cache - Deal with dependencies - Can distribute the package - Can fetch the package - Can build the package - Can Test the package - Can install the package from source or binary - Trace and store what is installed - Can uninstall the package - Know how to deal with reverse dependencies problems, even if this is a tool with just rebuild them like the gentoo's ones [2]. - Provide goods metadata - Have all the classical features cli tools like apt, emerge ... have: - Search on installed and available softwares - List installed files - Knows from which package a file is belonging to - Have an API to let us build, query, distribute, get, ... our packages. - Have some hook/plugin registration system like the egg's entry points. And the according packages's metadata must contain: - package name - package version - Detailed dependencies requirements which include also incompatibilities (like OS ones) - Maybe some knobs system like the USES in gentoo. - where we get the package - The way we build/install/uninstall it - Author/website/license and so on - Where are the tests - How the tests are run - hooks registration information There is nothing new there, Those are classical package manager needs. Maybe more "source packages manager" needs because we are dealing with something which can be built on the target. What is different there is that i try introduce an "environment" approach more than a "system centric" one. [1] http://www.minitage.org/doc/rst [2] - emerge @preserved-rebuild: http://r0bertz.blogspot.com/2008/06/portage-22-preserve-libs-features.html - revdep-rebuild : http://gentoo-wiki.com/TIP_Control_revdep-rebuild -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From chris at simplistix.co.uk Tue Sep 30 22:26:35 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 21:26:35 +0100 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <48E28739.7080901@v.loewis.de> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de> Message-ID: <48E28B7B.3060602@simplistix.co.uk> Martin v. L?wis wrote: >> That said, I didn't see any indication of what I consider to be a >> critical failure in PyPI: No dependency metadata prior to downloading >> the package. > > Contributions are welcome. The source code of PyPI is available > publically, Where? > and I'm willing to accept patches. I won't have time > to work on this in the next 12 months myself. These two don't seem to go hand in hand and don't really seem to fit my experiene of the catalog-sig :-( Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Sep 30 22:41:12 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 22:41:12 +0200 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> <20080930175201.106863A409C@sparrow.telecommunity.com> <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> Message-ID: <94bdd2610809301341j60830399s37cf310939783b86@mail.gmail.com> On Tue, Sep 30, 2008 at 8:21 PM, Rob Cakebread wrote: > On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby wrote: >> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >>> >>> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >>> > FWIW, pyinstall can collect all the packages before installing any of >>> > them. You do have to download all packages, though, as that's the only >>> > way to get the metadata. >>> >>> Does the DOAP output for a package not contain enough metadata? >> >> Nope. And it can't possibly do so, unless it contains dependency data for >> every possible variation of the package. For example, a package might >> dynamically declare dependency on ctypes, depending on whether you're >> installing it for Python 2.4 or Python 2.5. (Dependencies can also be >> platform-specific and build-option-specific, as well as >> Python-version-specific.) >> > > Not to mention the DOAP vocabulary lacks a way to describe dependency > information. This is planned but it has to be well thought out because of > all the variations Philip mentions. out of curiosity, : - can a RDF-based database can possibly handle such a graph ? - would it make sense for PyPI to query the doap server to get those dependency infos ? - > > The good news is much of this dependency info is already in existence in > Linux distributions. Take a Gentoo ebuild, for example. It has separate > run-time, build-time and test dependency info, dependencies based on > enabled features, and dependencies based on the version of Python used. > > Ebuilds also have metadata mapping the PyPI name to the Gentoo > package name, so it'll be easy enough to create a database with all this info. > > I'm working on this now at http://doapspace.org/ where you can find DOAP > for Python packages with a bit more metadata than the DOAP supplied on > PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME ) > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From dpeterson at enthought.com Tue Sep 30 22:46:55 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Tue, 30 Sep 2008 15:46:55 -0500 Subject: [Distutils] "just use debian" In-Reply-To: <1222801361.32659.1.camel@shizuru> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> <1222801361.32659.1.camel@shizuru> Message-ID: <48E2903F.9000307@enthought.com> Josselin Mouette wrote: > Le mardi 30 septembre 2008 ? 11:37 -0500, Ian Bicking a ?crit : > >> Say I have a package that represents an application. >> > [snip] > > >> Then HardJSON 2.0 is released, and Turplango only required >> HardJSON>=1.2, so new installations start installing HardJSON 2.0. But >> my app happens not to be compatible with that library, and so it's >> broken. OK... >> > > No, please stop here. That?s not OK. If a new version of HardJSON breaks > your application, it is friggin? broken. If that new version is not > compatible, it should be called HardJSON2, and nothing will break. > I disagree with your assertion that the name HAS to imply API compatibility. There ought to be something that specifies API / ABI compatibility, such as the combination of name and some portion of a version number, but too many people depend on a name for marketing or other purposes for us to impose that it indicate technical aspects. If your OS distribution chooses to do things that way, then fine, when your OS builds the distribution, it can rename it to HardJSON2 but that shouldn't be required of every platform. -- Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Sep 30 22:49:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Sep 2008 22:49:12 +0200 Subject: [Distutils] [Catalog-sig] PEP for distutils In-Reply-To: <48E28B7B.3060602@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de> <48E28B7B.3060602@simplistix.co.uk> Message-ID: <48E290C8.4030008@v.loewis.de> Chris Withers wrote: >> Contributions are welcome. The source code of PyPI is available >> publically, > > Where? https://svn.python.org/packages/trunk/ >> and I'm willing to accept patches. I won't have time >> to work on this in the next 12 months myself. > > These two don't seem to go hand in hand and don't really seem to fit my > experiene of the catalog-sig :-( Saying what? that I do have time in the next twelve months? That's nice to hear. Regards, Martin From setuptools at bugs.python.org Tue Sep 30 23:21:22 2008 From: setuptools at bugs.python.org (Zooko O'Whielacronx) Date: Tue, 30 Sep 2008 21:21:22 +0000 Subject: [Distutils] [issue48] ignores --build-directory= option if argument is a local file In-Reply-To: <1222809682.7.0.213082672644.issue48@psf.upfronthosting.co.za> Message-ID: <1222809682.7.0.213082672644.issue48@psf.upfronthosting.co.za> New submission from Zooko O'Whielacronx : It seems like if the argument to easy_install is a local file instead of a distribution name, then it ignores the --build-directory option: easy_install -v -v -v --prefix=./instdir --build-directory=./builddir ./pyutil-1.3.21.tar.gz | grep ^Running | head -1 Running pyutil-1.3.21/setup.py -vvv bdist_egg --dist-dir /tmp/easy_install-2fiWD4/pyutil-1.3.21/egg-dist-tmp-PU48xQ $ mkdir -p instdir/lib/python2.5/site-packages ; PYTHONPATH=./instdir/lib/python2.5/site-packages easy_install -v -v -v --prefix=./instdir --build-directory=./builddir ./pyutil-1.3.21.tar.gz | grep ^Running | head -1 Running pyutil-1.3.21/setup.py -vvv bdist_egg --dist-dir /tmp/easy_install-8CDXG9/pyutil-1.3.21/egg-dist-tmp-V3Tmzo versus $ mkdir -p instdir/lib/python2.5/site-packages ; PYTHONPATH=./instdir/lib/python2.5/site-packages easy_install -v -v -v --prefix=./instdir --build-directory=./builddir pyutil | grep ^Running | head -1 Running setup.py -vvv bdist_egg --dist-dir ./builddir/pyutil/egg-dist-tmp-sFObAa ---------- messages: 184 nosy: zooko priority: bug status: unread title: ignores --build-directory= option if argument is a local file _______________________________________________ Setuptools tracker _______________________________________________ From joss at debian.org Tue Sep 30 23:32:14 2008 From: joss at debian.org (Josselin Mouette) Date: Tue, 30 Sep 2008 23:32:14 +0200 Subject: [Distutils] "just use debian" In-Reply-To: <48E2903F.9000307@enthought.com> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> <1222801361.32659.1.camel@shizuru> <48E2903F.9000307@enthought.com> Message-ID: <1222810334.32659.18.camel@shizuru> Le mardi 30 septembre 2008 ? 15:46 -0500, Dave Peterson a ?crit : > Josselin Mouette wrote: > > No, please stop here. That?s not OK. If a new version of HardJSON breaks > > your application, it is friggin? broken. If that new version is not > > compatible, it should be called HardJSON2, and nothing will break. > > I disagree with your assertion that the name HAS to imply API > compatibility. There ought to be something that specifies API / ABI > compatibility, such as the combination of name and some portion of a > version number, but too many people depend on a name for marketing or > other purposes for us to impose that it indicate technical aspects. The marketing name does not have to be the same of the name of the module you import. The situation where they differ is even quite common. You can also argue for separating the name from the API version, like the soname of a library, and I?ll agree, but in the end it is very similar. > If your OS distribution chooses to do things that way, then fine, when > your OS builds the distribution, it can rename it to HardJSON2 but > that shouldn't be required of every platform. We can do that, but we won?t as long as it is possible to do otherwise. It completely breaks compatibility with third-party packages or modules, and it is unnecessarily hard to maintain. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From S.Pascoe at rl.ac.uk Tue Sep 30 23:22:29 2008 From: S.Pascoe at rl.ac.uk (Pascoe, S (Stephen)) Date: Tue, 30 Sep 2008 22:22:29 +0100 Subject: [Distutils] Just downloading an egg Message-ID: I often just want to do with easy_install is download an egg from pypi without installing it. I've studied the easy_install documentation and never found a way to do it. Even giving it the "-d" option results in easy-install.pth being created and other unwanted stuff. Looking at the setuptools pydoc I worked out a way to do it: >>> import setuptools >>> d = setuptools.Distribution() >>> d.fetch_build_egg(requirement) Voila! the egg is downloaded into cwd. It even seems built an egg from a tarball. My question is, can I rely on this feature and is it the best way of doing what I want? I'd like to use it in my code and hope it stays. It would be ideal if I could do this through easy_install. Thanks, Stephen. From exarkun at divmod.com Tue Sep 30 23:47:35 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 30 Sep 2008 17:47:35 -0400 Subject: [Distutils] "just use debian" In-Reply-To: <1222810334.32659.18.camel@shizuru> Message-ID: <20080930214735.29191.32031601.divmod.quotient.32359@ohm> On Tue, 30 Sep 2008 23:32:14 +0200, Josselin Mouette wrote: > [snip] > >The marketing name does not have to be the same of the name of the >module you import. The situation where they differ is even quite common. > >You can also argue for separating the name from the API version, like >the soname of a library, and I?ll agree, but in the end it is very >similar. Do you think this is practical for non-trivial libraries? For any library which has more than one API, the possibility exists for one API to change incompatibly and the other to remain compatible. With larger libraries, the value of changing the module name because one (or some other small fraction of the whole) API changed incompatibly decreases as compared to the cost of updating all software which uses the library to use the new name (much of which may well be unaffected by the incompatible change). I am a huge fan of backward compatibility. You may not find a bigger one (at least in the Python community). I can't understand how this approach can be made feasible though. Should the next release of Twisted include a Python packaged named "twisted2" instead of "twisted"? And the one after that "twisted3"? There are thousands of APIs in Twisted, and we do change them incompatibly (after giving notice programmatically for no less than 12 months). Should we instead give up on this and make all users of Twisted update their code to reflect the new name with each release? Jean-Paul From dpeterson at enthought.com Tue Sep 30 23:57:15 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Tue, 30 Sep 2008 16:57:15 -0500 Subject: [Distutils] "just use debian" In-Reply-To: <1222810334.32659.18.camel@shizuru> References: <20080917171851.GA27261@logilab.fr> <20080930084253.GF4494@volans.logilab.fr> <94bdd2610809300505u1291ae6an3c44e2ce31d43e3f@mail.gmail.com> <1222780673.4166.55.camel@shizuru> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> <1222801361.32659.1.camel@shizuru> <48E2903F.9000307@enthought.com> <1222810334.32659.18.camel@shizuru> Message-ID: <48E2A0BB.6090300@enthought.com> Josselin Mouette wrote: > Le mardi 30 septembre 2008 ? 15:46 -0500, Dave Peterson a ?crit : > >> Josselin Mouette wrote: >> >>> No, please stop here. That?s not OK. If a new version of HardJSON breaks >>> your application, it is friggin? broken. If that new version is not >>> compatible, it should be called HardJSON2, and nothing will break. >>> >> I disagree with your assertion that the name HAS to imply API >> compatibility. There ought to be something that specifies API / ABI >> compatibility, such as the combination of name and some portion of a >> version number, but too many people depend on a name for marketing or >> other purposes for us to impose that it indicate technical aspects. >> > > The marketing name does not have to be the same of the name of the > module you import. The situation where they differ is even quite common. > But we already have a separation between project name and module names that are contained within that project. We don't currently declare dependencies on the module names but on the project name. i.e. a dependency on HardJSON > 2.0 does not say anything about what modules you're expecting to import or use, only that you expect to use version 2 of a project called HardJSON. Were you suggesting that change? I think the rest of the comments are easily resolved after the above is clear. -- Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.chauvat at logilab.fr Tue Sep 30 19:51:50 2008 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Tue, 30 Sep 2008 19:51:50 +0200 Subject: [Distutils] package information Message-ID: <20080930175150.GC11672@volans.logilab.fr> As a follow up to the thread named [Fwd: Re: Announcing distribute project] where Tarek ended up proposing a package_info.py file or a PackageInfo class: http://www.logilab.org/cgi-bin/hgwebdir.cgi/logilab/common/file/02993a85ee3c/__pkginfo__.py -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances