From ronaldoussoren at mac.com  Wed Apr  1 00:21:50 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 31 Mar 2009 17:21:50 -0500
Subject: [Distutils] A change to how platform name is generated for
 binary distributions?
In-Reply-To: <49D28773.1070508@enthought.com>
References: <49D28773.1070508@enthought.com>
Message-ID: <4C88B3C0-DE3A-4D66-8320-86918D555D91@mac.com>


On 31 Mar, 2009, at 16:13, Dave Peterson wrote:

> We're starting to try and distribute some pre-built binaries on  
> Solaris and have come across an issue with how pkg_resources /  
> distutils generates the platform specification when the distribution  
> includes Python extensions.
> In particular, we'd like to distribute both x86 and amd64 (or  
> x86_64) binaries for Solaris on Intel since the OS can run in either  
> mode, but all our egg distributions get the same file name no matter  
> what architecture they're actually built for.  Diving into how the  
> platform part of a distribution name gets generated, I find that it  
> relies on the results of 'os.uname' via  
> distutils.util.get_platform().    This seems like it would cause  
> problems on more than just Solaris as it should also happen anytime  
> someone is using a 32-bit Python on a 64-bit OS/CPU, no?

One option for this is to patch pkg_resources.get_build_platform  
(although IMHO it would be better to fix this in distutils itself).  
There's currently some logic for OSX, it should be possible to add  
some more code for solaris that sets the rigth architecture based on  
the detected architecture and sys.maxint (if uname claims your on an  
x86 style CPU and sys.maxint is (much) larger than 2**32 you're on a  
x86_64 system and otherwise you're on a x86/i386 system).

Distutils tries to do this automaticly on OSX, based on the compiler  
flags that are used to build binaries.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090331/83279870/attachment.bin>

From ben+python at benfinney.id.au  Fri Apr  3 02:25:57 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 03 Apr 2009 11:25:57 +1100
Subject: [Distutils] UnicodeDecodeError bug in distutils
References: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com>
	<200702242309.46022.pogonyshev@gmx.net>
	<94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<45E0C012.7090801@palladion.com>
	<5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com>
Message-ID: <877i22fuqy.fsf_-_@benfinney.id.au>

"Phillip J. Eby" <pje at telecommunity.com> writes:

> However, there's currently no standard, as far as I know, for what
> encoding the PKG-INFO file should use.

Who would define such a standard? My vote goes for ?default is UTF-8?.

> Meanwhile, the 'register' command accepts Unicode, but is broken in
> handling it. [?]
> 
> Unfortunately, this isn't fixable until there's a new 2.5.x release.
> For previous Python versions, both register and write_pkg_info()
> accepted 8-bit strings and passed them on as-is, so the only
> workaround for this issue at the moment is to revert to Python 2.4
> or less.

What is the prognosis on this issue? It's still hitting me in Python
2.5.4.

-- 
 \       ?Everything you read in newspapers is absolutely true, except |
  `\        for that rare story of which you happen to have first-hand |
_o__)                                         knowledge.? ?Erwin Knoll |
Ben Finney


From ziade.tarek at gmail.com  Fri Apr  3 10:46:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 3 Apr 2009 10:46:56 +0200
Subject: [Distutils] [Python-Dev] UnicodeDecodeError bug in distutils
In-Reply-To: <877i22fuqy.fsf_-_@benfinney.id.au>
References: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com>
	<200702242309.46022.pogonyshev@gmx.net>
	<94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<45E0C012.7090801@palladion.com>
	<5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com>
	<877i22fuqy.fsf_-_@benfinney.id.au>
Message-ID: <94bdd2610904030146m569aa5a2q5b7bdc542f4570e5@mail.gmail.com>

On Fri, Apr 3, 2009 at 2:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Phillip J. Eby" <pje at telecommunity.com> writes:
>
>> However, there's currently no standard, as far as I know, for what
>> encoding the PKG-INFO file should use.
>
> Who would define such a standard?

PEP 376 where we can explain that all files in egg-info should be in a
specific encoding

> ?My vote goes for ?default is UTF-8?.

+1

>
>> Meanwhile, the 'register' command accepts Unicode, but is broken in
>> handling it. [?]

how so ?

Tarek

From chris at simplistix.co.uk  Fri Apr  3 13:32:57 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 03 Apr 2009 12:32:57 +0100
Subject: [Distutils] [buildout] cool behaviour I didn't know about? :-)
Message-ID: <49D5F3E9.4080502@simplistix.co.uk>

Hi All,

I've been working on getting Zope 2.12 instances to build from a simple 
buildout without any funky recipes.

This morning I tried this at the start:

[buildout]
parts = zeoinstance
extends = 
http://svn.zope.org/*checkout*/Zope/tags/2.12.0a1/versions-zope2.cfg

...expecting buildout to barf 'cos it couldn't find the 
versions-zope3.cfg that's referenced in versions-zope2.cfg.

However, buildout happily did the right thing and downloaded it from
http://svn.zope.org/*checkout*/Zope/tags/2.12.0a1/versions-zope3.cfg

Is this documented, expected behaviour (if so, where and why?) or have I 
just been lucky?

Chris

PS: If it is documented and expected, what would happen if I had my own 
versions-zope3.cfg in the same folder as my buildout.cfg? Which one 
would get picked?

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From setuptools at bugs.python.org  Fri Apr  3 20:54:10 2009
From: setuptools at bugs.python.org (Glyph Lefkowitz)
Date: Fri, 03 Apr 2009 18:54:10 +0000
Subject: [Distutils] [issue67] You can't tell easy_install not to talk to
	the network
In-Reply-To: <1238784849.61.0.0501315327384.issue67@psf.upfronthosting.co.za>
Message-ID: <1238784849.61.0.0501315327384.issue67@psf.upfronthosting.co.za>


New submission from Glyph Lefkowitz <glyph at twistedmatrix.com>:

If I want to use easy_install for nice dependency resolution, there's no way
(that I can figure out, at least) to tell it "never talk to the network, use
only this local directory where you can find dependencies".

----------
messages: 260
nosy: glyph
priority: feature
status: unread
title: You can't tell easy_install not to talk to the network

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue67>
_______________________________________________

From jim at zope.com  Fri Apr  3 21:34:46 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 3 Apr 2009 15:34:46 -0400
Subject: [Distutils] [buildout] cool behaviour I didn't know about? :-)
In-Reply-To: <49D5F3E9.4080502@simplistix.co.uk>
References: <49D5F3E9.4080502@simplistix.co.uk>
Message-ID: <F285C7CC-40C3-47DD-984B-B54163D5797A@zope.com>


On Apr 3, 2009, at 7:32 AM, Chris Withers wrote:

> Hi All,
>
> I've been working on getting Zope 2.12 instances to build from a  
> simple buildout without any funky recipes.
>
> This morning I tried this at the start:
>
> [buildout]
> parts = zeoinstance
> extends = http://svn.zope.org/*checkout*/Zope/tags/2.12.0a1/versions-zope2.cfg
>
> ...expecting buildout to barf 'cos it couldn't find the versions- 
> zope3.cfg that's referenced in versions-zope2.cfg.
>
> However, buildout happily did the right thing and downloaded it from
> http://svn.zope.org/*checkout*/Zope/tags/2.12.0a1/versions-zope3.cfg
>
> Is this documented, expected behaviour (if so, where

Yes. See the two sections starting with:
http://pypi.python.org/pypi/zc.buildout#multiple-configuration-files

> why?)

Why is this documented? Or why does this work?

Documentation is good and the features are useful.

> PS: If it is documented and expected, what would happen if I had my  
> own versions-zope3.cfg in the same folder as my buildout.cfg? Which  
> one would get picked?

Since it is documented, why not read the documentation and find out?

Jim

--
Jim Fulton
Zope Corporation



From jeff at taupro.com  Fri Apr  3 22:17:25 2009
From: jeff at taupro.com (Jeff Rush)
Date: Fri, 03 Apr 2009 15:17:25 -0500
Subject: [Distutils] [issue67] You can't tell easy_install not to talk
 to	the network
In-Reply-To: <1238784849.61.0.0501315327384.issue67@psf.upfronthosting.co.za>
References: <1238784849.61.0.0501315327384.issue67@psf.upfronthosting.co.za>
Message-ID: <49D66ED5.5090702@taupro.com>

Glyph Lefkowitz wrote:
> New submission from Glyph Lefkowitz <glyph at twistedmatrix.com>:
> 
> If I want to use easy_install for nice dependency resolution, there's no way
> (that I can figure out, at least) to tell it "never talk to the network, use
> only this local directory where you can find dependencies".
> 
> ----------
> messages: 260
> nosy: glyph
> priority: feature
> status: unread
> title: You can't tell easy_install not to talk to the network

Sure you can.  To tell it to pull ONLY from a local directory, use:

    easy_install -H None -f /tmp/cachedir SQLObject

and to *generate* that directory when you -are- online, use:

    easy_install -zmaxd /tmp/cachedir SQLObject

It's the '--allow-hosts=None' or '-H None' that does the magic.

BTW my slides and handouts from the tutorial I ran at PyCon about distutils, 
setuptools and buildout are linked to at the bottom of the page:

    http://us.pycon.org/2009/tutorials/schedule/1PM3/

It's broken into four sub-topics of virtualenv, distutils, setuptools and 
buildout and the handout text includes many tips and techniques that are not 
often well known like the one you asked about.

-Jeff

From pje at telecommunity.com  Fri Apr  3 23:32:42 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 03 Apr 2009 17:32:42 -0400
Subject: [Distutils] [issue67] You can't tell easy_install not to talk
 to	the network
In-Reply-To: <49D66ED5.5090702@taupro.com>
References: <1238784849.61.0.0501315327384.issue67@psf.upfronthosting.co.za>
	<49D66ED5.5090702@taupro.com>
Message-ID: <20090403213016.8332D3A40A7@sparrow.telecommunity.com>

By the way, these options and the procedures are also listed in the 
official documentation at:

<http://peak.telecommunity.com/DevCenter/EasyInstall#installing-on-un-networked-machines>http://peak.telecommunity.com/DevCenter/EasyInstall#installing-on-un-networked-machines

and:

<http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts>http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts



At 03:17 PM 4/3/2009 -0500, Jeff Rush wrote:
>Glyph Lefkowitz wrote:
>>New submission from Glyph Lefkowitz <glyph at twistedmatrix.com>:
>>If I want to use easy_install for nice dependency resolution, there's no way
>>(that I can figure out, at least) to tell it "never talk to the network, use
>>only this local directory where you can find dependencies".
>>----------
>>messages: 260
>>nosy: glyph
>>priority: feature
>>status: unread
>>title: You can't tell easy_install not to talk to the network
>
>Sure you can.  To tell it to pull ONLY from a local directory, use:
>
>    easy_install -H None -f /tmp/cachedir SQLObject
>
>and to *generate* that directory when you -are- online, use:
>
>    easy_install -zmaxd /tmp/cachedir SQLObject
>
>It's the '--allow-hosts=None' or '-H None' that does the magic.
>
>BTW my slides and handouts from the tutorial I ran at PyCon about 
>distutils, setuptools and buildout are linked to at the bottom of the page:
>
>    http://us.pycon.org/2009/tutorials/schedule/1PM3/
>
>It's broken into four sub-topics of virtualenv, distutils, 
>setuptools and buildout and the handout text includes many tips and 
>techniques that are not often well known like the one you asked about.
>
>-Jeff
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From asmodai at in-nomine.org  Sat Apr  4 13:14:40 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 4 Apr 2009 13:14:40 +0200
Subject: [Distutils] bdist_egg vs bdist_wininst oddity
Message-ID: <20090404111440.GE13110@nexus.in-nomine.org>

So I was building two new packages for Genshi and encountered this using
setuptools 0.6c9:

Genshi-0.5.1-py2.6-win32.egg
Genshi-0.5.1.win32-py2.6.exe

Why does the python version designation and platform specifier arbitrarily
change place?

Should this be logged as an issue to make sure such filenames are
consistent?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Sometimes I wonder why are we so blind to face...

From pje at telecommunity.com  Sat Apr  4 19:47:40 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 04 Apr 2009 13:47:40 -0400
Subject: [Distutils] bdist_egg vs bdist_wininst oddity
In-Reply-To: <20090404111440.GE13110@nexus.in-nomine.org>
References: <20090404111440.GE13110@nexus.in-nomine.org>
Message-ID: <20090404174513.A1D263A40A7@sparrow.telecommunity.com>

At 01:14 PM 4/4/2009 +0200, Jeroen Ruigrok van der Werven wrote:
>So I was building two new packages for Genshi and encountered this 
>using setuptools 0.6c9: Genshi-0.5.1-py2.6-win32.egg 
>Genshi-0.5.1.win32-py2.6.exe Why does the python version designation 
>and platform specifier arbitrarily change place? Should this be 
>logged as an issue to make sure such filenames are consistent?

If either format is to be changed, it needs to be for a reason more 
important than just being "consistent", as tools exist that parse the 
filenames as they are currently coded.

Since .egg filenames have to be machine-parseable without ambiguity, 
their filename format is fixed, and was chosen in such a way as to 
make parts of the filename optional.  Setuptools also parses 
bdist_wininst filenames and other distutils-generated filenames, 
however, and this is based on their current generating patterns (even 
though individual filenames can be ambiguous as to their meaning).


From ben+python at benfinney.id.au  Sun Apr  5 03:07:26 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 05 Apr 2009 11:07:26 +1000
Subject: [Distutils] UnicodeDecodeError bug in distutils
References: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com>
	<200702242309.46022.pogonyshev@gmx.net>
	<94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<45E0C012.7090801@palladion.com>
	<5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com>
	<877i22fuqy.fsf_-_@benfinney.id.au>
Message-ID: <87iqljc3ht.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> "Phillip J. Eby" <pje at telecommunity.com> writes:
> 
> > Meanwhile, the 'register' command accepts Unicode, but is broken in
> > handling it. [?]
> > 
> > Unfortunately, this isn't fixable until there's a new 2.5.x release.
> > For previous Python versions, both register and write_pkg_info()
> > accepted 8-bit strings and passed them on as-is, so the only
> > workaround for this issue at the moment is to revert to Python 2.4
> > or less.
> 
> What is the prognosis on this issue? It's still hitting me in Python
> 2.5.4.

Any word on this? Is there an open bug tracker issue with more
information? Who's working on this?

-- 
 \      ?If sharing a thing in no way diminishes it, it is not rightly |
  `\                      owned if it is not shared.? ?Saint Augustine |
_o__)                                                                  |
Ben Finney


From martin at v.loewis.de  Sun Apr  5 03:40:22 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 05 Apr 2009 03:40:22 +0200
Subject: [Distutils] [Python-Dev] UnicodeDecodeError bug in distutils
In-Reply-To: <87iqljc3ht.fsf@benfinney.id.au>
References: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>	<94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com>	<200702242309.46022.pogonyshev@gmx.net>	<94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>	<45E0C012.7090801@palladion.com>	<5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com>	<877i22fuqy.fsf_-_@benfinney.id.au>
	<87iqljc3ht.fsf@benfinney.id.au>
Message-ID: <49D80C06.20809@v.loewis.de>

>>> Meanwhile, the 'register' command accepts Unicode, but is broken in
>>> handling it. [?]
>>>
>>> Unfortunately, this isn't fixable until there's a new 2.5.x release.
>>> For previous Python versions, both register and write_pkg_info()
>>> accepted 8-bit strings and passed them on as-is, so the only
>>> workaround for this issue at the moment is to revert to Python 2.4
>>> or less.
>> What is the prognosis on this issue? It's still hitting me in Python
>> 2.5.4.
> 
> Any word on this? Is there an open bug tracker issue with more
> information? Who's working on this?

For Python 2.5.4, no further changes will be made. If you can reproduce
with 2.6, and can't find a tracker issue, make a new report.

Regards,
Martin


From ben+python at benfinney.id.au  Sun Apr  5 04:48:58 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 05 Apr 2009 12:48:58 +1000
Subject: [Distutils] UnicodeDecodeError bug in distutils
References: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com>
	<200702242309.46022.pogonyshev@gmx.net>
	<94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com>
	<45E0C012.7090801@palladion.com>
	<5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com>
	<877i22fuqy.fsf_-_@benfinney.id.au>
	<87iqljc3ht.fsf@benfinney.id.au>
Message-ID: <87eiw7bysl.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Is there an open bug tracker issue with more information?

Answer: <URL:http://bugs.python.org/issue2562>. Apparently the issue
is resolved <URL:http://bugs.python.org/msg72385> for Python 2.6. I
will need to wait for my distribution to catch up before I can know
whether it's resolved.

-- 
 \        ?The World is not dangerous because of those who do harm but |
  `\          because of those who look at it without doing anything.? |
_o__)                                                 ?Albert Einstein |
Ben Finney


From ziade.tarek at gmail.com  Sun Apr  5 18:45:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 5 Apr 2009 18:45:45 +0200
Subject: [Distutils] Deprecate MANIFEST.in
Message-ID: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>

Hello

After some discussions with people at Pycon, we think that the
MANIFEST template should be removed.

I would like to deprecate MANIFEST.in in Distutils, in favor of a
package_data that can match directories with a glob-style pattern
see : http://bugs.python.org/issue5302

If no one objects I will:

- deprecate it with a warning in Python 2.7
- add the glob-style pattern for package_data

Cheers
Tarek


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From marius at pov.lt  Sun Apr  5 22:24:43 2009
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 5 Apr 2009 23:24:43 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
Message-ID: <20090405202443.GA20907@fridge.pov.lt>

On Sun, Apr 05, 2009 at 06:45:45PM +0200, Tarek Ziad? wrote:
> After some discussions with people at Pycon, we think that the
> MANIFEST template should be removed.

+1 for fixing the mess that setuptools + distutils manage to make out of
the manifest (MANIFEST.in + old MANIFEST lying around + .svn metadata =>
lots of confusion and lack of reproducibility)

> I would like to deprecate MANIFEST.in in Distutils, in favor of a
> package_data that can match directories with a glob-style pattern
> see : http://bugs.python.org/issue5302

How would the new syntax look like?

Would it allow you to separately specify the files that

  1) need to be included in the installed Python packages (e.g. 
     templates and static resources for web pages)

  2) need to be included only in the sdist (e.g. HACKING.txt, shell
     scripts for setting up a dev environment, things like buildout.cfg)

> If no one objects I will:
> 
> - deprecate it with a warning in Python 2.7
> - add the glob-style pattern for package_data

Marius Gedminas
-- 
"Linux was made by foreign terrorists to take money from true US companies
like Microsoft." -Some AOL'er.
"To this end we dedicate ourselves..." -Don
(From the sig of "Don" <don at cs.byu.edu>)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090405/9a583dae/attachment.pgp>

From jeremy.kloth at 4suite.org  Sun Apr  5 22:37:25 2009
From: jeremy.kloth at 4suite.org (Jeremy Kloth)
Date: Sun, 5 Apr 2009 14:37:25 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
Message-ID: <200904051437.26386.jeremy.kloth@4suite.org>

On Sunday 05 April 2009 10:45:45 am Tarek Ziad? wrote:
> I would like to deprecate MANIFEST.in in Distutils, in favor of a
> package_data that can match directories with a glob-style pattern
> see : http://bugs.python.org/issue5302

Not all files that are released in an sdist would be considered for 
installation (ala package_data). Think of top-level README, COPYRIGHT, 
LICENSE or files (modules) used only by the setup.py script (ala custom 
commands). It would become an abuse of a keyword `package_data` to indicate 
files that are not actually *package* data.

Remember not all files are meant to be used by the Python code, some, like C 
header files, would only be used at build time. This just strikes me as being 
an over-simplification that really benefits no one.

Note, however, `sdist` does need to be updated to include more files 
automatically, like the aforementioned C header files. With those fixes in 
place, then, maybe, MANIFEST.in would not be required.

-- 
Jeremy Kloth
4Suite Developer	http://4suite.org/

From pje at telecommunity.com  Mon Apr  6 01:49:49 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 05 Apr 2009 19:49:49 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
Message-ID: <20090405234722.A356C3A4063@sparrow.telecommunity.com>

At 06:45 PM 4/5/2009 +0200, Tarek Ziad? wrote:
>Hello
>
>After some discussions with people at Pycon, we think that the
>MANIFEST template should be removed.
>
>I would like to deprecate MANIFEST.in in Distutils, in favor of a
>package_data that can match directories with a glob-style pattern
>see : http://bugs.python.org/issue5302

What about documentation files, data files, examples, and tests -- 
all of which might need to be included in an sdist, but would not be 
"package data"?


From ziade.tarek at gmail.com  Mon Apr  6 01:49:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 01:49:01 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <200904051437.26386.jeremy.kloth@4suite.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
Message-ID: <94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>

On Sun, Apr 5, 2009 at 10:37 PM, Jeremy Kloth <jeremy.kloth at 4suite.org> wrote:
> On Sunday 05 April 2009 10:45:45 am Tarek Ziad? wrote:
>> I would like to deprecate MANIFEST.in in Distutils, in favor of a
>> package_data that can match directories with a glob-style pattern
>> see : http://bugs.python.org/issue5302
>
> Not all files that are released in an sdist would be considered for
> installation (ala package_data). Think of top-level README, COPYRIGHT,
> LICENSE or files (modules) used only by the setup.py script (ala custom
> commands). It would become an abuse of a keyword `package_data` to indicate
> files that are not actually *package* data.
>
> Remember not all files are meant to be used by the Python code, some, like C
> header files, would only be used at build time. This just strikes me as being
> an over-simplification that really benefits no one.

The benefit I am seeing is having one and only one explicit way to
define what goes into your package
distribution. We have many right now, and the worst is :

now people define some packages that depends on the source control
system they use (like svn) !

So basically, if you get a source distribution out there and work on
it in your own DVCS,
you are unable to create a distro.

>
> Note, however, `sdist` does need to be updated to include more files
> automatically, like the aforementioned C header files. With those fixes in
> place, then, maybe, MANIFEST.in would not be required.

Answering to Marius here too.

Right now, when you create a sdist, without any template file, it will
include by default these files
if it finds them :

- README or README.txt
- setup.py
- test/test*.py
- all pure Python modules mentioned in setup script
- all files pointed by package_data (build_py)
- all files defined in data_files.(build_py)
- all files defined as scripts.
- all C sources listed as part of extensions or C libraries
  in the setup script (doesn't catch C headers!)

So basically, build_py does most of the hard work, by computing
package_data and data_files to build the
file list.

What build_py misses is the ability to recursively add elements found
in subdirectories,
which is the new feature I would like to add. People would be able to
provide glob-style patterns,
to macth all their files.

But maybe we would need to define another name for some files that
would not fit under "package_data"
and "data_files"

Last, I find it a bit odd to have README and test/test*.py added
explicitely while others have to be defined.



>
> --
> Jeremy Kloth
> 4Suite Developer ? ? ? ?http://4suite.org/
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Mon Apr  6 01:51:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 05 Apr 2009 19:51:41 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090405202443.GA20907@fridge.pov.lt>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405202443.GA20907@fridge.pov.lt>
Message-ID: <20090405234914.32A633A4108@sparrow.telecommunity.com>

At 11:24 PM 4/5/2009 +0300, Marius Gedminas wrote:
>On Sun, Apr 05, 2009 at 06:45:45PM +0200, Tarek Ziad? wrote:
> > After some discussions with people at Pycon, we think that the
> > MANIFEST template should be removed.
>
>+1 for fixing the mess that setuptools + distutils manage to make out of
>the manifest (MANIFEST.in + old MANIFEST lying around + .svn metadata =>
>lots of confusion and lack of reproducibility)

If you've reported this as a problem or filed a bug for it, I appear 
to have missed it.  Steps to reproduce, and the expected vs. actual 
behavior would be helpful.


From ziade.tarek at gmail.com  Mon Apr  6 01:53:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 01:53:06 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090405234722.A356C3A4063@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
Message-ID: <94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>

2009/4/6 P.J. Eby <pje at telecommunity.com>:
> At 06:45 PM 4/5/2009 +0200, Tarek Ziad? wrote:
>>
>> Hello
>>
>> After some discussions with people at Pycon, we think that the
>> MANIFEST template should be removed.
>>
>> I would like to deprecate MANIFEST.in in Distutils, in favor of a
>> package_data that can match directories with a glob-style pattern
>> see : http://bugs.python.org/issue5302
>
> What about documentation files, data files, examples, and tests -- all of
> which might need to be included in an sdist, but would not be "package
> data"?

that would be data_files, but I am wondering if we should have an explicit
one for documentation files.

From ziade.tarek at gmail.com  Mon Apr  6 02:12:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 02:12:58 +0200
Subject: [Distutils] Distutils register command and reStructuredText
Message-ID: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>

Hello

Right now most people in the community uses reST for their
long_description metadata.
It looks good in PyPI since it's displayed in HTML.

But if the reST is broken, it does not look right. (which happen regularly)

I'd like to add an option in the register command that would verify
that the text is compiling
well using docutils

Any opinion on this ?

Cheers
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Mon Apr  6 02:49:35 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 05 Apr 2009 20:49:35 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
Message-ID: <20090406004710.3D2123A4063@sparrow.telecommunity.com>

At 01:53 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>2009/4/6 P.J. Eby <pje at telecommunity.com>:
> > At 06:45 PM 4/5/2009 +0200, Tarek Ziad? wrote:
> >>
> >> Hello
> >>
> >> After some discussions with people at Pycon, we think that the
> >> MANIFEST template should be removed.
> >>
> >> I would like to deprecate MANIFEST.in in Distutils, in favor of a
> >> package_data that can match directories with a glob-style pattern
> >> see : http://bugs.python.org/issue5302
> >
> > What about documentation files, data files, examples, and tests -- all of
> > which might need to be included in an sdist, but would not be "package
> > data"?
>
>that would be data_files, but I am wondering if we should have an explicit
>one for documentation files.

I mean, *non-installed* documentation, data, examples, and tests -- 
*none* of which would be currently listed in data_files.


From pje at telecommunity.com  Mon Apr  6 02:51:44 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 05 Apr 2009 20:51:44 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
Message-ID: <20090406004917.A5AC83A4063@sparrow.telecommunity.com>

At 01:49 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>So basically, if you get a source distribution out there and work on
>it in your own DVCS,
>you are unable to create a distro.

Why not?  Aren't there setuptools plugins available for the common DVCSes? 


From ziade.tarek at gmail.com  Mon Apr  6 02:55:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 02:55:06 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406004917.A5AC83A4063@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
Message-ID: <94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>

On Mon, Apr 6, 2009 at 2:51 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 01:49 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>>
>> So basically, if you get a source distribution out there and work on
>> it in your own DVCS,
>> you are unable to create a distro.
>
> Why not? ?Aren't there setuptools plugins available for the common DVCSes?

Yes, there are more and more available, but I don't want to rely on a
VCS or a DVCS to build a source distribution,
or to wait for the next plugin if I upgrade my (D)VCS or change it.

It doesn't make sense to  have the list of the files in .svn or .hg
files. for your package.
Plus, some svn viewers will not show you the .svn, so you ware unable
to now what gets into the dist.

It should be agnostic imho and explicitely defined in the package.

From lists at jwp.name  Mon Apr  6 02:41:14 2009
From: lists at jwp.name (James Pye)
Date: Sun, 5 Apr 2009 17:41:14 -0700
Subject: [Distutils] Distutils register command and reStructuredText
In-Reply-To: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>
References: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>
Message-ID: <1AF65F76-FC2D-4B9E-86B2-8D5B5D1D79EB@jwp.name>

On Apr 5, 2009, at 5:12 PM, Tarek Ziad? wrote:
> Any opinion on this ?

I'm +0.5 on it as I'm inclined to think that a "./setup.py check" may  
be in order for things like this.

Although, wouldn't such a check require a dependency on docutils? Or  
would this be a pypi, catalog, feature?

That is, I have been bitten by this, so some feedback from the tool  
would be handy..


An alternative may be to explicitly upload a long_description via the  
web interface where the user can get a preview prior to publishing it..

From ziade.tarek at gmail.com  Mon Apr  6 03:11:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 03:11:49 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406004710.3D2123A4063@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
Message-ID: <94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>

On Mon, Apr 6, 2009 at 2:49 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 01:53 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>>
>> 2009/4/6 P.J. Eby <pje at telecommunity.com>:
>> > At 06:45 PM 4/5/2009 +0200, Tarek Ziad? wrote:
>> >>
>> >> Hello
>> >>
>> >> After some discussions with people at Pycon, we think that the
>> >> MANIFEST template should be removed.
>> >>
>> >> I would like to deprecate MANIFEST.in in Distutils, in favor of a
>> >> package_data that can match directories with a glob-style pattern
>> >> see : http://bugs.python.org/issue5302
>> >
>> > What about documentation files, data files, examples, and tests -- all
>> > of
>> > which might need to be included in an sdist, but would not be "package
>> > data"?
>>
>> that would be data_files, but I am wondering if we should have an explicit
>> one for documentation files.
>
> I mean, *non-installed* documentation, data, examples, and tests -- *none*
> of which would be currently listed in data_files.

Right,
that would require something else, maybe a new one, ignored by the
install command
and using the glob-style pattern.

This means the glob-style pattern will need to have an exclude mechanism
if some non-installed data are inside a directory that is added in package_data
(like the tests directory if it's located in the src directory)

From ben+python at benfinney.id.au  Mon Apr  6 05:20:38 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 06 Apr 2009 13:20:38 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
Message-ID: <87tz52a2nt.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> It doesn't make sense to have the list of the files in .svn or .hg
> files. for your package.

Again, why not? If I'm using a VCS for my source files, then the VCS
is the Single Point of Truth for the inventory of source files.

That doesn't mean the VCS metadata goes *into* the source package; it
means that {distutils, setuptools} *queries* the VCS to get the
inventory when building the source package.

> It should be agnostic imho and explicitely defined in the package.

That duplicates state information in multiple locations, which means
it's either going to get out of date, or is mechanically duplicated
for little benefit.

-- 
 \        ?Of course, everybody says they're for peace. Hitler was for |
  `\      peace. Everybody is for peace. The question is: what kind of |
_o__)                                peace?? ?Noam Chomsky, 1984-05-14 |
Ben Finney


From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 05:38:24 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 12:38:24 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87tz52a2nt.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
Message-ID: <49D97930.3050703@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>   
>> It doesn't make sense to have the list of the files in .svn or .hg
>> files. for your package.
>>     
>
> Again, why not? If I'm using a VCS for my source files, then the VCS
> is the Single Point of Truth for the inventory of source files.
>   

depending on your definition of sources, it is not. The VCS is there to
track a project, but sdist is a *distribution* mean. It is a packaged
version of your software - simple, but packaged nonetheless. There has
to be a (potential) difference between what goes in sdist and what is
recorded in the VCS. Python packaging is the only tool I am aware which
say both are equal.

Specially with DVCS, I would think using the VCS for sdist makes really
little sense - if you want the whole sources, just use
clone/branch/checkout, or the automatically generated tarball as
available from most web tools (gitweb, hgweb, etc...).

The duplication argument could be made the other way: why duplicating
with sdist what the VCS offers you ?

cheers,

David

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 05:42:50 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 12:42:50 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
Message-ID: <49D97A3A.7030205@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> Hello
>
> After some discussions with people at Pycon, we think that the
> MANIFEST template should be removed.
>   

I am all for removing MANIFEST.in, but we should first make sure all its
uses are covered by distutils. Currently, they are far from being
fullfilled. The ones which come to mind are:
    - a reliable way to include package_data
    - a reliable way to add documentation (and install them somewhere
which can be customized at build time and installation time)

It is important to be able to notify where to put them in sdist and at
installation time,

cheers,

David

From ziade.tarek at gmail.com  Mon Apr  6 06:40:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 06:40:58 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D97A3A.7030205@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D97A3A.7030205@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610904052140he1dbf90ked6e388961c70593@mail.gmail.com>

On Mon, Apr 6, 2009 at 5:42 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Tarek Ziad? wrote:
>> Hello
>>
>> After some discussions with people at Pycon, we think that the
>> MANIFEST template should be removed.
>>
>
> I am all for removing MANIFEST.in, but we should first make sure all its
> uses are covered by distutils. Currently, they are far from being
> fullfilled. The ones which come to mind are:
> ? ?- a reliable way to include package_data

what do you mean ? (beside the feature that is decribed in the ticket
- which enables adding files recursively)

> ? ?- a reliable way to add documentation (and install them somewhere
> which can be customized at build time and installation time)

Yes, beside the "how to include it in sdist" there's the "where to install it"

This is somehow related to the work started here
http://wiki.python.org/moin/Distutils/StaticMetadata

Tres and Matthias were talking about adding an "install_doc" new command.

Cheers,
Tarek

From ziade.tarek at gmail.com  Mon Apr  6 06:44:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 06:44:18 +0200
Subject: [Distutils] Distutils register command and reStructuredText
In-Reply-To: <1AF65F76-FC2D-4B9E-86B2-8D5B5D1D79EB@jwp.name>
References: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>
	<1AF65F76-FC2D-4B9E-86B2-8D5B5D1D79EB@jwp.name>
Message-ID: <94bdd2610904052144x34089eawc41f7f2156647d96@mail.gmail.com>

On Mon, Apr 6, 2009 at 2:41 AM, James Pye <lists at jwp.name> wrote:
> On Apr 5, 2009, at 5:12 PM, Tarek Ziad? wrote:
>>
>> Any opinion on this ?
>
> I'm +0.5 on it as I'm inclined to think that a "./setup.py check" may be in
> order for things like this.

I guess you can already do this with:

$ python setup.py --long-description | rst2html.py > /dev/null

you'll get any warning/error in the stdout.

>
> Although, wouldn't such a check require a dependency on docutils? Or would
> this be a pypi, catalog, feature?

A docutils dependency, on client side, trigger at register time, if
docutils is installed.

>
> That is, I have been bitten by this, so some feedback from the tool would be
> handy..
>
>
> An alternative may be to explicitly upload a long_description via the web
> interface where the user can get a preview prior to publishing it..

Maybe so. That said, I (and the people I know); never use the web UI
to register/upload a package at PyPI
the command line is faster

Cheers
Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ben+python at benfinney.id.au  Mon Apr  6 07:13:53 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 06 Apr 2009 15:13:53 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
Message-ID: <87prfq9xf2.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Ben Finney wrote:
> > Tarek Ziad? <ziade.tarek at gmail.com> writes:
> >> It doesn't make sense to have the list of the files in .svn or
> >> .hg files. for your package.
> >
> > Again, why not? If I'm using a VCS for my source files, then the
> > VCS is the Single Point of Truth for the inventory of source
> > files.
> 
> depending on your definition of sources, it is not. The VCS is there to
> track a project, but sdist is a *distribution* mean. It is a packaged
> version of your software - simple, but packaged nonetheless. There has
> to be a (potential) difference between what goes in sdist and what is
> recorded in the VCS. Python packaging is the only tool I am aware which
> say both are equal.

Okay, but that's a far cry from saying that it ?doesn't make sense? to
use the VCS inventory.

> The duplication argument could be made the other way: why
> duplicating with sdist what the VCS offers you ?

Yes, that's exactly my point.

The sdist needs to be generated and distributed, since that's the
common interface for recipients to get at the software for
installation. My point is that that process of generating the sdist
should re-use available information, e.g. from the existing VCS
information which the developer has access to, it should be used from
that source.

-- 
 \     ?Pinky, are you pondering what I'm pondering?? ?Um, I think so, |
  `\    Brainie, but why would anyone want to Pierce Brosnan?? ?_Pinky |
_o__)                                                   and The Brain_ |
Ben Finney


From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 07:11:40 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 14:11:40 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87prfq9xf2.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
Message-ID: <49D98F0C.9090505@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
> David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:
>
>   
>> Ben Finney wrote:
>>     
>>> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>>>       
>>>> It doesn't make sense to have the list of the files in .svn or
>>>> .hg files. for your package.
>>>>         
>>> Again, why not? If I'm using a VCS for my source files, then the
>>> VCS is the Single Point of Truth for the inventory of source
>>> files.
>>>       
>> depending on your definition of sources, it is not. The VCS is there to
>> track a project, but sdist is a *distribution* mean. It is a packaged
>> version of your software - simple, but packaged nonetheless. There has
>> to be a (potential) difference between what goes in sdist and what is
>> recorded in the VCS. Python packaging is the only tool I am aware which
>> say both are equal.
>>     
>
> Okay, but that's a far cry from saying that it ?doesn't make sense? to
> use the VCS inventory.
>   

Not that a far cry: python packaging tools are the only systems that I
know of which does this. For example, in autotools, the tarball
generated by make dist is never generated from the VCS.

I may have some weird tools in my VCS, or some huge test data, things
which do not make sense to distribute.

>   
>> The duplication argument could be made the other way: why
>> duplicating with sdist what the VCS offers you ?
>>     
>
> Yes, that's exactly my point.
>   

Maybe I was not clear, but my point cannot be further from your
suggestion :) sdist should not use the VCS, because a source tarball
generated from the VCS and from sdist are not the same thing at all. If
you want a source checkout (which is what sdist does if you use the
VCS), then use the VCS. Most good systems offer a service to
automatically generate the source from the VCS if you need to make it
available to people wo the VCS (trac, git and hg web frontends, etc...).

cheers,

David

From ben+python at benfinney.id.au  Mon Apr  6 08:02:08 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 06 Apr 2009 16:02:08 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
Message-ID: <87ljqe9v6n.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Ben Finney wrote:
> > Okay, but that's a far cry from saying that it ?doesn't make
> > sense? to use the VCS inventory.
> 
> Not that a far cry: python packaging tools are the only systems that
> I know of which does this. For example, in autotools, the tarball
> generated by make dist is never generated from the VCS.

I'm not advocating using the VCS to *generate the tarball*. I'm
talking about using the VCS to *determine what files to put in the
tarball*.

> I may have some weird tools in my VCS, or some huge test data,
> things which do not make sense to distribute.

Okay. I was addressing the claim that it makes no sense (i.e. it's
*always* wrong) to use the VCS inventory.

> Maybe I was not clear, but my point cannot be further from your
> suggestion :) sdist should not use the VCS, because a source tarball
> generated from the VCS and from sdist are not the same thing at all.

Your leap from ?use the VCS inventory? to ?cause the VCS to generate
the sdist tarball?, which is wrong.

I'm saying sdist should (in some cases, i.e. it makes sense sometimes,
i.e. it's not something that doesn't make sense) use the VCS inventory
to know what files form the source distribution.

> If you want a source checkout (which is what sdist does if you use
> the VCS), then use the VCS. Most good systems offer a service to
> automatically generate the source from the VCS if you need to make
> it available to people wo the VCS (trac, git and hg web frontends,
> etc...).

As you're no doubt aware, there's more to making an sdist than merely
making a tarball of the files; if nothing else, the metadata needs to
be assembled by {distutils, setuptools} and isn't stored directly in
the VCS. So I'm really not following your point here.

-- 
 \     ?I was trying to daydream, but my mind kept wandering.? ?Steven |
  `\                                                            Wright |
_o__)                                                                  |
Ben Finney


From regebro at gmail.com  Mon Apr  6 08:08:54 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 08:08:54 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87ljqe9v6n.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
Message-ID: <319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>

On Mon, Apr 6, 2009 at 08:02, Ben Finney <ben+python at benfinney.id.au> wrote:
> I'm not advocating using the VCS to *generate the tarball*. I'm
> talking about using the VCS to *determine what files to put in the
> tarball*.

What's the difference?


Personally, I agree that if there is a VCS, and no other information
on what files to include, using the VCS file info makes sense to me.

I don't understand a large part of this discussion, but it seems to me
that the expected and reasonable way of determening what files to
include would be to:

1. Specify files in setup.py
2. Use the files that is a part of the VCS.
3. Include all files, except *.pyc, *.bak and other well known
extensions for backup files or intermediary/temporary files.

I don't personally see the reason for any other ways of specifying
files, in particularly not a MANIFEST file, which I never saw the
point of and always found confusing, and hence never have used. I have
also never understood why distutils is so picky on what it includes.

Just my 2 cents.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 08:02:18 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 15:02:18 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87ljqe9v6n.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
Message-ID: <49D99AEA.3090207@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
>
> I'm not advocating using the VCS to *generate the tarball*. I'm
> talking about using the VCS to *determine what files to put in the
> tarball*.
>   

Currently, using the vcs plugin does include everything as far as I
know. So what do you have in mind ?

David

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 08:05:35 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 15:05:35 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904052140he1dbf90ked6e388961c70593@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<49D97A3A.7030205@ar.media.kyoto-u.ac.jp>
	<94bdd2610904052140he1dbf90ked6e388961c70593@mail.gmail.com>
Message-ID: <49D99BAF.3080006@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> what do you mean ? (beside the feature that is decribed in the ticket
> - which enables adding files recursively)
>   

I mean something more fine-grained than package_data, and more explicit.
So that we can separate tests, documentation, etc... as every linux
packaging expects. For people who do not care, we can keep the
package_data which put everything together.

cheers,

David

From ben+python at benfinney.id.au  Mon Apr  6 08:40:58 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 06 Apr 2009 16:40:58 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
Message-ID: <87hc129tdx.fsf@benfinney.id.au>

Lennart Regebro <regebro at gmail.com> writes:

> On Mon, Apr 6, 2009 at 08:02, Ben Finney <ben+python at benfinney.id.au> wrote:
> > I'm not advocating using the VCS to *generate the tarball*. I'm
> > talking about using the VCS to *determine what files to put in the
> > tarball*.
> 
> What's the difference?

The former is having the sdist process tell the VCS ?generate a
tarball export of the versioned files? which isn't much use since the
result still isn't a complete sdist.

The latter is having the sdist process query the VCS ?tell me the set
of version files by name? and then using that information in the rest
of the normal sdist creation process.


David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Currently, using the vcs plugin does include everything as far as I
> know. So what do you have in mind ?

That, in the presence of a VCS and in the absence of a ?MANIFEST.in?
file, the ?create an sdist? procedure should query the VCS inventory
for the information it would otherwise get from the ?MANIFEST.in?.
Thus making that file unnecessary in many cases that I care about.

-- 
 \         ?A fine is a tax for doing wrong. A tax is a fine for doing |
  `\                                                 well.? ?anonymous |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Mon Apr  6 09:23:28 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 09:23:28 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
Message-ID: <94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>

2009/4/6 Lennart Regebro <regebro at gmail.com>:
> On Mon, Apr 6, 2009 at 08:02, Ben Finney <ben+python at benfinney.id.au> wrote:
>> I'm not advocating using the VCS to *generate the tarball*. I'm
>> talking about using the VCS to *determine what files to put in the
>> tarball*.
>
> What's the difference?
>
>
> Personally, I agree that if there is a VCS, and no other information
> on what files to include, using the VCS file info makes sense to me.
>
> I don't understand a large part of this discussion, but it seems to me
> that the expected and reasonable way of determening what files to
> include would be to:
>
> 1. Specify files in setup.py
> 2. Use the files that is a part of the VCS.

What if you don't want to include some files that are present in the VCS ?
This is an implicit behavior.

Tarek

From regebro at gmail.com  Mon Apr  6 09:29:28 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 09:29:28 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87hc129tdx.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au>
Message-ID: <319e029f0904060029wf52ab9apf3426440cb4f0407@mail.gmail.com>

On Mon, Apr 6, 2009 at 08:40, Ben Finney <ben+python at benfinney.id.au> wrote:
> Lennart Regebro <regebro at gmail.com> writes:
>
>> On Mon, Apr 6, 2009 at 08:02, Ben Finney <ben+python at benfinney.id.au> wrote:
>> > I'm not advocating using the VCS to *generate the tarball*. I'm
>> > talking about using the VCS to *determine what files to put in the
>> > tarball*.
>>
>> What's the difference?
>
> The former is having the sdist process tell the VCS ?generate a
> tarball export of the versioned files? which isn't much use since the
> result still isn't a complete sdist.

I didn't even know you could do that.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr  6 09:30:34 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 09:30:34 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
Message-ID: <319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>

On Mon, Apr 6, 2009 at 09:23, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> I don't understand a large part of this discussion, but it seems to me
>> that the expected and reasonable way of determening what files to
>> include would be to:
>>
>> 1. Specify files in setup.py
>> 2. Use the files that is a part of the VCS.
>
> What if you don't want to include some files that are present in the VCS ?

Well, then you need to specify stuff in setup.py.

> This is an implicit behavior.

It's a practical default.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Mon Apr  6 09:51:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 09:51:19 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
Message-ID: <94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>

On Mon, Apr 6, 2009 at 9:30 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Apr 6, 2009 at 09:23, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> I don't understand a large part of this discussion, but it seems to me
>>> that the expected and reasonable way of determening what files to
>>> include would be to:
>>>
>>> 1. Specify files in setup.py
>>> 2. Use the files that is a part of the VCS.
>>
>> What if you don't want to include some files that are present in the VCS ?
>
> Well, then you need to specify stuff in setup.py.

So part of your file list is declared implicitely in your (D)VCS and
part explicitely in your setup.
This is getting complicated.

>
>> This is an implicit behavior.
>
> It's a practical default.

For sure. But that means there will be still more than one way to do it

Unless the resulting MANIFEST file is always stored in the (D)VCS
besides setup.py maybe..

Tarek

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 09:45:45 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 16:45:45 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
Message-ID: <49D9B329.6040504@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> It's a practical default.
>   

It is practical on the short term, but a pain in the long term. The
single most problematic aspect of distutils (and the tools above it) is
all this magic which works in some simple cases, and breaks in other,
without any way to circumvent it. A distribution tool has to be simple
and customizable, not magic.

For example, all those different, slightly different behaviors of what
goes where means that depending on whether you use distutils, setup.py
sdist, paver sdist, setuptools setup.py sdist, your tarball is
different. That's just insane, and the only way to make it work is to
have *one simple way* to tell what goes where. Distutils should be
simplified, not made even more complicated. Tarball generation is
typically an area where there is too much magic.

If there is one simple way to make sdist, then you can build your own
tools to integrate with your DVCS if you want. But the data produced and
fed by/from tools have to be simple. Put the magic in the tools, not in
the data :)

cheers,

David


From regebro at gmail.com  Mon Apr  6 10:05:17 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 10:05:17 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
Message-ID: <319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>

On Mon, Apr 6, 2009 at 09:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> So part of your file list is declared implicitely in your (D)VCS and
> part explicitely in your setup.

I don't see how that would work. Reasonably, the VCS is a fallback of
nothing is declared in setup.py. I don't see why you should be forced
to explicitly declare every file in the setup.py, a sensible default
is not a problem.

> For sure. But that means there will be still more than one way to do it

Not really. There is one way to specify which files are included. If
it is not specified, distutils would make the reasonable assumption
that all versioned files should be included.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr  6 10:06:55 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 10:06:55 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9B329.6040504@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>

2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> It is practical on the short term, but a pain in the long term. The
> single most problematic aspect of distutils (and the tools above it) is
> all this magic which works in some simple cases, and breaks in other,
> without any way to circumvent it.

So make a way to circumvent it then.

> A distribution tool has to be simple
> and customizable, not magic.

Yes. And can you point out what in my suggestion is not simple, and
what is magic, and what is not circumventable?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Mon Apr  6 10:14:53 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 10:14:53 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
Message-ID: <94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>

On Mon, Apr 6, 2009 at 10:05 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Apr 6, 2009 at 09:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> So part of your file list is declared implicitely in your (D)VCS and
>> part explicitely in your setup.
>
> I don't see how that would work. Reasonably, the VCS is a fallback of
> nothing is declared in setup.py.

Ok so, in version 1.2 of your package, you use the implicit way (VCS based)
then you introduce a new file, that forces your to be explicit in 1.3
so you use setup.py
then in 1.4 you're back in an implicit list...  this is not a
long-term solution indeed.

> I don't see why you should be forced
> to explicitly declare every file in the setup.py, a sensible default
> is not a problem.

With glob-style patterns this is not a burden at all. and it's all in
clear text there,
not depending on any magic.

Cheers
Tarej

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 10:07:55 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 17:07:55 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87tz52a2nt.fsf@benfinney.id.au>	
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	
	<87prfq9xf2.fsf@benfinney.id.au>	
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>	
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>	
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
Message-ID: <49D9B85B.8090805@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> Yes. And can you point out what in my suggestion is not simple, and
> what is magic, and what is not circumventable?
>   
 - Not simple: you need a plugin for each VCS. To be consistent, it
needs to be included in distutils, so when the VCS format changes, it
breaks - you can't update it easily because it is in the stdlib. And if
the sources are not under a VCS, it breaks, again.
 - Magic: it depends on the VCS. So if the same sources is under a
different VCS, it may be different. How do you handle ignored files ? Etc...
 
I have experienced all those problems for almost any package I am
distributing, be it big (numpy/scipy) or small (my owns).

Now, if on the contrary, you have a simple format, then you can always
have your own tool to generate this format from the DVCS. Why should
python be different from every other type of software ?

cheers,

David





From regebro at gmail.com  Mon Apr  6 11:03:45 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 11:03:45 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
Message-ID: <319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>

On Mon, Apr 6, 2009 at 10:14, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Ok so, in version 1.2 of your package, you use the implicit way (VCS based)
> then you introduce a new file, that forces your to be explicit in 1.3

Why?

> so you use setup.py
> then in 1.4 you're back in an implicit list... ?this is not a
> long-term solution indeed.

No, that's a very strange solution. I don't believe I have suggested
anything like that.

> With glob-style patterns this is not a burden at all. and it's all in
> clear text there,
> not depending on any magic.

Could we in this discussion, instead of making fuzzy general
statements explain exactly what the problems are?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr  6 11:06:17 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 11:06:17 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>

2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> To be consistent, it
> needs to be included in distutils, so when the VCS format changes, it
> breaks - you can't update it easily because it is in the stdlib.

Sure, if the plugin can't use a stable API then it should not be
included in the stdlib.

> And if the sources are not under a VCS, it breaks, again.

How does it break?

> ?- Magic: it depends on the VCS. So if the same sources is under a
> different VCS, it may be different. How do you handle ignored files ? Etc...

How is this different from any non-VCS situation?

> Now, if on the contrary, you have a simple format, then you can always
> have your own tool to generate this format from the DVCS. Why should
> python be different from every other type of software ?

Can you explain what is different and when?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Mon Apr  6 11:10:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 11:10:51 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
Message-ID: <94bdd2610904060210g3886d42bjbd0c88d1992307d7@mail.gmail.com>

On Mon, Apr 6, 2009 at 11:03 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Apr 6, 2009 at 10:14, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> Ok so, in version 1.2 of your package, you use the implicit way (VCS based)
>> then you introduce a new file, that forces your to be explicit in 1.3
>
> Why?
>

because in 1.2 you don't want to add one file that is present in your VCS.

>> so you use setup.py
>> then in 1.4 you're back in an implicit list... ?this is not a
>> long-term solution indeed.
>
> No, that's a very strange solution. I don't believe I have suggested
> anything like that.

I am just describing a scenario using your logic.

I guess I'll write something down, it's simpler

Cheers
Tarek

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 11:06:44 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 18:06:44 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87prfq9xf2.fsf@benfinney.id.au>	
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>	
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>	
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>	
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>	
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
Message-ID: <49D9C624.7030305@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> Sure, if the plugin can't use a stable API then it should not be
> included in the stdlib.
>   

I was not talking about the plugin stability, but about the situation
where the VCS detection is broken. As it has happened with setuptools
very recently.

>
> How does it break?
>   

If I have say the git plugin, and I run sdist in a git repo, but later
someone run sdist from my tarball: how can I be sure it is the same ?

> How is this different from any non-VCS situation?
>   

It is totally different: in the simple situation, I say which file to
include/not to include. So it is reproducible, whatever the situation
is. I won't have some ignored files because my own import does not have
the same ignored files as the original package.

>
> Can you explain what is different and when?
>   

No other distribution mechanism that I know of uses magic to detect
which files to distribute. Neither debian tools, nor rpm tools, nor
autotools. Not being explicit about files for distribution is a terrible
idea; a packaging tool should be explicit, to avoid as much as possible
to do something unexpected.

cheers,

David


From marius at pov.lt  Mon Apr  6 11:25:19 2009
From: marius at pov.lt (Marius Gedminas)
Date: Mon, 6 Apr 2009 12:25:19 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090405234914.32A633A4108@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405202443.GA20907@fridge.pov.lt>
	<20090405234914.32A633A4108@sparrow.telecommunity.com>
Message-ID: <20090406092519.GA6527@fridge.pov.lt>

On Sun, Apr 05, 2009 at 07:51:41PM -0400, P.J. Eby wrote:
> At 11:24 PM 4/5/2009 +0300, Marius Gedminas wrote:
>> On Sun, Apr 05, 2009 at 06:45:45PM +0200, Tarek Ziad? wrote:
>> > After some discussions with people at Pycon, we think that the
>> > MANIFEST template should be removed.
>>
>> +1 for fixing the mess that setuptools + distutils manage to make out of
>> the manifest (MANIFEST.in + old MANIFEST lying around + .svn metadata =>
>> lots of confusion and lack of reproducibility)
>
> If you've reported this as a problem or filed a bug for it, I appear to 
> have missed it.  Steps to reproduce, and the expected vs. actual  
> behavior would be helpful.

Sorry about that, I though it was working as designed (having seen
discussions about it here on this list).

  1. Build a package from an svn checkout -> it works
  2. Build a package from a tree exported with svn export -> it fails,
     since the MANIFEST.in is nonexistent/incomplete, but you never
     notice that while developing
  3. Change the test fixture to ignore .svn subdirectories when tarring
     up the working tree and copying it into the virtual machine for
     building (cannot use svn export here, since the tests need to test
     your working tree with uncommitted changes).  Watch it *not* fail.
     Spend an hour until you figure out the old *output* file
     pkg.egg-info/MANIFEST is now suddenly an *input* file for the
     manifest generation process and therefore masks errors in your
     MANIFEST.in.  Change the tar logic to skip *.egg-info, watch the
     test fail, fix MANIFEST.in, watch the test pass, be happy.

Usage of .svn dirs and including the previous version of MANIFEST when
generating a new one are features intended to make the life simpler for
the developer, but when they break down, it's not fun figuring out
what happened or why.

Do you want me to report this as a bug?  The proposed fix cannot be
"remove code dealing with .svn and reading the old MANIFEST, instead
require MANIFEST.in to be always complete"---that would break backwards
compatibility and make people unhappy.

Tarek's plan of replacing MANIFEST.in with a new mechanism sounds like
an opportunity to get rid of the magic, whence my +1 with the ill-phrased
rationale.

Marius Gedminas
-- 
"Linux: the operating system with a CLUE... Command Line User Environment".
(seen in a posting in comp.software.testing)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090406/639e0d25/attachment.pgp>

From a.cavallo at acm.org  Mon Apr  6 11:25:26 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Mon, 6 Apr 2009 10:25:26 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
Message-ID: <a71153f0904060225p5c7cab9en26e872d16c66f31e@mail.gmail.com>

I know it might look sooo web 0.1 but has anyone thought to put
the whole sdist stuff under a flow chart (as well the distutils stuffs)?

Much of this discussion it would be easier in some way.


About the vcs magics.

Some vcs systems do not have "special" directories under the
source structures (it come to my mind perforce): they
hold meta data into a remote server.


Regards,
Antonio Cavallo


> Could we in this discussion, instead of making fuzzy general
> statements explain exactly what the problems are?

From marius at pov.lt  Mon Apr  6 11:28:42 2009
From: marius at pov.lt (Marius Gedminas)
Date: Mon, 6 Apr 2009 12:28:42 +0300
Subject: [Distutils] Distutils register command and reStructuredText
In-Reply-To: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>
References: <94bdd2610904051712j7b9718dwd2fd29408d9c4710@mail.gmail.com>
Message-ID: <20090406092842.GB6527@fridge.pov.lt>

On Mon, Apr 06, 2009 at 02:12:58AM +0200, Tarek Ziad? wrote:
> Right now most people in the community uses reST for their
> long_description metadata.  It looks good in PyPI since it's displayed
> in HTML.
> 
> But if the reST is broken, it does not look right. (which happen regularly)
> 
> I'd like to add an option in the register command that would verify
> that the text is compiling well using docutils

I assume by "verify" you mean it would fail register if setup.py passes
something like long_description_format="reST" to setup()?

+1

Marius Gedminas
-- 
ultimate_answer_t deep_thought(void) { sleep(years2secs(7500000)); return 42; }
        -- ConceptJunkie on Slashdot
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090406/6a8054d1/attachment-0001.pgp>

From jeff at taupro.com  Mon Apr  6 12:18:41 2009
From: jeff at taupro.com (Jeff Rush)
Date: Mon, 06 Apr 2009 05:18:41 -0500
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9C624.7030305@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>		<87prfq9xf2.fsf@benfinney.id.au>		<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>		<87ljqe9v6n.fsf@benfinney.id.au>		<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>		<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>		<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>		<49D9B329.6040504@ar.media.kyoto-u.ac.jp>		<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>		<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
Message-ID: <49D9D701.7090503@taupro.com>

David Cournapeau wrote:
> 
> No other distribution mechanism that I know of uses magic to detect
> which files to distribute. Neither debian tools, nor rpm tools, nor
> autotools. Not being explicit about files for distribution is a terrible
> idea; a packaging tool should be explicit, to avoid as much as possible
> to do something unexpected.

While true that the rpm tools do not magically detect which files to 
distribute, there are many that wish it did and have tried various approaches 
to adding it.  The following except from the RPM book speaks of the various, 
failed approaches.  Still it speaks of a essential need on the part of 
developers to automate this aspect of packaging, even if the perfect solution 
has not yet been found.

If we're going to be explicit on what to include, I would argue we should NOT 
allow wildcards, since in subsequent uses of sdist by another person, it could 
cause files to be included that otherwise would not be there.  The only way to 
be sure precisely the same set of files is always included is to explicitly 
list them one by one.  A big pain.


--- cut here --- excerpt from Maximum RPM book

How Do You Create the File List?

Since RPM automates so many aspects of software installation, it's easy to 
fall into the trap of assuming that RPM does everything for you. Not so! One 
task that is still a manual process is creating the file list. While it may 
seem at first glance, that it could be automated somehow, it's actually a more 
difficult problem than it seems.

Since the majority of an application's files are installed by its makefile, 
RPM has no control over that part of the build process, and therefore, cannot 
automatically determine which files should be part of the package. Some people 
have attempted to use a modified version of install that logs the name of 
every file it installs. But not every makefile uses install, or if it does, 
uses it sporadically.

Another approach tried was to obtain a list of every file on the build system, 
immediately before and after a build, and use the differences as the file 
list. While this approach will certainly find every file that the application 
installed, it can also pick up extraneous files, such as system logs, files in 
/tmp, and the like. The only way to begin to make this approach workable would 
be to do nothing else on the build system, which is highly inconvenient. This 
approach also precludes building more than one package on the system at any 
given time.

At present, the best way to create the file list is to read the makefile to 
see what files it installs, verify this against the files installed on the 
build system, and create the list.


From a.cavallo at acm.org  Mon Apr  6 12:53:34 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Mon, 6 Apr 2009 11:53:34 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9D701.7090503@taupro.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<49D9D701.7090503@taupro.com>
Message-ID: <a71153f0904060353w5f22b332hed0bc91f4b4f78f0@mail.gmail.com>

rpm(build) is a build tool, doens't deal with source distribution (but
it generates srpms
from the build sources).

There's a mild autodetection in rpmbuild when used in a restricted environment:
it detects installed but unpackaged files under $RPM_BUILD_ROOT.
It fails apart when used to build software under root.

I still think that a flowchart wuold be beneficial in identify the
usage scenarios better.


Regards,
Antonio


On Mon, Apr 6, 2009 at 11:18 AM, Jeff Rush <jeff at taupro.com> wrote:
> David Cournapeau wrote:
>>
>> No other distribution mechanism that I know of uses magic to detect
>> which files to distribute. Neither debian tools, nor rpm tools, nor
>> autotools. Not being explicit about files for distribution is a terrible
>> idea; a packaging tool should be explicit, to avoid as much as possible
>> to do something unexpected.
>
> While true that the rpm tools do not magically detect which files to
> distribute, there are many that wish it did and have tried various
> approaches to adding it. ?The following except from the RPM book speaks of
> the various, failed approaches. ?Still it speaks of a essential need on the
> part of developers to automate this aspect of packaging, even if the perfect
> solution has not yet been found.
>
> If we're going to be explicit on what to include, I would argue we should
> NOT allow wildcards, since in subsequent uses of sdist by another person, it
> could cause files to be included that otherwise would not be there. ?The
> only way to be sure precisely the same set of files is always included is to
> explicitly list them one by one. ?A big pain.
>
>
> --- cut here --- excerpt from Maximum RPM book
>
> How Do You Create the File List?
>
> Since RPM automates so many aspects of software installation, it's easy to
> fall into the trap of assuming that RPM does everything for you. Not so! One
> task that is still a manual process is creating the file list. While it may
> seem at first glance, that it could be automated somehow, it's actually a
> more difficult problem than it seems.
>
> Since the majority of an application's files are installed by its makefile,
> RPM has no control over that part of the build process, and therefore,
> cannot automatically determine which files should be part of the package.
> Some people have attempted to use a modified version of install that logs
> the name of every file it installs. But not every makefile uses install, or
> if it does, uses it sporadically.
>
> Another approach tried was to obtain a list of every file on the build
> system, immediately before and after a build, and use the differences as the
> file list. While this approach will certainly find every file that the
> application installed, it can also pick up extraneous files, such as system
> logs, files in /tmp, and the like. The only way to begin to make this
> approach workable would be to do nothing else on the build system, which is
> highly inconvenient. This approach also precludes building more than one
> package on the system at any given time.
>
> At present, the best way to create the file list is to read the makefile to
> see what files it installs, verify this against the files installed on the
> build system, and create the list.
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From regebro at gmail.com  Mon Apr  6 12:59:07 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 12:59:07 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060210g3886d42bjbd0c88d1992307d7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
	<94bdd2610904060210g3886d42bjbd0c88d1992307d7@mail.gmail.com>
Message-ID: <319e029f0904060359v19921411oade0bb80c63dad4@mail.gmail.com>

On Mon, Apr 6, 2009 at 11:10, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Mon, Apr 6, 2009 at 11:03 AM, Lennart Regebro <regebro at gmail.com> wrote:
>> On Mon, Apr 6, 2009 at 10:14, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> Ok so, in version 1.2 of your package, you use the implicit way (VCS based)
>>> then you introduce a new file, that forces your to be explicit in 1.3
>>
>> Why?
>>
>
> because in 1.2 you don't want to add one file that is present in your VCS.

If you mean 1.3, you make sense. OK, so in 1.3 you want to have a file
in the VCS, that should not be included in the distribution. Yes, that
forces you to be explicit. This is true no matter if you use VCS to
figure out the file list or not. In 1.2, you want all files. In 1.3
you want all but one. That will force you to be explicit. The method
to determine the files makes zero difference.

> I am just describing a scenario using your logic.

No, you aren't, sorry. You obviously make some sort of implicit
assumptions I'm not aware of, or you simply misunderstood me.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr  6 13:05:38 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 13:05:38 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9C624.7030305@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>

2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> Lennart Regebro wrote:
>>
>> Sure, if the plugin can't use a stable API then it should not be
>> included in the stdlib.
>>
>
> I was not talking about the plugin stability, but about the situation
> where the VCS detection is broken. As it has happened with setuptools
> very recently.

Yes, me too. That's why I said API stability, not plugin stability.
Because I didn't mean plugin stability, but the stability of the VCS
API, and hence VCS detection.

Am I really that unclear and hard to understand?

> If I have say the git plugin, and I run sdist in a git repo, but later
> someone run sdist from my tarball: how can I be sure it is the same ?

Well, because the tarball will include the files in the VCS. But sure,
if you in the tarball case add files in the directory that are not in
the VCS, that would be included too. Maybe that's a problem, I don't
know. I can't see the problem myself, but maybe somebody else would
have a case for that.

>> How is this different from any non-VCS situation?
>
> It is totally different: in the simple situation, I say which file to
> include/not to include. So it is reproducible, whatever the situation
> is. I won't have some ignored files because my own import does not have
> the same ignored files as the original package.

Are you suggesting that you always list all files explicitly without
wildcards? Because if not I don't see how your answer answers the
question.

>> Can you explain what is different and when?
>
> No other distribution mechanism that I know of uses magic to detect
> which files to distribute. Neither debian tools, nor rpm tools, nor
> autotools. Not being explicit about files for distribution is a terrible
> idea; a packaging tool should be explicit, to avoid as much as possible
> to do something unexpected.

We are now moving in circles, with every time I answer what the
problem is, you vagely talk about "magic". I take that as you don't
know, but you have made up your mind anyway. I don't think discussing
the issue more is going to get us anywhere.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 13:08:05 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 20:08:05 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>	
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>	
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>	
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>	
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>	
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>	
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
Message-ID: <49D9E295.8000308@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:

> Well, because the tarball will include the files in the VCS. But sure,
> if you in the tarball case add files in the directory that are not in
> the VCS, that would be included too. Maybe that's a problem, I don't
> know. I can't see the problem myself, but maybe somebody else would
> have a case for that.
>   

My use case is very simple, and yet very common. If you have your
sources in a VCS system, say svn:

python setup.py sdist # put everything under svn into the tarball
cd dist && uncompress tarball && python setup.py sdist # the tarball is
not the same

That's a concrete example of what I mean by magic. The distributed files
depends on how where you build it. It means that someone who get this
tarball, modify it, and regenerate it cannot do it correctly.

The whole point of packaging tools is reproducibility. Using the VCS as
is done currently breaks this.

cheers,

David

From regebro at gmail.com  Mon Apr  6 13:31:28 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 13:31:28 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9E295.8000308@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904060431p3db7a67el1e316df41b73e6f7@mail.gmail.com>

2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> python setup.py sdist # put everything under svn into the tarball
> cd dist && uncompress tarball && python setup.py sdist # the tarball is
> not the same

Why not?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Mon Apr  6 15:40:33 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 09:40:33 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
Message-ID: <20090406133808.755F63A406A@sparrow.telecommunity.com>

At 03:11 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>Right,
>that would require something else, maybe a new one, ignored by the
>install command
>and using the glob-style pattern.
>
>This means the glob-style pattern will need to have an exclude mechanism
>if some non-installed data are inside a directory that is added in 
>package_data
>(like the tests directory if it's located in the src directory)

We already have this...  it's called MANIFEST.in.


From tarek.ziade at alterway.fr  Mon Apr  6 15:40:11 2009
From: tarek.ziade at alterway.fr (Tarek Ziade)
Date: Mon, 6 Apr 2009 15:40:11 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406133808.755F63A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
Message-ID: <24d471f10904060640l5724a3fct48a35273c537add4@mail.gmail.com>

2009/4/6 P.J. Eby <pje at telecommunity.com>

> At 03:11 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>
>> Right,
>> that would require something else, maybe a new one, ignored by the
>> install command
>> and using the glob-style pattern.
>>
>> This means the glob-style pattern will need to have an exclude mechanism
>> if some non-installed data are inside a directory that is added in
>> package_data
>> (like the tests directory if it's located in the src directory)
>>
>
> We already have this...  it's called MANIFEST.in.


hum, ok let's deprecate what package_data provides then, (the ability to
define a list
of files without a MANIFEST.in template)....




>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090406/46469112/attachment.htm>

From ziade.tarek at gmail.com  Mon Apr  6 15:40:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 15:40:51 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406133808.755F63A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>

On Mon, Apr 6, 2009 at 3:40 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 03:11 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>>
>> Right,
>> that would require something else, maybe a new one, ignored by the
>> install command
>> and using the glob-style pattern.
>>
>> This means the glob-style pattern will need to have an exclude mechanism
>> if some non-installed data are inside a directory that is added in
>> package_data
>> (like the tests directory if it's located in the src directory)
>
> We already have this... ?it's called MANIFEST.in.
>

hum, ok let's deprecate what package_data provides then, (the ability
to define a list
of files without a MANIFEST.in template)....


>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Mon Apr  6 15:43:53 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 09:43:53 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D99AEA.3090207@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<49D99AEA.3090207@ar.media.kyoto-u.ac.jp>
Message-ID: <20090406134126.6A6D73A406A@sparrow.telecommunity.com>

At 03:02 PM 4/6/2009 +0900, David Cournapeau wrote:
>Ben Finney wrote:
> >
> > I'm not advocating using the VCS to *generate the tarball*. I'm
> > talking about using the VCS to *determine what files to put in the
> > tarball*.
> >
>
>Currently, using the vcs plugin does include everything as far as I
>know. So what do you have in mind ?

You can *exclude* files from the sdist, however, using MANIFEST.in.


From pje at telecommunity.com  Mon Apr  6 15:50:05 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 09:50:05 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com
 >
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
Message-ID: <20090406134738.D516B3A406A@sparrow.telecommunity.com>

At 10:14 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>On Mon, Apr 6, 2009 at 10:05 AM, Lennart Regebro <regebro at gmail.com> wrote:
> > On Mon, Apr 6, 2009 at 09:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> >> So part of your file list is declared implicitely in your (D)VCS and
> >> part explicitely in your setup.
> >
> > I don't see how that would work. Reasonably, the VCS is a fallback of
> > nothing is declared in setup.py.
>
>Ok so, in version 1.2 of your package, you use the implicit way (VCS based)
>then you introduce a new file, that forces your to be explicit in 1.3
>so you use setup.py
>then in 1.4 you're back in an implicit list...  this is not a
>long-term solution indeed.

setuptools merges information from MANIFEST.in with information from 
source control. Essentially, MANIFEST.in lets you add things that are 
not in source control (e.g. if you have generated files) and to 
exclude things that *are* in soruce control.

In other words, with setuptools, MANIFEST.in simply describes any 
exceptions to the defaults, with regard to sdist inclusion.


> > I don't see why you should be forced
> > to explicitly declare every file in the setup.py, a sensible default
> > is not a problem.
>
>With glob-style patterns this is not a burden at all.

Have you ever *used* plain distutils for a significant project?  I 
invented the source control feature in setuptools because I was 
constantly ending up with files missing from my sdists, due to 
forgetting to add some sort of glob pattern to MANIFEST.in.  It's not 
going to be any better by putting those globs into setup.py.


From ziade.tarek at gmail.com  Mon Apr  6 15:53:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 15:53:01 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406134738.D516B3A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<20090406134738.D516B3A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060653n7e908a5ex64c4a12a91176c0e@mail.gmail.com>

2009/4/6 P.J. Eby <pje at telecommunity.com>:
> Have you ever *used* plain distutils for a significant project? ?I invented
> the source control feature in setuptools because I was constantly ending up
> with files missing from my sdists, due to forgetting to add some sort of
> glob pattern to MANIFEST.in. ?It's not going to be any better by putting
> those globs into setup.py.

Well, I used setuptools unfortunately. The day I switched from svn to
mercurial, people started
to complain because my distribution had missing files.

So I eventually switch to use MANIFEST.in, because package_data
features were incomplete.

So, I know exactly on my side why I don't want to be tighted to a VCS plugin.

From pje at telecommunity.com  Mon Apr  6 15:55:47 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 09:55:47 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406092519.GA6527@fridge.pov.lt>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405202443.GA20907@fridge.pov.lt>
	<20090405234914.32A633A4108@sparrow.telecommunity.com>
	<20090406092519.GA6527@fridge.pov.lt>
Message-ID: <20090406135320.B58BC3A406A@sparrow.telecommunity.com>

At 12:25 PM 4/6/2009 +0300, Marius Gedminas wrote:
>On Sun, Apr 05, 2009 at 07:51:41PM -0400, P.J. Eby wrote:
> > At 11:24 PM 4/5/2009 +0300, Marius Gedminas wrote:
> >> On Sun, Apr 05, 2009 at 06:45:45PM +0200, Tarek Ziad? wrote:
> >> > After some discussions with people at Pycon, we think that the
> >> > MANIFEST template should be removed.
> >>
> >> +1 for fixing the mess that setuptools + distutils manage to make out of
> >> the manifest (MANIFEST.in + old MANIFEST lying around + .svn metadata =>
> >> lots of confusion and lack of reproducibility)
> >
> > If you've reported this as a problem or filed a bug for it, I appear to
> > have missed it.  Steps to reproduce, and the expected vs. actual
> > behavior would be helpful.
>
>Sorry about that, I though it was working as designed (having seen
>discussions about it here on this list).
>
>   1. Build a package from an svn checkout -> it works
>   2. Build a package from a tree exported with svn export -> it fails,
>      since the MANIFEST.in is nonexistent/incomplete, but you never
>      notice that while developing
>   3. Change the test fixture to ignore .svn subdirectories when tarring
>      up the working tree and copying it into the virtual machine for
>      building (cannot use svn export here, since the tests need to test
>      your working tree with uncommitted changes).

Can't you use an sdist here instead of a manually-created 
tarball?  Or would that not fix the problem for some reason?


From pje at telecommunity.com  Mon Apr  6 15:59:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 09:59:37 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9E295.8000308@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
Message-ID: <20090406135709.E06463A406A@sparrow.telecommunity.com>

At 08:08 PM 4/6/2009 +0900, David Cournapeau wrote:

>My use case is very simple, and yet very common. If you have your
>sources in a VCS system, say svn:
>
>python setup.py sdist # put everything under svn into the tarball
>cd dist && uncompress tarball && python setup.py sdist # the tarball is
>not the same

Does this actually fail under setuptools?  It should not, as RPM 
building from an sdist would then fail, and there was code 
specifically added to ensure that use case works.

Have you actually tested this, or are you simply theorizing?


>That's a concrete example of what I mean by magic. The distributed files
>depends on how where you build it. It means that someone who get this
>tarball, modify it, and regenerate it cannot do it correctly.
>
>The whole point of packaging tools is reproducibility. Using the VCS as
>is done currently breaks this.
>
>cheers,
>
>David
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From pje at telecommunity.com  Mon Apr  6 16:04:28 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 10:04:28 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com
 >
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
Message-ID: <20090406140200.AC6963A406A@sparrow.telecommunity.com>

At 03:40 PM 4/6/2009 +0200, Tarek Ziad? wrote:
>On Mon, Apr 6, 2009 at 3:40 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 03:11 AM 4/6/2009 +0200, Tarek Ziad? wrote:
> >>
> >> Right,
> >> that would require something else, maybe a new one, ignored by the
> >> install command
> >> and using the glob-style pattern.
> >>
> >> This means the glob-style pattern will need to have an exclude mechanism
> >> if some non-installed data are inside a directory that is added in
> >> package_data
> >> (like the tests directory if it's located in the src directory)
> >
> > We already have this...  it's called MANIFEST.in.
> >
>
>hum, ok let's deprecate what package_data provides then, (the ability
>to define a list
>of files without a MANIFEST.in template)....

MANIFEST.in (under setuptools at least) is merely a way to specify 
*exceptions* to the defaults, for what goes in the sdist.

I don't understand why you're so anxious to deprecate something 
without first understanding what it's for.

If you want to put some sort of sdist_exceptions in setup.py to 
replace the *file*, that's one thing, but the *same functionality* of 
MANIFEST.in is still needed, or else setuptools would've deprecated it.


From pje at telecommunity.com  Mon Apr  6 16:12:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 10:12:41 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060653n7e908a5ex64c4a12a91176c0e@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<20090406134738.D516B3A406A@sparrow.telecommunity.com>
	<94bdd2610904060653n7e908a5ex64c4a12a91176c0e@mail.gmail.com>
Message-ID: <20090406141016.DDFAC3A406A@sparrow.telecommunity.com>

At 03:53 PM 4/6/2009 +0200, Tarek Ziad? wrote:
>2009/4/6 P.J. Eby <pje at telecommunity.com>:
> > Have you ever *used* plain distutils for a significant project?  I invented
> > the source control feature in setuptools because I was constantly ending up
> > with files missing from my sdists, due to forgetting to add some sort of
> > glob pattern to MANIFEST.in.  It's not going to be any better by putting
> > those globs into setup.py.
>
>Well, I used setuptools unfortunately. The day I switched from svn to
>mercurial, people started
>to complain because my distribution had missing files.
>
>So I eventually switch to use MANIFEST.in, because package_data
>features were incomplete.
>
>So, I know exactly on my side why I don't want to be tighted to a VCS plugin.

Are you saying the mercurial plugin doesn't work?  Did you contact the author?


From santagada at gmail.com  Mon Apr  6 16:17:03 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 6 Apr 2009 11:17:03 -0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060431p3db7a67el1e316df41b73e6f7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<319e029f0904060431p3db7a67el1e316df41b73e6f7@mail.gmail.com>
Message-ID: <217FEA28-ADEA-4276-8811-E283BD0881A7@gmail.com>


On Apr 6, 2009, at 8:31 AM, Lennart Regebro wrote:

> 2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
>> python setup.py sdist # put everything under svn into the tarball
>> cd dist && uncompress tarball && python setup.py sdist # the  
>> tarball is
>> not the same
>
> Why not?


It will be missing the .hg,.svn or whatever it used to generate the  
file list.

I think the non magic way would be to have a tool to extract a file  
listing from (d)vcs and using that to generate a file, or maybe the  
api be made so that you have to explicitly put "file_list =  
distutils.generate_files_hg()" somewhere, then it is not magic  
anymore, and changing is just creating a list or a function that  
returns a list.


--
Leonardo Santagada
santagada at gmail.com




From ziade.tarek at gmail.com  Mon Apr  6 16:23:15 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 16:23:15 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406140200.AC6963A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060723o288bfe95p7e35fd7b0431a4ee@mail.gmail.com>

On Mon, Apr 6, 2009 at 4:04 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > We already have this... ?it's called MANIFEST.in.
>> >
>>
>> hum, ok let's deprecate what package_data provides then, (the ability
>> to define a list
>> of files without a MANIFEST.in template)....
>
> MANIFEST.in (under setuptools at least) is merely a way to specify
> *exceptions* to the defaults, for what goes in the sdist.

I doubt this was the first goal of this templating system (dealing
with what setuptools was unable to deal with)

>
> I don't understand why you're so anxious to deprecate something without
> first understanding what it's for.

Because we have too many ways to handle this problem right now. It's a mess.

(and I think this mess is partly because of your project - besides all
the good features it has of course)

>
> If you want to put some sort of sdist_exceptions in setup.py to replace the
> *file*, that's one thing, but the *same functionality* of MANIFEST.in is
> still needed, or else setuptools would've deprecated it.

We need to describe a list of file. that's for sure. But not with :

a VCS Plugin,  + a MANIFEST.in but just for the exceptions + a bit of
package_data etc..

how do you expect people to do it right ?

Cheers
Tarek

From ziade.tarek at gmail.com  Mon Apr  6 16:27:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 16:27:24 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406141016.DDFAC3A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<20090406134738.D516B3A406A@sparrow.telecommunity.com>
	<94bdd2610904060653n7e908a5ex64c4a12a91176c0e@mail.gmail.com>
	<20090406141016.DDFAC3A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060727w24cfae58ldfea1ebafdde6cd8@mail.gmail.com>

On Mon, Apr 6, 2009 at 4:12 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 03:53 PM 4/6/2009 +0200, Tarek Ziad? wrote:
>>
>> 2009/4/6 P.J. Eby <pje at telecommunity.com>:
>> > Have you ever *used* plain distutils for a significant project? ?I
>> > invented
>> > the source control feature in setuptools because I was constantly ending
>> > up
>> > with files missing from my sdists, due to forgetting to add some sort of
>> > glob pattern to MANIFEST.in. ?It's not going to be any better by putting
>> > those globs into setup.py.
>>
>> Well, I used setuptools unfortunately. The day I switched from svn to
>> mercurial, people started
>> to complain because my distribution had missing files.
>>
>> So I eventually switch to use MANIFEST.in, because package_data
>> features were incomplete.
>>
>> So, I know exactly on my side why I don't want to be tighted to a VCS
>> plugin.
>
> Are you saying the mercurial plugin doesn't work? ?Did you contact the
> author?

It was not existing back then unfortunately.

Another example :

Once, we switched to SVN 1.5, and we were
totally dependant of you for many days because setuptools was broken
with the new subversion layout.

(we didn't want to change all our packages back to the Distutils way,
but we should have had because at
the end, it was shorter)

Again, I don't want to depend on a third party (D)VCS. It's a nice
addon, but not for a robust distribution tool imho.

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 16:18:01 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 06 Apr 2009 23:18:01 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406135709.E06463A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<20090406135709.E06463A406A@sparrow.telecommunity.com>
Message-ID: <49DA0F19.2090109@ar.media.kyoto-u.ac.jp>

P.J. Eby wrote:
> At 08:08 PM 4/6/2009 +0900, David Cournapeau wrote:
>
>> My use case is very simple, and yet very common. If you have your
>> sources in a VCS system, say svn:
>>
>> python setup.py sdist # put everything under svn into the tarball
>> cd dist && uncompress tarball && python setup.py sdist # the tarball is
>> not the same
>
> Does this actually fail under setuptools?  It should not, as RPM
> building from an sdist would then fail, and there was code
> specifically added to ensure that use case works.
>
> Have you actually tested this, or are you simply theorizing?

I had some recent failure with paver + numpy.distutils + setuptools
combination as late as last week. I am not saying it is setuptools's
fault (it may well be a problem in numpy), but I can't know for sure
because there are many different ways to generate this list of sources.
The problem is not doing it with VCS as much as there are many ways in
different conditions that only a very few people understand as of today.

As a numpy developer who messes quite a bit with distutils and
numpy.distutils extensions, I hope we can get rid of all this magic as
much as possible. It is ok for *tools* to be magic, as long as they
write their *data* to a commonly understood and specified format. But
for now, setuptools enforce some behavior that other tools can't mimic,
because setuptools' way is implementation defined. And for other parts
of packaging, maybe numpy.distutils is implementation defined, without
the possibility for other tools to mimic this either.

That's why  IMHO distutils should be as straightforward as possible, so
you can build up things on top of it. Some people like setuptools'
behavior w.r.t VCS; but it should be possible for my package and for my
potential distutils extensions to stay compatible with distutils,
whether I am using setuptools or not. Otherwise, when there are N tools
which are all implementation defined for packaging, it becomes
intractable very quickly.

cheers,

David

From regebro at gmail.com  Mon Apr  6 16:42:50 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 16:42:50 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <217FEA28-ADEA-4276-8811-E283BD0881A7@gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<319e029f0904060431p3db7a67el1e316df41b73e6f7@mail.gmail.com>
	<217FEA28-ADEA-4276-8811-E283BD0881A7@gmail.com>
Message-ID: <319e029f0904060742u5d2b44c6y9439e104bd187e95@mail.gmail.com>

On Mon, Apr 6, 2009 at 16:17, Leonardo Santagada <santagada at gmail.com> wrote:
> It will be missing the .hg,.svn or whatever it used to generate the file
> list.

Why would that break anything? As it's not a checkout, it will not use
these files to generate the file list, but instead include all files.
That is not a breakage.

> I think the non magic way

My proposal was 100% utterly free from any sort of magic. Please,
guys, stop claiming that there is magic unless you can explain exactly
what is magical. I repeat my request for this discussion to explain in
concrete terms what the problems are, instead of vague hand waving and
talk about "magic".

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr  6 16:43:41 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 16:43:41 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406140200.AC6963A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
Message-ID: <319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>

On Mon, Apr 6, 2009 at 16:04, P.J. Eby <pje at telecommunity.com> wrote:
> I don't understand why you're so anxious to deprecate something without
> first understanding what it's for.

If nobody understands it, that is in itself reason to replace it with
something people understand.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Mon Apr  6 17:12:14 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 11:12:14 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <217FEA28-ADEA-4276-8811-E283BD0881A7@gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<319e029f0904060431p3db7a67el1e316df41b73e6f7@mail.gmail.com>
	<217FEA28-ADEA-4276-8811-E283BD0881A7@gmail.com>
Message-ID: <20090406150946.1119E3A406A@sparrow.telecommunity.com>

At 11:17 AM 4/6/2009 -0300, Leonardo Santagada wrote:

>On Apr 6, 2009, at 8:31 AM, Lennart Regebro wrote:
>
>>2009/4/6 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
>>>python setup.py sdist # put everything under svn into the tarball
>>>cd dist && uncompress tarball && python setup.py sdist # the
>>>tarball is
>>>not the same
>>
>>Why not?
>
>
>It will be missing the .hg,.svn or whatever it used to generate the
>file list.

But it *will* have a SOURCES.txt, that lists all those files in it.


From pje at telecommunity.com  Mon Apr  6 17:16:55 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 11:16:55 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
Message-ID: <20090406151428.2D5253A406A@sparrow.telecommunity.com>

At 04:43 PM 4/6/2009 +0200, Lennart Regebro wrote:
>On Mon, Apr 6, 2009 at 16:04, P.J. Eby <pje at telecommunity.com> wrote:
> > I don't understand why you're so anxious to deprecate something without
> > first understanding what it's for.
>
>If nobody understands it, that is in itself reason to replace it with
>something people understand.

I mean understanding the use cases and how distutils features are 
used in the field.  Deprecating something that people do understand 
and use, simply because you don't understand or use it, is not 
responsible stewardship for a stdlib package, IMO.


From pje at telecommunity.com  Mon Apr  6 17:19:16 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 11:19:16 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DA0F19.2090109@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<20090406135709.E06463A406A@sparrow.telecommunity.com>
	<49DA0F19.2090109@ar.media.kyoto-u.ac.jp>
Message-ID: <20090406151648.9D2CF3A406A@sparrow.telecommunity.com>

At 11:18 PM 4/6/2009 +0900, David Cournapeau wrote:
>P.J. Eby wrote:
> > At 08:08 PM 4/6/2009 +0900, David Cournapeau wrote:
> >
> >> My use case is very simple, and yet very common. If you have your
> >> sources in a VCS system, say svn:
> >>
> >> python setup.py sdist # put everything under svn into the tarball
> >> cd dist && uncompress tarball && python setup.py sdist # the tarball is
> >> not the same
> >
> > Does this actually fail under setuptools?  It should not, as RPM
> > building from an sdist would then fail, and there was code
> > specifically added to ensure that use case works.
> >
> > Have you actually tested this, or are you simply theorizing?
>
>I had some recent failure with paver + numpy.distutils + setuptools
>combination as late as last week. I am not saying it is setuptools's
>fault (it may well be a problem in numpy), but I can't know for sure
>because there are many different ways to generate this list of sources.
>The problem is not doing it with VCS as much as there are many ways in
>different conditions that only a very few people understand as of today.
>
>As a numpy developer who messes quite a bit with distutils and
>numpy.distutils extensions, I hope we can get rid of all this magic as
>much as possible. It is ok for *tools* to be magic, as long as they
>write their *data* to a commonly understood and specified format. But
>for now, setuptools enforce some behavior that other tools can't mimic,
>because setuptools' way is implementation defined. And for other parts
>of packaging, maybe numpy.distutils is implementation defined, without
>the possibility for other tools to mimic this either.
>
>That's why  IMHO distutils should be as straightforward as possible, so
>you can build up things on top of it. Some people like setuptools'
>behavior w.r.t VCS; but it should be possible for my package and for my
>potential distutils extensions to stay compatible with distutils,
>whether I am using setuptools or not. Otherwise, when there are N tools
>which are all implementation defined for packaging, it becomes
>intractable very quickly.

I agree with you.  But I 100% disagree with Tarek that the way to get 
there is by refactoring the distutils.  The distutils are stable 
(i.e., dead and obsolete) and need to be *replaced*, not refactored.

We need a new source manifest format and to deprecate the distutils 
as a whole, not add an API to it and deprecate the parts that actually work.

Nobody is arguing that distutils (or even setuptools) is the "right" 
way to do things.  But you can't get to the right thing by working 
forward from the wrong...  you've got to work backwards from the right.

That means creating an installation manifest format that is 100% 
explicit, and then having tools (including distutils/setuptools 
commands, and plugins for other build systems) that can generate that 
manifest, as well as tools to install files according to different 
platforms' predilections.


From pje at telecommunity.com  Mon Apr  6 17:23:54 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 11:23:54 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060723o288bfe95p7e35fd7b0431a4ee@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<94bdd2610904060723o288bfe95p7e35fd7b0431a4ee@mail.gmail.com>
Message-ID: <20090406152126.C454E3A406A@sparrow.telecommunity.com>

At 04:23 PM 4/6/2009 +0200, Tarek Ziad? wrote:
>Because we have too many ways to handle this problem right now.

Then why add another one?  Because AFAICT you won't be able to 
support the full range of existing distutils use cases without 
something approximating MANIFEST.in.  And what is the motivation for 
developers using MANIFEST.in to move *off* of it?


From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 17:09:47 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 00:09:47 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406151648.9D2CF3A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<49D9B329.6040504@ar.media.kyoto-u.ac.jp>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<20090406135709.E06463A406A@sparrow.telecommunity.com>
	<49DA0F19.2090109@ar.media.kyoto-u.ac.jp>
	<20090406151648.9D2CF3A406A@sparrow.telecommunity.com>
Message-ID: <49DA1B3B.5020105@ar.media.kyoto-u.ac.jp>

P.J. Eby wrote:
> That means creating an installation manifest format that is 100%
> explicit, and then having tools (including distutils/setuptools
> commands, and plugins for other build systems) that can generate that
> manifest, as well as tools to install files according to different
> platforms' predilections.

I am 100 % in agreement with the above (I have been thinking a bit about
something a bit more powerful that MANIFEST and PKG-INFO for the
static-metadata, as a path for a 'next'-gen distutils, but I don't have
anything working yet),

cheers,

David


From ziade.tarek at gmail.com  Mon Apr  6 17:32:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 17:32:29 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406151428.2D5253A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
	<20090406151428.2D5253A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060832n6ddd6246ib29a5d0867e0ed1a@mail.gmail.com>

2009/4/6 P.J. Eby <pje at telecommunity.com>:
> At 04:43 PM 4/6/2009 +0200, Lennart Regebro wrote:
>>
>> On Mon, Apr 6, 2009 at 16:04, P.J. Eby <pje at telecommunity.com> wrote:
>> > I don't understand why you're so anxious to deprecate something without
>> > first understanding what it's for.
>>
>> If nobody understands it, that is in itself reason to replace it with
>> something people understand.
>
> I mean understanding the use cases and how distutils features are used in
> the field. ?Deprecating something that people do understand and use, simply
> because you don't understand or use it, is not responsible stewardship for a
> stdlib package, IMO.

This is your point of view. But not the point of view of other people.

And quit saying this is a unresponsible stewardship.

IMHO the way you are driving setuptools since a few month now is *way* worse
than anything done in this field.

Unlike you, I am here to discuss things and try to find a consensus among
people. And we have made more progress in a few months than you did I think.
(people are working together and are not facing a bottleneck anymore)

I didn't understand in the first place why people were moving away from
distutils-SIG, now I get it .

So please make you points but don't attack people that are trying to move things
forward this is not a correct behavior in the Python community.


>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Mon Apr  6 17:40:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 17:40:02 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406151648.9D2CF3A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>
	<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>
	<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>
	<49D9C624.7030305@ar.media.kyoto-u.ac.jp>
	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<20090406135709.E06463A406A@sparrow.telecommunity.com>
	<49DA0F19.2090109@ar.media.kyoto-u.ac.jp>
	<20090406151648.9D2CF3A406A@sparrow.telecommunity.com>
Message-ID: <94bdd2610904060840s3f593ab8j4bc2008890c50172@mail.gmail.com>

On Mon, Apr 6, 2009 at 5:19 PM, P.J. Eby <pje at telecommunity.com> wrote:
>
> I agree with you. ?But I 100% disagree with Tarek that the way to get there
> is by refactoring the distutils. ?The distutils are stable (i.e., dead and
> obsolete) and need to be *replaced*, not refactored.

No it's not. As discussed in the Summit., it'll get lighter, and we will build
a backport for 2.5 and 2.6. (check the summit mails and Guido's answers)

>
> We need a new source manifest format and to deprecate the distutils as a
> whole, not add an API to it and deprecate the parts that actually work.
>
> Nobody is arguing that distutils (or even setuptools) is the "right" way to
> do things. ?But you can't get to the right thing by working forward from the
> wrong... ?you've got to work backwards from the right.
>
> That means creating an installation manifest format that is 100% explicit,
> and then having tools (including distutils/setuptools commands, and plugins
> for other build systems) that can generate that manifest, as well as tools
> to install files according to different platforms' predilections.

We have been working on all those topics, sprinting at Pycon too,

Please, help us at where we at:  pick a workgroup here
http://wiki.python.org/moin/Distutils (Pycon tasks)

I will not let this happen : a big step backward because you didn't
take part to the summit

From regebro at gmail.com  Mon Apr  6 17:42:40 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 17:42:40 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406151428.2D5253A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
	<20090406151428.2D5253A406A@sparrow.telecommunity.com>
Message-ID: <319e029f0904060842y5e8b2f8dud418024c5ab8f7@mail.gmail.com>

2009/4/6 P.J. Eby <pje at telecommunity.com>:
> I mean understanding the use cases and how distutils features are used in
> the field.

This is of course true. But quite likely, the only one who does that
is you. So it's up to you to document it so others can make solutions
that are easy to understand. :)

> I agree with you.  But I 100% disagree with Tarek that the way to get there
> is by refactoring the distutils.  The distutils are stable (i.e., dead and
> obsolete) and need to be *replaced*, not refactored.

One of the language summit conclusions was that many of the features
if pkg_resources should make it into the stdlib, and that since
distutils was stable, it was probably best to make that a new library.
The same problem is true for any new feature in distutils, so quite
possibly you are right. We need a "refactored" distutils with new
features and with some of pkg_resources in it. And the easiest way to
make that so it also works for Python 2.4 etc is to make it a
completely new module.

This also fixes the current problem with setuptools: That you are one
of the few people who actually understand it, which makes you a
bottleneck for development. The new module could very well start as a
copy/fork of distutils. In fact, it probably should, so that nobody
needs to start from scratch. But calling it something else means
distribution of it gets easy, and we don't have to worry about
backwards compatibility.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Mon Apr  6 17:46:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Apr 2009 17:46:18 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060842y5e8b2f8dud418024c5ab8f7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
	<20090406151428.2D5253A406A@sparrow.telecommunity.com>
	<319e029f0904060842y5e8b2f8dud418024c5ab8f7@mail.gmail.com>
Message-ID: <94bdd2610904060846o6a88a256w91c01828ad1e37b7@mail.gmail.com>

2009/4/6 Lennart Regebro <regebro at gmail.com>:
> 2009/4/6 P.J. Eby <pje at telecommunity.com>:
>> I mean understanding the use cases and how distutils features are used in
>> the field.
>
> This is of course true. But quite likely, the only one who does that
> is you. So it's up to you to document it so others can make solutions
> that are easy to understand. :)
>
>> I agree with you. ?But I 100% disagree with Tarek that the way to get there
>> is by refactoring the distutils. ?The distutils are stable (i.e., dead and
>> obsolete) and need to be *replaced*, not refactored.
>
> One of the language summit conclusions was that many of the features
> if pkg_resources should make it into the stdlib,

Some people are currently looking at that : extracting the good bits
of pkg_resources.

>  and that since
> distutils was stable, it was probably best to make that a new library.

That was not exactly the conclusion of the summit. The idea is to make
distutils
a light, reference library, on wich third party libraries could implements
OS-dependant features and so on. So part of the plan is to remove bdist_*, etc..

From regebro at gmail.com  Mon Apr  6 17:52:36 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 17:52:36 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060846o6a88a256w91c01828ad1e37b7@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
	<20090406151428.2D5253A406A@sparrow.telecommunity.com>
	<319e029f0904060842y5e8b2f8dud418024c5ab8f7@mail.gmail.com>
	<94bdd2610904060846o6a88a256w91c01828ad1e37b7@mail.gmail.com>
Message-ID: <319e029f0904060852r70d89dbdp77cd192437f083c7@mail.gmail.com>

On Mon, Apr 6, 2009 at 17:46, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> That was not exactly the conclusion of the summit. The idea is to make
> distutils
> a light, reference library, on wich third party libraries could implements
> OS-dependant features and so on. So part of the plan is to remove bdist_*, etc..

Well, I wasn't involved in the final discussions, as I had to leave,
but one of the problems mentioned with this is that distribution
becomes a problem, as 2.4 already has a module called distutils, so
there is no obvious way to know that you need to install a new version
of it. Unless there is some magic hack for this which is guaranteed to
always work (ha!) I would side with those who said it would be better
if this distutils version is called something else than distutils.

But if you can promise me that a package that needs a newer version of
distutils will at a minimum exit with an error saying exactly how to
upgrade it I will remove this objection.  :)

Also, renaming means you don't have to be backwards compatible. In
particular, we don't have to make sure setuptools still works.

So I would say both you and Philip are correct. Distutils should be
refactored. And replaced. :)

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Mon Apr  6 18:57:12 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 12:57:12 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060832n6ddd6246ib29a5d0867e0ed1a@mail.gmail.co
 m>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
	<20090406004710.3D2123A4063@sparrow.telecommunity.com>
	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>
	<20090406133808.755F63A406A@sparrow.telecommunity.com>
	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>
	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<319e029f0904060743t1a74bbeatacf16903e0772676@mail.gmail.com>
	<20090406151428.2D5253A406A@sparrow.telecommunity.com>
	<94bdd2610904060832n6ddd6246ib29a5d0867e0ed1a@mail.gmail.com>
Message-ID: <20090406165445.AC43D3A406A@sparrow.telecommunity.com>

At 05:32 PM 4/6/2009 +0200, Tarek Ziad? wrote:
>2009/4/6 P.J. Eby <pje at telecommunity.com>:
> > At 04:43 PM 4/6/2009 +0200, Lennart Regebro wrote:
> >>
> >> On Mon, Apr 6, 2009 at 16:04, P.J. Eby <pje at telecommunity.com> wrote:
> >> > I don't understand why you're so anxious to deprecate something without
> >> > first understanding what it's for.
> >>
> >> If nobody understands it, that is in itself reason to replace it with
> >> something people understand.
> >
> > I mean understanding the use cases and how distutils features are used in
> > the field.  Deprecating something that people do understand and use, simply
> > because you don't understand or use it, is not responsible 
> stewardship for a
> > stdlib package, IMO.
>
>This is your point of view. But not the point of view of other people.
>
>And quit saying this is a unresponsible stewardship.
>
>IMHO the way you are driving setuptools since a few month now is *way* worse
>than anything done in this field.
>
>Unlike you, I am here to discuss things and try to find a consensus among
>people. And we have made more progress in a few months than you did I think.
>(people are working together and are not facing a bottleneck anymore)
>
>I didn't understand in the first place why people were moving away from
>distutils-SIG, now I get it .
>
>So please make you points but don't attack people that are trying to 
>move things
>forward this is not a correct behavior in the Python community.

I'm not attacking you for trying to move the distutils forward, I'm 
attacking the idea of deprecating MANIFEST.in -- or any other feature 
that people are using -- on the basis that another group of people 
doesn't like it or see how it's used.

(I'm against that in general, pretty much regardless of the 
feature...  and I'm aware that this is not an opinion shared by all 
of Python-Dev.)

My concern is simply that the route of refactoring distutils is going 
to create enough breakage *without* deprecating any features.  And 
deprecating features because you don't understand the use cases, 
means that in all probability, the new design will not adequately 
support those use cases.


From zooko at zooko.com  Mon Apr  6 19:11:47 2009
From: zooko at zooko.com (zooko)
Date: Mon, 6 Apr 2009 11:11:47 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406004917.A5AC83A4063@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
Message-ID: <2808AE04-031E-40F1-9D41-FCBF834F8F92@zooko.com>

> Why not?  Aren't there setuptools plugins available for the common  
> DVCSes?


http://pypi.python.org/pypi/setuptools_hg
http://pypi.python.org/pypi/setuptools_darcs
http://pypi.python.org/pypi/setuptools_bzr
http://pypi.python.org/pypi/setuptools-git
http://pypi.python.org/pypi/setuptools_mtn

Regards,

Zooko

From bob at redivi.com  Mon Apr  6 19:28:28 2009
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 6 Apr 2009 10:28:28 -0700
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <2808AE04-031E-40F1-9D41-FCBF834F8F92@zooko.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<2808AE04-031E-40F1-9D41-FCBF834F8F92@zooko.com>
Message-ID: <6a36e7290904061028l69fff99v8b394c96b99fbd8c@mail.gmail.com>

On Mon, Apr 6, 2009 at 10:11 AM, zooko <zooko at zooko.com> wrote:
>> Why not? ?Aren't there setuptools plugins available for the common DVCSes?
>
>
> http://pypi.python.org/pypi/setuptools_hg
> http://pypi.python.org/pypi/setuptools_darcs
> http://pypi.python.org/pypi/setuptools_bzr
> http://pypi.python.org/pypi/setuptools-git
> http://pypi.python.org/pypi/setuptools_mtn

One particular annoyance is that the svn integration built-in to
setuptools tends to break every time a major release of Subversion
comes out (e.g. setuptools broke when 1.5 came out, is probably
currently broken with 1.6, etc.). The "stable" version of setuptools
was broken wrt. svn 1.5 for so long that someone made a temporary fork
of setuptools and provided a svn 1.5 compatible build. It was annoying
to say the least, because it prevented us from upgrading to 1.5 for a
while.

Given what setuptools is trying to accomplish I think its release
cycle is just too slow... although I am grateful it exists because it
saves me of having to remember a substantial quantity of distutils
minutiae and it does in general make it easier for me to install
(most) things.

-bob

From zooko at zooko.com  Mon Apr  6 19:28:14 2009
From: zooko at zooko.com (zooko)
Date: Mon, 6 Apr 2009 11:28:14 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <6a36e7290904061028l69fff99v8b394c96b99fbd8c@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<2808AE04-031E-40F1-9D41-FCBF834F8F92@zooko.com>
	<6a36e7290904061028l69fff99v8b394c96b99fbd8c@mail.gmail.com>
Message-ID: <DF15F130-258E-400E-B68F-7EF000642151@zooko.com>

> One particular annoyance is that the svn integration built-in to  
> setuptools tends to break every time a major release of Subversion  
> comes out (e.g. setuptools broke when 1.5 came out, is probably  
> currently broken with 1.6, etc.). The "stable" version of setuptools
> was broken wrt. svn 1.5 for so long that someone made a temporary  
> fork of setuptools and provided a svn 1.5 compatible build. It was  
> annoying to say the least, because it prevented us from upgrading  
> to 1.5 for a while.

I'm the maintainer of the setuptools_darcs plugin, and it broke when  
darcs v2.0 came out, but since it is a little separately-packaged  
plugin it was easy for me to fix it and distribute a fixed plugin.

Regards,

Zooko

From regebro at gmail.com  Mon Apr  6 19:39:27 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 6 Apr 2009 19:39:27 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <6a36e7290904061028l69fff99v8b394c96b99fbd8c@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<2808AE04-031E-40F1-9D41-FCBF834F8F92@zooko.com>
	<6a36e7290904061028l69fff99v8b394c96b99fbd8c@mail.gmail.com>
Message-ID: <319e029f0904061039o700eac44p5ab89785b287b4b8@mail.gmail.com>

On Mon, Apr 6, 2009 at 19:28, Bob Ippolito <bob at redivi.com> wrote:
> One particular annoyance is that the svn integration built-in to
> setuptools tends to break every time a major release of Subversion
> comes out

This is absolutely true, and annoying. Setuptools read the .svn files
directly, as I understand it. Those file formats change often, and
therefore that way of solving it is inherently unstable. But if this
isn't fixable, that just means that there shouldn't be an svn plugin,
at least not in the stdlib, as that requires "dead stable" API's.

This is however not an argument against asking the VCS for a list of
files. It's just an argument against doing it by parsing subversions
internal files.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From tseaver at palladion.com  Mon Apr  6 20:43:28 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 14:43:28 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
Message-ID: <grdigg$nn9$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Mon, Apr 6, 2009 at 2:51 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 01:49 AM 4/6/2009 +0200, Tarek Ziad? wrote:
>>> So basically, if you get a source distribution out there and work on
>>> it in your own DVCS,
>>> you are unable to create a distro.
>> Why not?  Aren't there setuptools plugins available for the common DVCSes?
> 
> Yes, there are more and more available, but I don't want to rely on a
> VCS or a DVCS to build a source distribution,
> or to wait for the next plugin if I upgrade my (D)VCS or change it.
> 
> It doesn't make sense to  have the list of the files in .svn or .hg
> files. for your package.
> Plus, some svn viewers will not show you the .svn, so you ware unable
> to now what gets into the dist.
> 
> It should be agnostic imho and explicitely defined in the package.

- -1.  Maintaining a separate list of files to be included is a DRY
violation, given that the VCS already *knows* what files need to be there.

+1 to improving / documenting / pimping the various VCS plugins, even to
the extent that we land them into the setuptools distribution when they
are deemed "mature".	


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2k1Q+gerLs4ltQ4RAuLbAJ4+a5uY5fwzOzVhXdGvlYaEPudAtwCfXFQx
5tA49uDrjAjB+R8QhQ4rbG8=
=xMTv
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Mon Apr  6 20:48:21 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 14:48:21 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87hc129tdx.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au>
Message-ID: <grdipm$nn9$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> Lennart Regebro <regebro at gmail.com> writes:
> 
>> On Mon, Apr 6, 2009 at 08:02, Ben Finney <ben+python at benfinney.id.au> wrote:
>>> I'm not advocating using the VCS to *generate the tarball*. I'm
>>> talking about using the VCS to *determine what files to put in the
>>> tarball*.
>> What's the difference?
> 
> The former is having the sdist process tell the VCS ?generate a
> tarball export of the versioned files? which isn't much use since the
> result still isn't a complete sdist.
> 
> The latter is having the sdist process query the VCS ?tell me the set
> of version files by name? and then using that information in the rest
> of the normal sdist creation process.
> 
> 
> David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:
> 
>> Currently, using the vcs plugin does include everything as far as I
>> know. So what do you have in mind ?
> 
> That, in the presence of a VCS and in the absence of a ?MANIFEST.in?
> file, the ?create an sdist? procedure should query the VCS inventory
> for the information it would otherwise get from the ?MANIFEST.in?.
> Thus making that file unnecessary in many cases that I care about.

In *any* case I know of:  including non-VCS-controleld files in an sdist
is an anti-usecase for me.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2k51+gerLs4ltQ4RAjdQAJ0cURt1pAcGVrZseKTNZ+2OUcVDWwCgzw8q
n7Pxy/CMbRUatp8WQkbpTzk=
=HwNM
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Mon Apr  6 20:50:59 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 14:50:59 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
Message-ID: <grdiuk$nn9$3@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Mon, Apr 6, 2009 at 9:30 AM, Lennart Regebro <regebro at gmail.com> wrote:
>> On Mon, Apr 6, 2009 at 09:23, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>> I don't understand a large part of this discussion, but it seems to me
>>>> that the expected and reasonable way of determening what files to
>>>> include would be to:
>>>>
>>>> 1. Specify files in setup.py
>>>> 2. Use the files that is a part of the VCS.
>>> What if you don't want to include some files that are present in the VCS ?
>> Well, then you need to specify stuff in setup.py.
> 
> So part of your file list is declared implicitely in your (D)VCS and
> part explicitely in your setup.
> This is getting complicated.

Or we add support allow specifying the exclusions directly.

>>> This is an implicit behavior.
>> It's a practical default.
> 
> For sure. But that means there will be still more than one way to do it
> 
> Unless the resulting MANIFEST file is always stored in the (D)VCS
> besides setup.py maybe..

- -1 to any file which looks like the current MANIFEST{,.in} at all,
anywhere, in the "one way" model (as opposed to BBB).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2k8T+gerLs4ltQ4RAnJ0AKDCbdc66fA4npvYhs1iQ9+h74tzXwCdFU1Z
/GH9J+mXkyFAw9cyZ7CiAv8=
=OAxe
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Mon Apr  6 21:00:52 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 15:00:52 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49D9E295.8000308@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>		<87ljqe9v6n.fsf@benfinney.id.au>		<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>		<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>		<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>		<49D9B329.6040504@ar.media.kyoto-u.ac.jp>		<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>		<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>		<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>		<49D9C624.7030305@ar.media.kyoto-u.ac.jp>	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>
	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
Message-ID: <grdjh5$s67$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> Lennart Regebro wrote:
> 
>> Well, because the tarball will include the files in the VCS. But sure,
>> if you in the tarball case add files in the directory that are not in
>> the VCS, that would be included too. Maybe that's a problem, I don't
>> know. I can't see the problem myself, but maybe somebody else would
>> have a case for that.
>>   
> 
> My use case is very simple, and yet very common. If you have your
> sources in a VCS system, say svn:
> 
> python setup.py sdist # put everything under svn into the tarball
> cd dist && uncompress tarball && python setup.py sdist # the tarball is
> not the same

That is an "iced tea spoon"[1]:  there is no guarantee that running
sdist will (or even should) work the same way.  Note that under
setuptools, the actual files included in the original 'sdist' are
generated into the 'sources.txt' file in the EGG-INFO directory,

It is often true for *lots* of release management strategies that the
release managers have to do extra work to create a release tarball
(e.g., re-run autogen, etc., or flex / bison, etc.), and that the
released tarball does not include enough information to rebuild itself
(as opposed to re-tgz'ing it).

> That's a concrete example of what I mean by magic. The distributed files
> depends on how where you build it. It means that someone who get this
> tarball, modify it, and regenerate it cannot do it correctly.
> 
> The whole point of packaging tools is reproducibility. Using the VCS as
> is done currently breaks this.

Nobody but a package maintainer should be making sdists.  If you are
making private sdists of a package maintained elsewhere, they you should
be prepared to do extra work, like checking the sources into a VCS, or
hacking the packaging data yourself.


[1] "Doctor!  Doctor!  whenever I drink iced tea, I get a cold stabbing
    pain in my eye!"  "Take out the spoon!"


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2lFk+gerLs4ltQ4RAt3pAJ96zikIssJkAtifbZUNf40ZFVvryQCfVl9d
0+1vwn8q91zzTBRyPeBcViY=
=ydao
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Mon Apr  6 21:06:38 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 15:06:38 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<20090405234722.A356C3A4063@sparrow.telecommunity.com>
	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>
Message-ID: <grdjrv$t8l$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> 2009/4/6 P.J. Eby <pje at telecommunity.com>:
>> At 06:45 PM 4/5/2009 +0200, Tarek Ziad? wrote:
>>> Hello
>>>
>>> After some discussions with people at Pycon, we think that the
>>> MANIFEST template should be removed.
>>>
>>> I would like to deprecate MANIFEST.in in Distutils, in favor of a
>>> package_data that can match directories with a glob-style pattern
>>> see : http://bugs.python.org/issue5302
>> What about documentation files, data files, examples, and tests -- all of
>> which might need to be included in an sdist, but would not be "package
>> data"?
> 
> that would be data_files, but I am wondering if we should have an explicit
> one for documentation files.

At the PyCon sprints, Matthias Klose and I worked up a recommendation
for data files which are not supposed to be included with the software.
 The substance was to add commands for the different groups, and then
add support for enumerating those files in setup.cfg:

 --install-docs

 --install-locales

 --install-tests

The '--install-data' command could call these, I suppose, and should
likely avoid picking up any file which they claimed.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2lK++gerLs4ltQ4RAtOgAJ4n+O3IoUiNhOtmu4WI1ksHPPqqqgCfWw46
QcmbvuwcY9O/tUwqszHRq2Q=
=Jmkt
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Mon Apr  6 21:09:08 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Apr 2009 15:09:08 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <94bdd2610904060723o288bfe95p7e35fd7b0431a4ee@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<20090405234722.A356C3A4063@sparrow.telecommunity.com>	<94bdd2610904051653m630b06aepc0df56868cf90072@mail.gmail.com>	<20090406004710.3D2123A4063@sparrow.telecommunity.com>	<94bdd2610904051811n23292ff7m46071f529d80928f@mail.gmail.com>	<20090406133808.755F63A406A@sparrow.telecommunity.com>	<94bdd2610904060640r4486a94j1daf47606421a997@mail.gmail.com>	<20090406140200.AC6963A406A@sparrow.telecommunity.com>
	<94bdd2610904060723o288bfe95p7e35fd7b0431a4ee@mail.gmail.com>
Message-ID: <grdk0k$t8l$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Mon, Apr 6, 2009 at 4:04 PM, P.J. Eby <pje at telecommunity.com> wrote:
>>>> We already have this...  it's called MANIFEST.in.
>>>>
>>> hum, ok let's deprecate what package_data provides then, (the ability
>>> to define a list
>>> of files without a MANIFEST.in template)....
>> MANIFEST.in (under setuptools at least) is merely a way to specify
>> *exceptions* to the defaults, for what goes in the sdist.
> 
> I doubt this was the first goal of this templating system (dealing
> with what setuptools was unable to deal with)
> 
>> I don't understand why you're so anxious to deprecate something without
>> first understanding what it's for.
> 
> Because we have too many ways to handle this problem right now. It's a mess.
> 
> (and I think this mess is partly because of your project - besides all
> the good features it has of course)
> 
>> If you want to put some sort of sdist_exceptions in setup.py to replace the
>> *file*, that's one thing, but the *same functionality* of MANIFEST.in is
>> still needed, or else setuptools would've deprecated it.
> 
> We need to describe a list of file. that's for sure. But not with :
> 
> a VCS Plugin,  + a MANIFEST.in but just for the exceptions + a bit of
> package_data etc..

- -1 to anything which breaks that pattern, which requires less effort and
braaks less often than any "explicit" manifest.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ2lNU+gerLs4ltQ4RAuw0AKDQTPVKfR9XP82UU2gvhuyBYB/FHwCcC1PX
zdCne3q39TS2ufdUC3lhBMk=
=+p3C
-----END PGP SIGNATURE-----


From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 21:01:59 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 04:01:59 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <grdipm$nn9$2@ger.gmane.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<87hc129tdx.fsf@benfinney.id.au>
	<grdipm$nn9$2@ger.gmane.org>
Message-ID: <49DA51A7.2040404@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> In *any* case I know of:  including non-VCS-controleld files in an sdist
> is an anti-usecase for me.

A simple example: including sphinx-generated documentation. If including
non-VCS controled stuff is an anti-case, then why having sdist at all ?
Any half decent VCS can generate a tarball directly from the files it
controls.

cheers,

David

From david at ar.media.kyoto-u.ac.jp  Mon Apr  6 21:14:26 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 04:14:26 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <grdjh5$s67$1@ger.gmane.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>		<87ljqe9v6n.fsf@benfinney.id.au>		<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>		<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>		<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>		<49D9B329.6040504@ar.media.kyoto-u.ac.jp>		<319e029f0904060106m4f9cabb6u3488459a14f6ca6@mail.gmail.com>		<49D9B85B.8090805@ar.media.kyoto-u.ac.jp>		<319e029f0904060206u6a5e74e0od009485657babf6c@mail.gmail.com>		<49D9C624.7030305@ar.media.kyoto-u.ac.jp>	<319e029f0904060405p1ce506a4y2aaa0d4a466f5cd0@mail.gmail.com>	<49D9E295.8000308@ar.media.kyoto-u.ac.jp>
	<grdjh5$s67$1@ger.gmane.org>
Message-ID: <49DA5492.6050802@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:

>
> It is often true for *lots* of release management strategies that the
> release managers have to do extra work to create a release tarball
> (e.g., re-run autogen, etc., or flex / bison, etc.), and that the
> released tarball does not include enough information to rebuild itself
> (as opposed to re-tgz'ing it).

I have yet encountered a package from autotools where make dist did not
work from the pristine sources. Something like autogen.sh is only needed
if you modify the autotools "scripts" (configure.ac, Makefile.in/am,
etc...).

> Nobody but a package maintainer should be making sdists.

But my use cases are from a maintainer POV ! In particular, I like
checking that sdist actually works in my release scripts (something like
make distcheck) - which is exactly my rationale for the above example.

cheers,

David



From zooko at zooko.com  Mon Apr  6 23:00:24 2009
From: zooko at zooko.com (zooko)
Date: Mon, 6 Apr 2009 15:00:24 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<94bdd2610904060023x20d18a63h44316aa849497694@mail.gmail.com>
	<319e029f0904060030n3b37fb5do266b037d0ab602c1@mail.gmail.com>
	<94bdd2610904060051mbccbfe7g23ab9e9789f2f7dd@mail.gmail.com>
	<319e029f0904060105u61d8767ama204203b63e4cd12@mail.gmail.com>
	<94bdd2610904060114wfcaf11aidda48e006c1ac721@mail.gmail.com>
	<319e029f0904060203k2ce5a522q113c1b6741af9210@mail.gmail.com>
Message-ID: <9F2740DB-9D38-4B3F-ADD7-D33BC6441E50@zooko.com>

On Apr 6, 2009, at 3:03 AM, Lennart Regebro wrote:

> Could we in this discussion, instead of making fuzzy general  
> statements explain exactly what the problems are?

So far, the setuptools behavior has worked fine for me on both  
numerous small projects (e.g. pyutil [1], zfec [2], pycryptopp [3],  
dupfilefind [4], zbase32 [5]) and on a couple of larger projects  
(allmydata-tahoe [6] and Nevow [7]).  I've used the builtin svn  
integration and a plugin for darcs that I wrote myself  
(setuptools_darcs [8]).  The setuptools options of  
"include_package_data", "exclude_package_data", and "package_data"  
provide control over the behavior.

Regards,

Zooko

[1] http://pypi.python.org/pypi/pyutil
[1] http://pypi.python.org/pypi/zfec
[1] http://pypi.python.org/pypi/pycryptopp
[1] http://pypi.python.org/pypi/dupfilefind
[1] http://pypi.python.org/pypi/zbase32
[1] http://pypi.python.org/pypi/pyutil
[1] http://pypi.python.org/pypi/allmydata-tahoe
[1] http://pypi.python.org/pypi/Nevow
[1] http://pypi.python.org/pypi/setuptools_darcs

From nicholas at metaweb.com  Tue Apr  7 02:04:55 2009
From: nicholas at metaweb.com (Nicholas Veeser)
Date: Mon, 06 Apr 2009 17:04:55 -0700
Subject: [Distutils] pkg_resource require does not seem to find my egg
Message-ID: <49DA98A7.8090003@metaweb.com>

I am working on a tool we call dino which uses sqlalchemy 0.5.3
Its an update of a previous version (called dino) which uses sqlalchemy 
0.4.4.

For reasons I don't have to go into, I would like to have both tools 
installed on the same host,
with almost no changes to the existing tool.   Thus both versions of 
sqlalchemy installed

So my solution seemed to be use pkg_resources and egg's:

- leave sqlalchemy 0.4.4 in:
    /usr/lib64/python2.4/site-packages/sqlalchemy
- build sqlalchemy 0.5.3 as an Egg and install into:
    /usr/lib64/python2.4/site-packages/SQLAlchemy-0.5.3-py2.4.egg
- in the root package of the new code,  specify the correct version
    from pkg_resources import require; require("SQLAlchemy>=0.5.0")


This does not work.
It does not seem to find the egg, and only finds the old version, and 
then (correctly) fails.

Trying to read through the pkg_resource.py, I am trying to make sense 
why this is, and
how to get around it.  I am having trouble wrapping my head around some 
of the terms...


Am I doing something obviously wrong?
Is there a known bug?
I have installed setuptools-0.6_rc7

Here is the traceback
Traceback (most recent call last):
  File "/usr/bin/dinoadm", line 8, in ?
    import dino.cmd.cli
  File "/usr/lib64/python2.4/site-packages/dino/__init__.py", line 6, in ?
    require("SQLAlchemy>=0.5.0")
  File "/usr/lib64/python2.4/site-packages/pkg_resources.py", line 626, 
in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib64/python2.4/site-packages/pkg_resources.py", line 528, 
in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (SQLAlchemy 0.4.4 
(/usr/lib64/python2.4/site-packages), 
Requirement.parse('SQLAlchemy>=0.5.0'))



Thanx
Nicholas



From ben+python at benfinney.id.au  Tue Apr  7 02:29:39 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 07 Apr 2009 10:29:39 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
Message-ID: <87skkl8fws.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Tres Seaver wrote:
> > In *any* case I know of: including non-VCS-controleld files in an
> > sdist is an anti-usecase for me.
> 
> A simple example: including sphinx-generated documentation.

That's a good counter-point to Tres's point, true.

> If including non-VCS controled stuff is an anti-case, then why
> having sdist at all ? Any half decent VCS can generate a tarball
> directly from the files it controls.

This continues to be irrelevant. I don't know why you keep repeating
it.

An sdist is *not* just a tarball of the source files. The important
difference is that it contains metadata generated and assembled by
distutils (or an equivalent) that is specific to Python's distutils
system. Hence, the job of creating an sdist is *not* something for the
VCS to do.

-- 
 \     ?Many are stubborn in pursuit of the path they have chosen, few |
  `\                     in pursuit of the goal.? ?Friedrich Nietzsche |
_o__)                                                                  |
Ben Finney


From pje at telecommunity.com  Tue Apr  7 04:49:28 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 06 Apr 2009 22:49:28 -0400
Subject: [Distutils] pkg_resource require does not seem to find my  egg
In-Reply-To: <49DA98A7.8090003@metaweb.com>
References: <49DA98A7.8090003@metaweb.com>
Message-ID: <20090407024702.D79E73A4063@sparrow.telecommunity.com>

At 05:04 PM 4/6/2009 -0700, Nicholas Veeser wrote:
>I am working on a tool we call dino which uses sqlalchemy 0.5.3
>Its an update of a previous version (called dino) which uses sqlalchemy 0.4.4.
>
>For reasons I don't have to go into, I would like to have both tools 
>installed on the same host,
>with almost no changes to the existing tool.   Thus both versions of 
>sqlalchemy installed
>
>So my solution seemed to be use pkg_resources and egg's:
>
>- leave sqlalchemy 0.4.4 in:
>    /usr/lib64/python2.4/site-packages/sqlalchemy
>- build sqlalchemy 0.5.3 as an Egg and install into:
>    /usr/lib64/python2.4/site-packages/SQLAlchemy-0.5.3-py2.4.egg
>- in the root package of the new code,  specify the correct version
>    from pkg_resources import require; require("SQLAlchemy>=0.5.0")

Change this to defining that dependency in its setup.py, and develop 
using 'setup.py develop'.  It will then work correctly.


From david at ar.media.kyoto-u.ac.jp  Tue Apr  7 06:17:29 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 13:17:29 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87skkl8fws.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<87hc129tdx.fsf@benfinney.id.au>
	<grdipm$nn9$2@ger.gmane.org>	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
Message-ID: <49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
>
> An sdist is *not* just a tarball of the source files.

It is if you consider any non-tracked file to be an anti-usecase, which
is what I was answering to :)

Moving away from "using VCS is good" vs "using VCS is not good", maybe
we should focus on the workflows to support. The most significant to me
is reproducibility (python setup.py sdist output should depend as little
as possible on its environment - whether the sources are under VCS or
not, etc....). At least one person said this is not a requirement. Can
we reach a consensus on this point, and then focus on the technical
solutions ?

cheers,

David

From ben+python at benfinney.id.au  Tue Apr  7 06:47:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 07 Apr 2009 14:47:50 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
Message-ID: <877i1x83yh.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Ben Finney wrote:
> >
> > An sdist is *not* just a tarball of the source files.
> 
> It is if you consider any non-tracked file to be an anti-usecase,

No. In *no case* is an sdist equal to a tarball of the source files.
Please stop blurring the distinction.

-- 
 \       ?One of the most important things you learn from the internet |
  `\   is that there is no ?them? out there. It's just an awful lot of |
_o__)                                            ?us?.? ?Douglas Adams |
Ben Finney


From regebro at gmail.com  Tue Apr  7 07:20:11 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 07:20:11 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>

On Tue, Apr 7, 2009 at 06:17, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Ben Finney wrote:
>>
>> An sdist is *not* just a tarball of the source files.
>
> It is if you consider any non-tracked file to be an anti-usecase, which
> is what I was answering to :)

Except for files generated by the sdist/build-process, yes. Why would
you want to include files in a distribution that are neither
controlled source files or a result of the sdist/build process?

> Moving away from "using VCS is good" vs "using VCS is not good", maybe
> we should focus on the workflows to support. The most significant to me
> is reproducibility (python setup.py sdist output should depend as little
> as possible on its environment - whether the sources are under VCS or
> not, etc....).

OK, here is a usecase. I have a directoy with a module, foo.bar. I
have also in that directory saved a project file from my IDE, because
naturally I save a project file into the directory of every project I
have. But since that's not a part of the module itself I never check
it in.

My friend Bobo, however, does not use an IDE, and does not create a
project file.

In case number one, I specify the files to be included with a wildcard.

In case number two, I do not specify the files to be included at all,
but let sdist determine the files by looking at which files are
version controlled.

Now, in which of these cases will the sdist created by me an d by Bobo
be identical? Right. In the one where the file list is determined by
which files are version controlled. Can we then really say that this
case is more dependent on the environment?

Yes, for me, the sdist will be different if I have a checkout or not,
since I have the project file in the directory. But that's a rather
unusual usecase, don't you think? Why would I have a non-VCS directory
of a mdule with a project file in it, that I distribute? If I modify a
project, it should be checked in.

Asking the VCS for which files are checked in i sthe most reliable way
of figuring out which files are actually source files I can think of.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From robert.kern at gmail.com  Tue Apr  7 07:36:01 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 07 Apr 2009 00:36:01 -0500
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<87hc129tdx.fsf@benfinney.id.au>
	<grdipm$nn9$2@ger.gmane.org>	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	<87skkl8fws.fsf@benfinney.id.au>	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
Message-ID: <greoo2$pb0$1@ger.gmane.org>

On 2009-04-07 00:20, Lennart Regebro wrote:
> On Tue, Apr 7, 2009 at 06:17, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp>  wrote:
>> Ben Finney wrote:
>>> An sdist is *not* just a tarball of the source files.
>> It is if you consider any non-tracked file to be an anti-usecase, which
>> is what I was answering to :)
>
> Except for files generated by the sdist/build-process, yes. Why would
> you want to include files in a distribution that are neither
> controlled source files or a result of the sdist/build process?

Unfortunately, the setup.py file is not always capable of performing all of the 
build processes (at least, not easily). One might use a Makefile or other build 
tool to build the docs or other assets. So you need a manual way of telling the 
setup.py file what additional files to include.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From david at ar.media.kyoto-u.ac.jp  Tue Apr  7 07:20:37 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 14:20:37 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87prfq9xf2.fsf@benfinney.id.au>	
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
Message-ID: <49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> Except for files generated by the sdist/build-process, yes. Why would
> you want to include files in a distribution that are neither
> controlled source files or a result of the sdist/build process?
>   

Depends on what you mean by the build process. I sometimes need to
alleviate distutils limitations with some scripts, or because of
incompatible monkey patching between numpy.distutils, paver and setuptools.

> Now, in which of these cases will the sdist created by me an d by Bobo
> be identical? Right. In the one where the file list is determined by
> which files are version controlled. Can we then really say that this
> case is more dependent on the environment?
>   

It depends on how you set your wildcard. If you use *.$ext, seems to me
that it would alleviate those cases, or do you mean something else ?

> Yes, for me, the sdist will be different if I have a checkout or not,
> since I have the project file in the directory. But that's a rather
> unusual usecase, don't you think?

Well, no, I don't think so. You can't make assumptions about what I
need, as I can't make assumption about what you need. But if we can
agree on a "low level communication format", then you can use your tools
to support your usecases, and I can use mine for my usecases, so that we
can end up with something which works everywhere.

>  Why would I have a non-VCS directory
> of a mdule with a project file in it, that I distribute? If I modify a
> project, it should be checked in.
>   

What if the file is generated ? For example, I have some .c files which
are generated from a script from some templates. I want to include the
generated file in the sdist so people don't need the scripts (and
potential dependencies) to build the software. That's something which is
well supported on autotools, for example.

> Asking the VCS for which files are checked in i sthe most reliable way
> of figuring out which files are actually source files I can think of.
>   

I am not denying that it works for you. I am just trying to argue that
it does not work for me, as I have given several examples already
(generated sources, built documentation, sources obtained from a
different VCS than the original source). So instead of trying to impose
each solution to each other, we should focus on the commonalities,

cheers,

David


From regebro at gmail.com  Tue Apr  7 07:43:19 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 07:43:19 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <greoo2$pb0$1@ger.gmane.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<greoo2$pb0$1@ger.gmane.org>
Message-ID: <319e029f0904062243k5d07efbew8f4394f88bdfaac7@mail.gmail.com>

On Tue, Apr 7, 2009 at 07:36, Robert Kern <robert.kern at gmail.com> wrote:
> Unfortunately, the setup.py file is not always capable of performing all of
> the build processes (at least, not easily). One might use a Makefile or
> other build tool to build the docs or other assets. So you need a manual way
> of telling the setup.py file what additional files to include.

OK, this is a good usecase for including non-VCS files.

But then again, nobody ever said that this would *not* be possible. In
this case you are likely to have several intermediate files that you
are not interested in including, and the question is if you want to
include the source files used for this external build tool at all. So
would automatic globbing be better? Wouldn't you need to specify which
files that should be included here?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Tue Apr  7 07:50:46 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 07:50:46 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>

On Tue, Apr 7, 2009 at 07:20, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> It depends on how you set your wildcard. If you use *.$ext, seems to me
> that it would alleviate those cases, or do you mean something else ?

Sure, but now we are getting closer and closer to having to specify
each file separately. Which is not a practical state of affairs. This
is about which process to use to determine which files should be
included when we don't specify things. Take all files, or take the
ones that are handled by a VCS. There is no doubt that taking the
files that are being version controlled are a sensible default.

> Well, no, I don't think so. You can't make assumptions about what I
> need, as I can't make assumption about what you need.

I'm not making any such assumptions.

> I am not denying that it works for you. I am just trying to argue that
> it does not work for me,

I've never claimed it works for you. You are saying that this method
should *not* be supported. Hence you are saying that it somehow does
not work for people in general, which it clearly does, or that it is
useless, which I don't think it is.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Tue Apr  7 07:41:07 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 14:41:07 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>	
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
Message-ID: <49DAE773.4080406@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
> On Tue, Apr 7, 2009 at 07:20, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> It depends on how you set your wildcard. If you use *.$ext, seems to me
>> that it would alleviate those cases, or do you mean something else ?
>>     
>
> Sure, but now we are getting closer and closer to having to specify
> each file separately. Which is not a practical state of affairs. This
> is about which process to use to determine which files should be
> included when we don't specify things. 

I think it is obvious that we can't agree on the process :)

> I've never claimed it works for you. You are saying that this method
> should *not* be supported.
>   

I am not saying that it should not be possible - but that it should be
built on top of a lower system which we can all agree upon. Then, you
can use your tool to depend on the VCS to feed this lower system, and I
can reuse this lower system myself without depending on the VCS thing,
and everybody is happy ?

cheers,

David


From regebro at gmail.com  Tue Apr  7 08:19:39 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 08:19:39 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DAE773.4080406@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>

On Tue, Apr 7, 2009 at 07:41, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> I am not saying that it should not be possible - but that it should be
> built on top of a lower system which we can all agree upon. Then, you
> can use your tool to depend on the VCS to feed this lower system, and I
> can reuse this lower system myself without depending on the VCS thing,
> and everybody is happy ?

"Lower system"? What would that be? This is about if we should support
getting a file list from the VCS or not to determine what files should
go in a sdist. I have a hard time envisioning anything "lower" than
that.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From marius at pov.lt  Tue Apr  7 09:35:56 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 7 Apr 2009 10:35:56 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090406135320.B58BC3A406A@sparrow.telecommunity.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<20090405202443.GA20907@fridge.pov.lt>
	<20090405234914.32A633A4108@sparrow.telecommunity.com>
	<20090406092519.GA6527@fridge.pov.lt>
	<20090406135320.B58BC3A406A@sparrow.telecommunity.com>
Message-ID: <20090407073555.GA11077@fridge.pov.lt>

On Mon, Apr 06, 2009 at 09:55:47AM -0400, P.J. Eby wrote:
> At 12:25 PM 4/6/2009 +0300, Marius Gedminas wrote:
>> On Sun, Apr 05, 2009 at 07:51:41PM -0400, P.J. Eby wrote:
>> > At 11:24 PM 4/5/2009 +0300, Marius Gedminas wrote:
>> >> On Sun, Apr 05, 2009 at 06:45:45PM +0200, Tarek Ziad? wrote:
>> >> > After some discussions with people at Pycon, we think that the
>> >> > MANIFEST template should be removed.
>> >>
>> >> +1 for fixing the mess that setuptools + distutils manage to make out of
>> >> the manifest (MANIFEST.in + old MANIFEST lying around + .svn metadata =>
>> >> lots of confusion and lack of reproducibility)
>> >
>> > If you've reported this as a problem or filed a bug for it, I appear to
>> > have missed it.  Steps to reproduce, and the expected vs. actual
>> > behavior would be helpful.
>>
>> Sorry about that, I though it was working as designed (having seen
>> discussions about it here on this list).
>>
>>   1. Build a package from an svn checkout -> it works
>>   2. Build a package from a tree exported with svn export -> it fails,
>>      since the MANIFEST.in is nonexistent/incomplete, but you never
>>      notice that while developing
>>   3. Change the test fixture to ignore .svn subdirectories when tarring
>>      up the working tree and copying it into the virtual machine for
>>      building (cannot use svn export here, since the tests need to test
>>      your working tree with uncommitted changes).
>
> Can't you use an sdist here instead of a manually-created tarball? Or 
> would that not fix the problem for some reason?

Not really; this is a tarball of a larger system which is not eggified
(and now contains a couple of setuptoolized packages in subdirectories).

The thing that worries me most is that when I listen to arguments on
this list, it all makes sense, and the sun is shining, and the feature
feels good.  But then you get new users unfamiliar with all the
intricacies, and they get confused, and sometimes release broken
packages into the wild.

Marius Gedminas
-- 
If it weren't for the last minute, nothing would ever get done.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090407/35d6fce5/attachment.pgp>

From david at ar.media.kyoto-u.ac.jp  Tue Apr  7 09:40:08 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 16:40:08 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>	
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>	
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
Message-ID: <49DB0358.8040603@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> "Lower system"? What would that be? This is about if we should support
> getting a file list from the VCS or not to determine what files should
> go in a sdist.

We can argue all day long - but since there is a disagreement, there is
only two issues: either someone decides for one behavior or the other,
or we acknowledge our difference and decide on less ambitious goals to
build other tools above.

>  I have a hard time envisioning anything "lower" than
> that.
>   

Why so ? Few packaging system use the VCS as the lower system to know
what goes in a distribution, most of them (regardless of the platform:
windows, unix, mac os x) uses something more low level than this at the
implementation level.

cheers,

David


From regebro at gmail.com  Tue Apr  7 10:09:33 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 10:09:33 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DB0358.8040603@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
	<49DB0358.8040603@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>

On Tue, Apr 7, 2009 at 09:40, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Lennart Regebro wrote:
>>
>> "Lower system"? What would that be? This is about if we should support
>> getting a file list from the VCS or not to determine what files should
>> go in a sdist.
>
> We can argue all day long

Possibly. I can't say we are arguing though. You are just saying that
you don't agree, but you still come with no arguments for your
standpoint or against mine. It's obvious that you have made up your
mind, but you don't know why. You may be correct, but to convince me
you need concrete arguments. Not fuzzy broad statements that you fail
to explain.

> but since there is a disagreement, there is
> only two issues: either someone decides for one behavior or the other,
> or we acknowledge our difference and decide on less ambitious goals to
> build other tools above.

Less ambitious goals? We are discussing if the VCS should be a part of
determining the list of files that goes into a distribution or not.
This is not in any way ambitious. You claim that it under no
circumstance should be a part of it, but you have no arguments for
that standpoint. I claim that it is a good and reasonable way to
determine which files should be included, unless the files are
otherwise specified.

How can we have any "less ambitions" goals than this?

>> ?I have a hard time envisioning anything "lower" than
>> that.
>
> Why so ? Few packaging system use the VCS as the lower system to know
> what goes in a distribution, most of them (regardless of the platform:
> windows, unix, mac os x) uses something more low level than this at the
> implementation level.

I totally fail to see how this is in any way "lower", and repeat my
question: "Lower system"? What would that be? What is a "lower system"
in determining a file list? I suggested a three step process:

1. If there is explicit definitions, use that.
2. If not, then if there is a VCS, use that.
3. If not, include all files except those extensions that are known to
be temporary files.

Where in this is there a "lower level"?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Tue Apr  7 09:59:23 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Apr 2009 16:59:23 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>	
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>	
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>	
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>	
	<49DB0358.8040603@ar.media.kyoto-u.ac.jp>
	<319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
Message-ID: <49DB07DB.3000008@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
>
> Possibly. I can't say we are arguing though. You are just saying that
> you don't agree, but you still come with no arguments for your
> standpoint or against mine. 

Ok.

David

From floris.bruynooghe at gmail.com  Tue Apr  7 11:03:44 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 7 Apr 2009 10:03:44 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DAE773.4080406@ar.media.kyoto-u.ac.jp>
References: <319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
Message-ID: <20090407090344.GA4604@laurie.devork>

On Tue, Apr 07, 2009 at 02:41:07PM +0900, David Cournapeau wrote:
> Lennart Regebro wrote:
> > On Tue, Apr 7, 2009 at 07:20, David Cournapeau
> > <david at ar.media.kyoto-u.ac.jp> wrote:
> >   
> >> It depends on how you set your wildcard. If you use *.$ext, seems to me
> >> that it would alleviate those cases, or do you mean something else ?
> >>     
> >
> > Sure, but now we are getting closer and closer to having to specify
> > each file separately. Which is not a practical state of affairs. This
> > is about which process to use to determine which files should be
> > included when we don't specify things. 
> 
> I think it is obvious that we can't agree on the process :)
> 
> > I've never claimed it works for you. You are saying that this method
> > should *not* be supported.
> >   
> 
> I am not saying that it should not be possible - but that it should be
> built on top of a lower system which we can all agree upon. Then, you
> can use your tool to depend on the VCS to feed this lower system, and I
> can reuse this lower system myself without depending on the VCS thing,
> and everybody is happy ?

+1


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From noah.gift at gmail.com  Tue Apr  7 13:54:46 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Tue, 7 Apr 2009 23:54:46 +1200
Subject: [Distutils] short circuiting module lookups
Message-ID: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>

I work off of a rather large NFS infrastructure where thousands of
machines are constantly doing things, and recently I discovered a few
things about both setuptools and standard Python lookup that are
causing problems.

I use nosetests, and noticed that it can take up to 10 seconds to 60
seconds to execute nosetests, a lot of this has to do with the load on
the file system I am on that is shared by thousands of machines, but I
was a bit troubled to notice the following behavior with an strace:

1.  In the case of entry points for setuptools, it actually recurses
into EVERY egg directory in your path, not just the egg you requested,
adds them to your sys.path and additionally looks for four files
inside of every egg.  On a laptop on local storage, this doesn't
matter, but when thousands of machines hit the same filer, with many
python processes, bad things happen...

2.  Python itself, also looks at quite a few locations in the search
for modules.

It looks like this behavior with eggs and setuptools makes them
virtually unusable in large installations.  Is there any advice for
people that have my situation, in which flexibility, i.e. path lookups
are not that important, what is important is the least possible
lookups to find a module, even to the best case scenario, in which I
can tell a command line tool, or wrapper script to only look at a
specific path, and to never look at any other location?

I haven't done much research yet, but thought I would field the
question before I went down a rat hole...

--
Cheers,

Noah

From floris.bruynooghe at gmail.com  Tue Apr  7 15:02:39 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 7 Apr 2009 14:02:39 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
References: <49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
	<49DB0358.8040603@ar.media.kyoto-u.ac.jp>
	<319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
Message-ID: <20090407130239.GA4867@laurie.devork>

On Tue, Apr 07, 2009 at 10:09:33AM +0200, Lennart Regebro wrote:
> On Tue, Apr 7, 2009 at 09:40, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
> > Lennart Regebro wrote:
> >>
> >> "Lower system"? What would that be? This is about if we should support
> >> getting a file list from the VCS or not to determine what files should
> >> go in a sdist.
> >
> > We can argue all day long
> 
> Possibly. I can't say we are arguing though. You are just saying that
> you don't agree, but you still come with no arguments for your
> standpoint or against mine. It's obvious that you have made up your
> mind, but you don't know why. You may be correct, but to convince me
> you need concrete arguments. Not fuzzy broad statements that you fail
> to explain.

Sorry if I am going to repeat something but I haven't followed the
entire discussion to the last detail.  But since I agree with David
I'll try to add my arguments.

> > but since there is a disagreement, there is
> > only two issues: either someone decides for one behavior or the other,
> > or we acknowledge our difference and decide on less ambitious goals to
> > build other tools above.
> 
> Less ambitious goals? We are discussing if the VCS should be a part of
> determining the list of files that goes into a distribution or not.
> This is not in any way ambitious. You claim that it under no
> circumstance should be a part of it, but you have no arguments for
> that standpoint. I claim that it is a good and reasonable way to
> determine which files should be included, unless the files are
> otherwise specified.
> 
> How can we have any "less ambitions" goals than this?

The VCS is an external piece of software outside of Python and the
distutils.  There also isn't just one VCS, there's many of them and
all of them are on different release schedules which differ from
Python's and/or distutils's release schedules and can thus break API,
or datastore semantics at any time (I think this was demonstrated with
setuptools and svn 1.5 but not sure since I don't use setuptools).
Therefore for distutils to actually depend on a VCS to function seems
dangerous and a massive maintenance burden.

This can of course be fixed with plugins etc like setuptools does.
But the basic problem stays the same, distutils can be broken by
external software.

I do acknowledge that for some people this is exactly what they want
and indeed can see the benefit of integrating with your VCS.  So yes,
that should be made possible.  But as has been said numerous times
distutils should move to some common data formats so that different
tools can be used to provide this type of extra-sweet functionality
for a subset of the users.


> >> ?I have a hard time envisioning anything "lower" than
> >> that.
> >
> > Why so ? Few packaging system use the VCS as the lower system to know
> > what goes in a distribution, most of them (regardless of the platform:
> > windows, unix, mac os x) uses something more low level than this at the
> > implementation level.
> 
> I totally fail to see how this is in any way "lower", and repeat my
> question: "Lower system"? What would that be? What is a "lower system"
> in determining a file list? I suggested a three step process:
> 
> 1. If there is explicit definitions, use that.
> 2. If not, then if there is a VCS, use that.
> 3. If not, include all files except those extensions that are known to
> be temporary files.
> 
> Where in this is there a "lower level"?

Number 1 could be the simple lower level, a list of files (I'm not
saying this is a good choice, I haven't studied all the issues good
enough).  That way the tool you use to consume setup.py (or
whatever it becomes) can construct this list from your VCS in an
automated fashion if you do like that.  But it avoids having the VCS
dependency inside distutils.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Tue Apr  7 15:28:13 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 07 Apr 2009 09:28:13 -0400
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.co
 m>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
Message-ID: <20090407132546.E202E3A4063@sparrow.telecommunity.com>

At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>1.  In the case of entry points for setuptools, it actually recurses
>into EVERY egg directory in your path, not just the egg you requested,
>adds them to your sys.path and additionally looks for four files
>inside of every egg.  On a laptop on local storage, this doesn't
>matter, but when thousands of machines hit the same filer, with many
>python processes, bad things happen...

Install your eggs with --multi-version, and then only the eggs that 
are required for the running script will be added to sys.path or have 
their contents opened.  (Installing them as zip files rather than 
directories may also speed this up.)


From ziade.tarek at gmail.com  Tue Apr  7 16:29:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 7 Apr 2009 16:29:24 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090407130239.GA4867@laurie.devork>
References: <49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
	<49DB0358.8040603@ar.media.kyoto-u.ac.jp>
	<319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
	<20090407130239.GA4867@laurie.devork>
Message-ID: <94bdd2610904070729j57ce5ba3u57715169efa2aaeb@mail.gmail.com>

On Tue, Apr 7, 2009 at 3:02 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
>> Where in this is there a "lower level"?
>
> Number 1 could be the simple lower level, a list of files (I'm not
> saying this is a good choice, I haven't studied all the issues good
> enough). ?That way the tool you use to consume setup.py (or
> whatever it becomes) can construct this list from your VCS in an
> automated fashion if you do like that.

That's an egg-and-chicken issue.

This is the MANIFEST final file, built by all the different mentioned
techniques.

> But it avoids having the VCS
> dependency inside distutils.

Maybe we could think about a plugin system to build this list. (a bit
the way vcs plugins works in setuptools)

Distutils would provide a default plugin and people will be able to
register their own
plugins. I'll write a small proposal for this.

(I hope this won't create a new flame here)

Regards
Tarek

From regebro at gmail.com  Tue Apr  7 16:53:47 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 16:53:47 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090407130239.GA4867@laurie.devork>
References: <49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<49DAE2A5.4090402@ar.media.kyoto-u.ac.jp>
	<319e029f0904062250j4b3ac78ds95c66901e3654767@mail.gmail.com>
	<49DAE773.4080406@ar.media.kyoto-u.ac.jp>
	<319e029f0904062319q956da3ej859dc8f4c60ab43@mail.gmail.com>
	<49DB0358.8040603@ar.media.kyoto-u.ac.jp>
	<319e029f0904070109q556570e5l945bef8190991043@mail.gmail.com>
	<20090407130239.GA4867@laurie.devork>
Message-ID: <319e029f0904070753idcde0d4yab78ee27a5856f32@mail.gmail.com>

On Tue, Apr 7, 2009 at 15:02, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> The VCS is an external piece of software outside of Python and the
> distutils. ?There also isn't just one VCS, there's many of them and
> all of them are on different release schedules which differ from
> Python's and/or distutils's release schedules and can thus break API,
> or datastore semantics at any time (I think this was demonstrated with
> setuptools and svn 1.5 but not sure since I don't use setuptools).
> Therefore for distutils to actually depend on a VCS to function seems
> dangerous and a massive maintenance burden.

I never said it should depend on it. Just that it should be able to
get a file list from the VCS, if a file list isn't otherwise
explicitly defined. No arguments against this has come forward.

> But the basic problem stays the same, distutils can be broken by
> external software.

As mentioned earlier we of course can't rely on internal VCS file
formats to stay the same. Code that does that should not be a part of
the standardlib. But is that really an argument against the principle?

> I do acknowledge that for some people this is exactly what they want
> and indeed can see the benefit of integrating with your VCS. ?So yes,
> that should be made possible. ?But as has been said numerous times
> distutils should move to some common data formats so that different
> tools can be used to provide this type of extra-sweet functionality
> for a subset of the users.

Of course.

>> 1. If there is explicit definitions, use that.
>> 2. If not, then if there is a VCS, use that.
>> 3. If not, include all files except those extensions that are known to
>> be temporary files.
>>
>> Where in this is there a "lower level"?
>
> Number 1 could be the simple lower level, a list of files

It's all a list of files. It's a question of how that list is
generated. "Manually" is not lower level, it's just impractical.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ndbecker2 at gmail.com  Tue Apr  7 17:05:46 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 07 Apr 2009 11:05:46 -0400
Subject: [Distutils] What's missing from easy_install
References: <grfism$7j5$1@ger.gmane.org>
Message-ID: <grfq4a$2df$1@ger.gmane.org>


1. easy_remove!

2. Various utilities to provide query package management.
   - easy_install --list (list files installed)





From tseaver at palladion.com  Tue Apr  7 17:16:23 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 07 Apr 2009 11:16:23 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<87hc129tdx.fsf@benfinney.id.au>	<grdipm$nn9$2@ger.gmane.org>	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
Message-ID: <grfqo8$695$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> Ben Finney wrote:
>> An sdist is *not* just a tarball of the source files.
> 
> It is if you consider any non-tracked file to be an anti-usecase, which
> is what I was answering to :)

There are three classes of files here:

- - "source" files, which should be under version control, and included
  (by default) in the sdist.

- - "derived" files, which should *not* be under version control, but
  which might (in certain circumstances) be included in the sdist.
  E.g., the C file generated from a Pyrex / Cython source file, to
  enable builds on systems without Pyrex / Cython installed.

- - "stray" files, not under version control.  These shuold *never* be
  in an sdist.

Some way of spelling exceptions to the rule "include all files under
source control, and only those under source contrl" is useful (this is
what MANIFEST.in does under setuptools, accordign to PJE).  The default
rule, which would apply in the 95% case, should be fine.

> Moving away from "using VCS is good" vs "using VCS is not good", maybe
> we should focus on the workflows to support. The most significant to me
> is reproducibility (python setup.py sdist output should depend as little
> as possible on its environment - whether the sources are under VCS or
> not, etc....). At least one person said this is not a requirement. Can
> we reach a consensus on this point, and then focus on the technical
> solutions ?

I don't understand the goal of "not depending on the environment" (it
will depend on the version of Python / distutils / setuptools / Cython /
Pyrex you use, for instance, just as C-based packages depend on the
version of autogen and any local autoconf macros).  I do know that
making packagers maintain manifests leads trivially to broken sdists.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ225H+gerLs4ltQ4RAjHzAKDYyW1wpPezB5F3eGWs0iRctUT4HwCfSJI7
p35LqrFxsRe8RsfUyLDyo6Q=
=cwfT
-----END PGP SIGNATURE-----


From zooko at zooko.com  Tue Apr  7 18:59:41 2009
From: zooko at zooko.com (zooko)
Date: Tue, 7 Apr 2009 10:59:41 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <grdipm$nn9$2@ger.gmane.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
Message-ID: <2691B0DE-6FD1-440F-BCEA-6CF7036AE573@zooko.com>

Let me just throw in that my uses cases for all of my projects so far  
are:

1.  There should not be any file included in the distribution that  
isn't included under VCS.

1.a.  *except* that there is this one Python source code file named  
_version.py which is autogenerated from VCS and is not itself stored  
in VCS, documented here:

http://allmydata.org/trac/darcsver/browser/README.txt

2.  There should not be any file included under VCS that isn't  
included in the distribution.

Therefore, for me, setuptools's VCS-integration and  
include_package_data=True is perfect.

Regards,

Zooko

From marius at pov.lt  Tue Apr  7 19:12:39 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 7 Apr 2009 20:12:39 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
References: <87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
Message-ID: <20090407171239.GA30192@fridge.pov.lt>

On Tue, Apr 07, 2009 at 07:20:11AM +0200, Lennart Regebro wrote:
> OK, here is a usecase. I have a directoy with a module, foo.bar. I
> have also in that directory saved a project file from my IDE, because
> naturally I save a project file into the directory of every project I
> have. But since that's not a part of the module itself I never check
> it in.
> 
> My friend Bobo, however, does not use an IDE, and does not create a
> project file.
> 
> In case number one, I specify the files to be included with a wildcard.
> 
> In case number two, I do not specify the files to be included at all,
> but let sdist determine the files by looking at which files are
> version controlled.
> 
> Now, in which of these cases will the sdist created by me an d by Bobo
> be identical? Right. In the one where the file list is determined by
> which files are version controlled. Can we then really say that this
> case is more dependent on the environment?

Consider a different use case: my friend's friend Alice does not like
svn.  She uses git-svn to get a local git repository containing a copy
of the subversion code.  She then uses the usual 'python setup.py sdist'
build process.  It produces a broken distribution (silently) because
setuptools doesn't support git, and Alice didn't know she had to install
a plugin.

Users don't expect version control system metadata to have an influence
on the build process (other than recording the revision number/ID
somewhere in the built package).

Marius Gedminas
-- 
   TCP_SeqNum - The 32-bit Sequence Number, encoded as an ASCII string
      representing the hex value of the Sequence number.  This field
      MUST be sent as lower case because it is not urgent.
                -- RFC 3093
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090407/427481e7/attachment.pgp>

From marius at pov.lt  Tue Apr  7 19:17:22 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 7 Apr 2009 20:17:22 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090407171239.GA30192@fridge.pov.lt>
References: <49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<20090407171239.GA30192@fridge.pov.lt>
Message-ID: <20090407171722.GA30727@fridge.pov.lt>

On Tue, Apr 07, 2009 at 08:12:39PM +0300, Marius Gedminas wrote:
> On Tue, Apr 07, 2009 at 07:20:11AM +0200, Lennart Regebro wrote:
> > OK, here is a usecase. I have a directoy with a module, foo.bar. I
> > have also in that directory saved a project file from my IDE, because
> > naturally I save a project file into the directory of every project I
> > have. But since that's not a part of the module itself I never check
> > it in.
> > 
> > My friend Bobo, however, does not use an IDE, and does not create a
> > project file.
> > 
> > In case number one, I specify the files to be included with a wildcard.
> > 
> > In case number two, I do not specify the files to be included at all,
> > but let sdist determine the files by looking at which files are
> > version controlled.
> > 
> > Now, in which of these cases will the sdist created by me an d by Bobo
> > be identical? Right. In the one where the file list is determined by
> > which files are version controlled. Can we then really say that this
> > case is more dependent on the environment?
> 
> Consider a different use case: my friend's friend Alice does not like
> svn.  She uses git-svn to get a local git repository containing a copy
> of the subversion code.  She then uses the usual 'python setup.py sdist'
> build process.  It produces a broken distribution (silently) because
> setuptools doesn't support git, and Alice didn't know she had to install
> a plugin.
> 
> Users don't expect version control system metadata to have an influence
> on the build process (other than recording the revision number/ID
> somewhere in the built package).

I guess, what I'm saying boils down to: the failure should not be silent
and unnoticed.  If the package's maintainer wants to use $VCS metadata
to define the contents of the sdist, that's fine, but

  (1) it should be specified explicitly, e.g.

          setup(use_vcs_for_manifest=True)

and

  (2) if setuptools is unable to find VCS metadata for any of the VCSes
      it supports (including through plugins), it should abort
      with an error (suggesting the user to look for plugins for the
      appropriate VCS)

The remaining question is: what do you do when the tree is included in
multiple VCSes?  (I've been known to place bzr trees into svn for backup
purposes.  It didn't work out too well.)

Marius Gedminas
-- 
Only great masters of style can succeed in being obtuse.
                -- Oscar Wilde

Most UNIX programmers are great masters of style.
                -- The Unnamed Usenetter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090407/e9baf812/attachment.pgp>

From regebro at gmail.com  Tue Apr  7 19:27:29 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 19:27:29 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090407171239.GA30192@fridge.pov.lt>
References: <87prfq9xf2.fsf@benfinney.id.au> <87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<20090407171239.GA30192@fridge.pov.lt>
Message-ID: <319e029f0904071027m7bd84a1cq141f05327461aab@mail.gmail.com>

On Tue, Apr 7, 2009 at 19:12, Marius Gedminas <marius at pov.lt> wrote:
> Consider a different use case: my friend's friend Alice does not like
> svn. ?She uses git-svn to get a local git repository containing a copy
> of the subversion code. ?She then uses the usual 'python setup.py sdist'
> build process. ?It produces a broken distribution (silently) because
> setuptools doesn't support git, and Alice didn't know she had to install
> a plugin.

But how would it break? It would in that case possibly include *more*
files, as it instead would default to including everything, and not
only VCS files, so you might get intermediary build files included as
well, in worst case. Or Alice's project file, or similar.

But the distribution would not be exactly the same, this is true. Is
that a problem?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From nicholas at metaweb.com  Tue Apr  7 20:06:04 2009
From: nicholas at metaweb.com (Nicholas Veeser)
Date: Tue, 7 Apr 2009 11:06:04 -0700
Subject: [Distutils] pkg_resource require does not seem to find my egg
In-Reply-To: <20090407024702.D79E73A4063@sparrow.telecommunity.com>
References: <49DA98A7.8090003@metaweb.com>
	<20090407024702.D79E73A4063@sparrow.telecommunity.com>
Message-ID: <C8424E2B-D37A-4A26-B5C2-D8D134D8B1DD@metaweb.com>

I am confused.   Doesn't this setup a development environment?
I am trying to build an OS package (gentoo) for deployment to a number  
of hosts.

I want to install my package, with a SqlAlchemy package (0.5.3), and  
an existing Sqlalchemy package (0.4.4)
and have the packages all get along.

My thoughts were that with the appropriate use of pkg_resources.py, I  
could use WorkSet, Environment, etc to
build a sys.path which would correctly have the newer SqlAlchemy egg  
file in the path.

I am doing this manually now, but it seems like pkg_resources.py is a  
more readable, maintainable
method to do the same thing.


What I have working is this:
dino/__init__.py:

....

egg = "SQLAlchemy-0.5.3-py2.4.egg"
for p in sys.path[:]:
     fullname = os.path.join(p, egg)
     if os.path.exists(fullname):
         sys.path.insert(sys.path.index(p), fullname)

...



On Apr 6, 2009, at 7:49 PM, P.J. Eby wrote:

> At 05:04 PM 4/6/2009 -0700, Nicholas Veeser wrote:
>> I am working on a tool we call dino which uses sqlalchemy 0.5.3
>> Its an update of a previous version (called dino) which uses  
>> sqlalchemy 0.4.4.
>>
>> For reasons I don't have to go into, I would like to have both  
>> tools installed on the same host,
>> with almost no changes to the existing tool.   Thus both versions  
>> of sqlalchemy installed
>>
>> So my solution seemed to be use pkg_resources and egg's:
>>
>> - leave sqlalchemy 0.4.4 in:
>>   /usr/lib64/python2.4/site-packages/sqlalchemy
>> - build sqlalchemy 0.5.3 as an Egg and install into:
>>   /usr/lib64/python2.4/site-packages/SQLAlchemy-0.5.3-py2.4.egg
>> - in the root package of the new code,  specify the correct version
>>   from pkg_resources import require; require("SQLAlchemy>=0.5.0")
>
> Change this to defining that dependency in its setup.py, and develop  
> using 'setup.py develop'.  It will then work correctly.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090407/8b0a4639/attachment.htm>

From ziade.tarek at gmail.com  Tue Apr  7 20:18:42 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 7 Apr 2009 20:18:42 +0200
Subject: [Distutils] Plugins for the MANIFEST file
Message-ID: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>

Hello,

I'd like to refactor the current Distutils code that builds the
MANIFEST file so it's pluggable,
like what setuptools introduced.

Distutils would provide some default plugins, and a new global display
option --manifest

I have started a new task in the wiki for that here :
http://wiki.python.org/moin/Distutils/ManifestPluginSystem

If anyone wants to work on it, or comment,

Cheers
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From jim at zope.com  Tue Apr  7 20:23:50 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 7 Apr 2009 14:23:50 -0400
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <20090407132546.E202E3A4063@sparrow.telecommunity.com>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
Message-ID: <A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>


On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:

> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>> 1.  In the case of entry points for setuptools, it actually recurses
>> into EVERY egg directory in your path, not just the egg you  
>> requested,
>> adds them to your sys.path and additionally looks for four files
>> inside of every egg.  On a laptop on local storage, this doesn't
>> matter, but when thousands of machines hit the same filer, with many
>> python processes, bad things happen...
>
> Install your eggs with --multi-version, and then only the eggs that  
> are required for the running script will be added to sys.path or  
> have their contents opened.  (Installing them as zip files rather  
> than directories may also speed this up.)


My experience on Linux is that installing eggs as Zip files slows  
imports.

Jim

--
Jim Fulton
Zope Corporation



From regebro at gmail.com  Tue Apr  7 20:47:22 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 7 Apr 2009 20:47:22 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
Message-ID: <319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>

On Tue, Apr 7, 2009 at 20:18, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> If anyone wants to work on it, or comment,

Some comments:

> "MANIFEST.in is fine"

Except that it introduces Yet Another Domain Specific Language. Wtf
does "prune" mean? It's not obvious. The docs indicate that it
actually excludes, but why it's not called "exclude" is then strange.
We already have a language, Python. We don't need more. ;-)

And the fact that is uses a separate file doesn't help.

Yes, I know I'm just whining. But I don't agree that MANIFEST.in is
fine. I think it is unfine. :)

> Like Bill, he is unable to launch the sdist command because he will probably get a incomplete list of files.

Not if the default is all files except well known temporary files. He
might get more files than desired, though.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Tue Apr  7 20:55:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 07 Apr 2009 14:55:26 -0400
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
Message-ID: <20090407185300.8B3693A4063@sparrow.telecommunity.com>

At 02:23 PM 4/7/2009 -0400, Jim Fulton wrote:

>On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>
>>At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>>1.  In the case of entry points for setuptools, it actually recurses
>>>into EVERY egg directory in your path, not just the egg you
>>>requested,
>>>adds them to your sys.path and additionally looks for four files
>>>inside of every egg.  On a laptop on local storage, this doesn't
>>>matter, but when thousands of machines hit the same filer, with many
>>>python processes, bad things happen...
>>
>>Install your eggs with --multi-version, and then only the eggs that
>>are required for the running script will be added to sys.path or
>>have their contents opened.  (Installing them as zip files rather
>>than directories may also speed this up.)
>
>
>My experience on Linux is that installing eggs as Zip files slows
>imports.

In general, perhaps.  But if they're not actually *on* sys.path, as I 
proposed above, then it should not slow down all imports, and instead 
should speed up the entry point lookups.  Were your tests using 
--multi-version install (i.e., eggs not on sys.path)? 


From zooko at zooko.com  Tue Apr  7 22:17:37 2009
From: zooko at zooko.com (zooko)
Date: Tue, 7 Apr 2009 14:17:37 -0600
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090407171239.GA30192@fridge.pov.lt>
References: <87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<20090407171239.GA30192@fridge.pov.lt>
Message-ID: <B85CA46D-B3E6-49A9-AA2B-8F67FC6153EA@zooko.com>

On Apr 7, 2009, at 11:12 AM, Marius Gedminas wrote:

> Consider a different use case: my friend's friend Alice does not  
> like svn.  She uses git-svn to get a local git repository  
> containing a copy of the subversion code.  She then uses the usual  
> 'python setup.py sdist' build process.  It produces a broken  
> distribution (silently) because setuptools doesn't support git, and  
> Alice didn't know she had to install a plugin.

For the Tahoe project, users can build from sdists (because they come  
with SOURCES.txt file listing all files) or from the official  
revision control tool (which is darcs).  Getting a copy of the files  
which didn't come from the official revision control tool and didn't  
come from an sdist is not supported, and as far as I know no-one has  
ever done it.  Nonetheless I worry about those sorts of rare, quiet  
failures so with PJE's help I added a clause in setuptools_darcs  
which will emit a warning message if there is no SOURCES.txt and also  
it fails to invoke the darcs executable:

http://allmydata.org/trac/setuptools_darcs/browser/setuptools_darcs/ 
setuptools_darcs.py?rev=54#L28

Here is the documentation about this issue in the setuptools_darcs  
plugin (I copied it from the setuptools-git plugin):

http://allmydata.org/trac/setuptools_darcs/browser/README.txt? 
rev=63#L103

Here is the documentation about this issue in the tahoe setup.py:

http://allmydata.org/trac/tahoe/browser/setup.py?rev=3669#L122

I have not heard any complaints about this particular detail.  I have  
heard plenty of complaint about lots of other setuptools issues from  
lots of people.  The Big Two are still:

http://bugs.python.org/setuptools/issue53 # respect the PYTHONPATH

and

http://bugs.python.org/setuptools/issue54 # be more like distutils  
with regard to --prefix=

Regards,

Zooko

From david at ar.media.kyoto-u.ac.jp  Wed Apr  8 04:58:52 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Apr 2009 11:58:52 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <grfqo8$695$1@ger.gmane.org>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<200904051437.26386.jeremy.kloth@4suite.org>	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>	<87tz52a2nt.fsf@benfinney.id.au>	<49D97930.3050703@ar.media.kyoto-u.ac.jp>	<87prfq9xf2.fsf@benfinney.id.au>	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>	<87ljqe9v6n.fsf@benfinney.id.au>	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	<87hc129tdx.fsf@benfinney.id.au>	<grdipm$nn9$2@ger.gmane.org>	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	<87skkl8fws.fsf@benfinney.id.au>	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>
Message-ID: <49DC12EC.7060008@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> David Cournapeau wrote:
> > Ben Finney wrote:
> >> An sdist is *not* just a tarball of the source files.
> > It is if you consider any non-tracked file to be an anti-usecase, which
> > is what I was answering to :)
>
> There are three classes of files here:
>
> - "source" files, which should be under version control, and included
>   (by default) in the sdist.
>
> - "derived" files, which should *not* be under version control, but
>   which might (in certain circumstances) be included in the sdist.
>   E.g., the C file generated from a Pyrex / Cython source file, to
>   enable builds on systems without Pyrex / Cython installed.
>
> - "stray" files, not under version control.  These shuold *never* be
>   in an sdist.

Why those assumptions on what people can do. That's exactly what's wrong
with distutils for me, all those untold assumptions, which are
'obviously' correct. As I said previously, including built documentation
is something I use MANIFEST.in for, and should be supported, or at least
possible.

Making things easier for some people is fine - as long as it does not
prevent other usage. Anything which does not solve the fact that sdist
command gives a different tarball depending on whether it is generated
by distutils, setuptools, paver and co is useless to me. Maybe it is not
important for distutils to solve this, but for me, that's much more
important than support from VCS or any convenience feature which can be
solved by a simple script anyway.

>
> > Moving away from "using VCS is good" vs "using VCS is not good", maybe
> > we should focus on the workflows to support. The most significant to me
> > is reproducibility (python setup.py sdist output should depend as little
> > as possible on its environment - whether the sources are under VCS or
> > not, etc....). At least one person said this is not a requirement. Can
> > we reach a consensus on this point, and then focus on the technical
> > solutions ?
>
> I don't understand the goal of "not depending on the environment" (it
> will depend on the version of Python / distutils / setuptools / Cython /
> Pyrex you use, for instance, just as C-based packages depend on the
> version of autogen and any local autoconf macros).

Environment was a poorly chosen word. Of course, if I use a different
cython, etc... version, the files are different. But on a same
environment (same tools, compilers, etc...), having a different tarball
depending on the sources are under a VCS or another or not is a
fundamental misfeature, at least for me. Whether the sources are under
cvs or not, make dist works the same for any autotools package. Would
people still think it is expected if bdist_msi or bdist_wininst targets
where different if built under VCS or not ?

cheers,

Davi

From ben+python at benfinney.id.au  Wed Apr  8 05:45:32 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 08 Apr 2009 13:45:32 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<200904051437.26386.jeremy.kloth@4suite.org>
	<94bdd2610904051649y1818d4ecs951bb24ccb39737d@mail.gmail.com>
	<20090406004917.A5AC83A4063@sparrow.telecommunity.com>
	<94bdd2610904051755y72220d84w7270cddb35555862@mail.gmail.com>
	<87tz52a2nt.fsf@benfinney.id.au>
	<49D97930.3050703@ar.media.kyoto-u.ac.jp>
	<87prfq9xf2.fsf@benfinney.id.au>
	<49D98F0C.9090505@ar.media.kyoto-u.ac.jp>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
Message-ID: <87vdpf95b7.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Tres Seaver wrote:
> > There are three classes of files here:
> >
> > - "source" files, which should be under version control, and
> > included (by default) in the sdist.
> >
> > - "derived" files, which should *not* be under version control,
> > but which might (in certain circumstances) be included in the
> > sdist. E.g., the C file generated from a Pyrex / Cython source
> > file, to enable builds on systems without Pyrex / Cython
> > installed.
> >
> > - "stray" files, not under version control. These shuold *never*
> > be in an sdist.
> 
> Why those assumptions on what people can do. That's exactly what's
> wrong with distutils for me, all those untold assumptions, which are
> 'obviously' correct. As I said previously, including built
> documentation is something I use MANIFEST.in for, and should be
> supported, or at least possible.

That is, IIUC, covered by the ?derived? class in Tres's list.

-- 
 \     ?Are you pondering what I'm pondering?? ?I think so, Brain, but |
  `\       wouldn't his movies be more suitable for children if he was |
_o__)              named Jean-Claude van Darn?? ?_Pinky and The Brain_ |
Ben Finney


From regebro at gmail.com  Wed Apr  8 06:09:50 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 06:09:50 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp> <grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>

On Wed, Apr 8, 2009 at 04:58, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Why those assumptions on what people can do. That's exactly what's wrong
> with distutils for me, all those untold assumptions, which are
> 'obviously' correct. As I said previously, including built documentation
> is something I use MANIFEST.in for, and should be supported, or at least
> possible.

That's covered by the "derived" case.

> Making things easier for some people is fine - as long as it does not
> prevent other usage.

Nobody has anywhere in this discussion suggested anything that in any
shape, form or way prevents any sort of other usage.

You still aren't listening.

> Anything which does not solve the fact that sdist
> command gives a different tarball depending on whether it is generated
> by distutils, setuptools, paver and co is useless to me.

You agree that distutils have a problem in this area. Setuptools
overrides the behaviour so that it is less broken, to fix your usecase
and include files you want included, but which distutils would not
include. And then you complain that setuptools doesn't create the
exact same sdist as distutils.

Yeah, that makes sense. Not.

> Anything which does not solve the fact

And exactly which anything are you referring to here? Again, you are
not being specific, but you come with implicit general accusations
against everything and everyone. I am not aware on any suggestion done
in this discussion which does *not* solve this problem.

> Environment was a poorly chosen word. Of course, if I use a different
> cython, etc... version, the files are different. But on a same
> environment (same tools, compilers, etc...), having a different tarball
> depending on the sources are under a VCS or another or not is a
> fundamental misfeature, at least for me.

Well, fine. So specify what files you want included then. Problem solved.

You still aren't listening, and as a result, you are wasting peoples
time by arguing against straw men.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Wed Apr  8 06:04:38 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Apr 2009 13:04:38 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87ljqe9v6n.fsf@benfinney.id.au>	
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>	
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>	
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
Message-ID: <49DC2256.2090707@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
> Nobody has anywhere in this discussion suggested anything that in any
> shape, form or way prevents any sort of other usage.
>   

It does if the underlying implementation is not shared correctly between
all the tools. Having plugins to support VCS is ok, as long as the data
used by it can be used.

>
> You agree that distutils have a problem in this area. Setuptools
> overrides the behaviour so that it is less broken, to fix your usecase
> and include files you want included, but which distutils would not
> include. And then you complain that setuptools doesn't create the
> exact same sdist as distutils.
>
> Yeah, that makes sense. Not.
>   

It is ok for different tools to handle things differently *if I can get
those data back*. That's the fundamental problem as of today. An example

distutils sdist  -> setuptools expands the sdist command -> paver
extends the sdist command from setuptools
                         -> another tool expands sdist directly from
distutils

Now, how can I use the distribution files produced by setuptools/paver
in another tool. If there is no way to share this data reliably, that's
a misfeature. That's why a common, underlying low level thing which can
be merged reliably by different tools is needed.

David

From noah.gift at gmail.com  Wed Apr  8 06:25:09 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Wed, 8 Apr 2009 16:25:09 +1200
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <20090407185300.8B3693A4063@sparrow.telecommunity.com>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
	<20090407185300.8B3693A4063@sparrow.telecommunity.com>
Message-ID: <e91cc0270904072125t542a71d9keba09d0378048dc2@mail.gmail.com>

On Wed, Apr 8, 2009 at 6:55 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 02:23 PM 4/7/2009 -0400, Jim Fulton wrote:
>
>> On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>>
>>> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>>>
>>>> 1. ?In the case of entry points for setuptools, it actually recurses
>>>> into EVERY egg directory in your path, not just the egg you
>>>> requested,
>>>> adds them to your sys.path and additionally looks for four files
>>>> inside of every egg. ?On a laptop on local storage, this doesn't
>>>> matter, but when thousands of machines hit the same filer, with many
>>>> python processes, bad things happen...
>>>
>>> Install your eggs with --multi-version, and then only the eggs that
>>> are required for the running script will be added to sys.path or
>>> have their contents opened. ?(Installing them as zip files rather
>>> than directories may also speed this up.)
>>
>>
>> My experience on Linux is that installing eggs as Zip files slows
>> imports.
>
> In general, perhaps. ?But if they're not actually *on* sys.path, as I
> proposed above, then it should not slow down all imports, and instead should
> speed up the entry point lookups. ?Were your tests using --multi-version
> install (i.e., eggs not on sys.path)?
>

Thanks for info, I was not aware of --multi-version.  I have a very
unusual situation so I may not be able to handle normal use cases very
well.  For one simple test, I manually crafted sys.path and was able
to get the following speed improvement, based on using the time
command and strace.  This is probably not what a lot of people want,
but it is interesting to see this is possible for people in my
situation that need raw speed over any possible flexibility.


Total Elapsed Time:  2066 % speed improvement
Lines of strace output:  3050| 1695 % reduction in calls to file system



-- 
Cheers,

Noah

From regebro at gmail.com  Wed Apr  8 06:45:25 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 06:45:25 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <49DC2256.2090707@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp> <grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>

On Wed, Apr 8, 2009 at 06:04, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> It does if the underlying implementation is not shared correctly between
> all the tools.

Since we are discussing what the underlying implementation to be
shared by all tools should be, that's a rather pointless comment.

> Now, how can I use the distribution files produced by setuptools/paver
> in another tool. If there is no way to share this data reliably, that's
> a misfeature. That's why a common, underlying low level thing which can
> be merged reliably by different tools is needed.

Your are fighting windmills.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Wed Apr  8 08:30:25 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Apr 2009 15:30:25 +0900
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>	
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>	
	<87skkl8fws.fsf@benfinney.id.au>	
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>	
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>	
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>	
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
Message-ID: <49DC4481.9080902@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
> On Wed, Apr 8, 2009 at 06:04, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> It does if the underlying implementation is not shared correctly between
>> all the tools.
>>     
>
> Since we are discussing what the underlying implementation to be
> shared by all tools should be, that's a rather pointless comment.
>
>   
>> Now, how can I use the distribution files produced by setuptools/paver
>> in another tool. If there is no way to share this data reliably, that's
>> a misfeature. That's why a common, underlying low level thing which can
>> be merged reliably by different tools is needed.
>>     
>
> Your are fighting windmills.
>   

Ok. Looks like you feel insulted for some reasons to keep insulting me
back. Since I can't make my point clearly, and nobody in this discussion
seems to care anyway, I will leave the discussion.

cheers,

David


From ben+python at benfinney.id.au  Wed Apr  8 09:27:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 08 Apr 2009 17:27:50 +1000
Subject: [Distutils] Deprecate MANIFEST.in
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
Message-ID: <87iqlf8v0p.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Ok. Looks like you feel insulted for some reasons to keep insulting
> me back. Since I can't make my point clearly, and nobody in this
> discussion seems to care anyway, I will leave the discussion.

I hope you can reconsider (but certainly a little time away from the
discussion would be good to help clear the mood). From what I've seen
in this discussion, it's not that nobody else cares, but rather that
we all care but aren't communicating too well.

-- 
 \        ?I went to the hardware store and bought some used paint. It |
  `\                      was in the shape of a house.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From marius at pov.lt  Wed Apr  8 09:40:18 2009
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 8 Apr 2009 10:40:18 +0300
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904071027m7bd84a1cq141f05327461aab@mail.gmail.com>
References: <87ljqe9v6n.fsf@benfinney.id.au>
	<319e029f0904052308i11872bc0uf900126025b27a8e@mail.gmail.com>
	<87hc129tdx.fsf@benfinney.id.au> <grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<20090407171239.GA30192@fridge.pov.lt>
	<319e029f0904071027m7bd84a1cq141f05327461aab@mail.gmail.com>
Message-ID: <20090408074018.GB25110@fridge.pov.lt>

On Tue, Apr 07, 2009 at 07:27:29PM +0200, Lennart Regebro wrote:
> On Tue, Apr 7, 2009 at 19:12, Marius Gedminas <marius at pov.lt> wrote:
> > Consider a different use case: my friend's friend Alice does not like
> > svn. ?She uses git-svn to get a local git repository containing a copy
> > of the subversion code. ?She then uses the usual 'python setup.py sdist'
> > build process. ?It produces a broken distribution (silently) because
> > setuptools doesn't support git, and Alice didn't know she had to install
> > a plugin.
> 
> But how would it break? It would in that case possibly include *more*
> files, as it instead would default to including everything, and not
> only VCS files, so you might get intermediary build files included as
> well, in worst case. Or Alice's project file, or similar.

Are we talking about the current design (which doesn't include
everything), or about some proposed change?

I kind of lost track.

Some redundant background (already seen upthread):

    My personal meeting with this issue was when the application failed
    to work correctly because it was missing some Javascript files due
    to the fact that the released version was built from a tarball
    produced with svn export.  And no, that tarball could not have been
    built with setup.py sdist instead of svn export---it contained more
    than one package and some of those weren't setuptoolized yet (and
    some of those weren't really Python packages).

    I'm repeating this to indicate that in my experience,
    svn-metadata-less builds with no explicit MANIFEST.in do not in fact
    include everything by default.

> But the distribution would not be exactly the same, this is true. Is
> that a problem?

Assuming the distribution is complete and has extra files: no, it
wouldn't necessarily be a problem.

Including extra cruft is inelegant, but not broken (unless that extra
cruft breaks builds somewhere else, because the files in question are
arch-specific and do not get rebuilt---a hypothetical use case probably
not applicable to Python packages).

Marius Gedminas
-- 
Favorite MAC error message: "Not enough memory to eject disk!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090408/303a8e9d/attachment.pgp>

From marius at pov.lt  Wed Apr  8 09:44:26 2009
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 8 Apr 2009 10:44:26 +0300
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
Message-ID: <20090408074426.GC25110@fridge.pov.lt>

On Tue, Apr 07, 2009 at 02:23:50PM -0400, Jim Fulton wrote:
> On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>> 1.  In the case of entry points for setuptools, it actually recurses
>>> into EVERY egg directory in your path, not just the egg you
>>> requested, adds them to your sys.path and additionally looks for
>>> four files inside of every egg.  On a laptop on local storage, this
>>> doesn't matter, but when thousands of machines hit the same filer,
>>> with many python processes, bad things happen...
>>
>> Install your eggs with --multi-version, and then only the eggs that  
>> are required for the running script will be added to sys.path or have 
>> their contents opened.  (Installing them as zip files rather than 
>> directories may also speed this up.)
>
> My experience on Linux is that installing eggs as Zip files slows  
> imports.

Was that in the presence of NFS?

Marius Gedminas
-- 
We have an advanced scalable groupware communication environment (email)
	-- Alan Cox
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090408/b6e15e97/attachment.pgp>

From regebro at gmail.com  Wed Apr  8 09:45:07 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 09:45:07 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <20090408074018.GB25110@fridge.pov.lt>
References: <87ljqe9v6n.fsf@benfinney.id.au> <87hc129tdx.fsf@benfinney.id.au>
	<grdipm$nn9$2@ger.gmane.org>
	<49DA51A7.2040404@ar.media.kyoto-u.ac.jp>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<319e029f0904062220l3b5cd2a7k40b6870475f4fe50@mail.gmail.com>
	<20090407171239.GA30192@fridge.pov.lt>
	<319e029f0904071027m7bd84a1cq141f05327461aab@mail.gmail.com>
	<20090408074018.GB25110@fridge.pov.lt>
Message-ID: <319e029f0904080045y24b1cf19w57d4b0c0f2c00bcb@mail.gmail.com>

On Wed, Apr 8, 2009 at 09:40, Marius Gedminas <marius at pov.lt> wrote:
> Are we talking about the current design (which doesn't include
> everything), or about some proposed change?

Proposed change, of course. That the current situation is broken is
well established. Nobody is defending it.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From noah.gift at gmail.com  Wed Apr  8 09:48:14 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Wed, 8 Apr 2009 19:48:14 +1200
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <20090408074426.GC25110@fridge.pov.lt>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
	<20090408074426.GC25110@fridge.pov.lt>
Message-ID: <e91cc0270904080048n118bc0d9x9620aa64dd3dae5b@mail.gmail.com>

On Wed, Apr 8, 2009 at 7:44 PM, Marius Gedminas <marius at pov.lt> wrote:
> On Tue, Apr 07, 2009 at 02:23:50PM -0400, Jim Fulton wrote:
>> On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>>> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>>> 1. ?In the case of entry points for setuptools, it actually recurses
>>>> into EVERY egg directory in your path, not just the egg you
>>>> requested, adds them to your sys.path and additionally looks for
>>>> four files inside of every egg. ?On a laptop on local storage, this
>>>> doesn't matter, but when thousands of machines hit the same filer,
>>>> with many python processes, bad things happen...
>>>
>>> Install your eggs with --multi-version, and then only the eggs that
>>> are required for the running script will be added to sys.path or have
>>> their contents opened. ?(Installing them as zip files rather than
>>> directories may also speed this up.)
>>
>> My experience on Linux is that installing eggs as Zip files slows
>> imports.
>
> Was that in the presence of NFS?

yes.
>
> Marius Gedminas
> --
> We have an advanced scalable groupware communication environment (email)
> ? ? ? ?-- Alan Cox
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iD8DBQFJ3FXakVdEXeem148RAtLXAJ9cn0H26iHmwCEsjA2c8hctwp6BBgCcCMhQ
> gEaVFxdg6wsSWOx0doulsM0=
> =qWLt
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Cheers,

Noah

From ziade.tarek at gmail.com  Wed Apr  8 10:21:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 10:21:45 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
Message-ID: <94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>

On Tue, Apr 7, 2009 at 8:47 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Tue, Apr 7, 2009 at 20:18, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> If anyone wants to work on it, or comment,
>
> Some comments:
>
>> "MANIFEST.in is fine"
>
> Except that it introduces Yet Another Domain Specific Language. Wtf
> does "prune" mean? It's not obvious. The docs indicate that it
> actually excludes, but why it's not called "exclude" is then strange.
> We already have a language, Python. We don't need more. ;-)
>
> And the fact that is uses a separate file doesn't help.
>
> Yes, I know I'm just whining. But I don't agree that MANIFEST.in is
> fine. I think it is unfine. :)

I agree with this. Remember: I wanted to deprecate MANIFEST.in in favor of
a pure python description. Then people strongly objected.

Tarek

From regebro at gmail.com  Wed Apr  8 10:30:35 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 10:30:35 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
Message-ID: <319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>

On Wed, Apr 8, 2009 at 10:21, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> I agree with this. Remember: I wanted to deprecate MANIFEST.in in favor of
> a pure python description. Then people strongly objected.

How many people objected? Would it be OK to only have support for this
in an external plugin, and not in stdlib?

Again, this is of course only possible if the new distutils is called
something else, as we need backwards compatibility for a Very Long
Time if it continues to be called distutils.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Wed Apr  8 11:44:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Apr 2009 10:44:22 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <87iqlf8v0p.fsf@benfinney.id.au>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<87skkl8fws.fsf@benfinney.id.au>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp> <grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
	<87iqlf8v0p.fsf@benfinney.id.au>
Message-ID: <79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>

2009/4/8 Ben Finney <ben+python at benfinney.id.au>:
> David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:
>
>> Ok. Looks like you feel insulted for some reasons to keep insulting
>> me back. Since I can't make my point clearly, and nobody in this
>> discussion seems to care anyway, I will leave the discussion.
>
> I hope you can reconsider (but certainly a little time away from the
> discussion would be good to help clear the mood). From what I've seen
> in this discussion, it's not that nobody else cares, but rather that
> we all care but aren't communicating too well.

As a bystander, what *I* care most about is that Python ends up with a
single approach to creating and distributing extensions. What concerns
me most about this whole discussion, is that it seems like no-one is
attempting to compromise, and participants are pulling further apart
rather than converging on a solution.

Two solutions is significantly worse than none, in my book.

Paul.

From p.f.moore at gmail.com  Wed Apr  8 11:48:21 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Apr 2009 10:48:21 +0100
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp> <grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
	<87iqlf8v0p.fsf@benfinney.id.au>
	<79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
Message-ID: <79990c6b0904080248u2f2599eeobe7b1610d107b29f@mail.gmail.com>

2009/4/8 Paul Moore <p.f.moore at gmail.com>:
> As a bystander, what *I* care most about is that Python ends up with a
> single approach to creating and distributing extensions. What concerns
> me most about this whole discussion, is that it seems like no-one is
> attempting to compromise, and participants are pulling further apart
> rather than converging on a solution.
>
> Two solutions is significantly worse than none, in my book.

BTW, I share David's pain here - I have spent some time in the past
explaining *my* requirements, and have found that I repeatedly come up
against either blank "I don't understand why you want that" responses,
or suggestions that I change how I work to match what's proposed.

Nobody seems to be attempting to collect requirements here.

Paul.

PS No, I won't state my requirements again. That'll only restart the
flames. Maybe later, if someone starts a genuine, unbiased, attempt to
collect requirements...

From ziade.tarek at gmail.com  Wed Apr  8 11:54:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 11:54:49 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <79990c6b0904080248u2f2599eeobe7b1610d107b29f@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<grfqo8$695$1@ger.gmane.org> <49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
	<87iqlf8v0p.fsf@benfinney.id.au>
	<79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
	<79990c6b0904080248u2f2599eeobe7b1610d107b29f@mail.gmail.com>
Message-ID: <94bdd2610904080254r44409be7s6801d1f42d5c8a3d@mail.gmail.com>

On Wed, Apr 8, 2009 at 11:48 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/4/8 Paul Moore <p.f.moore at gmail.com>:
>> As a bystander, what *I* care most about is that Python ends up with a
>> single approach to creating and distributing extensions. What concerns
>> me most about this whole discussion, is that it seems like no-one is
>> attempting to compromise, and participants are pulling further apart
>> rather than converging on a solution.
>>
>> Two solutions is significantly worse than none, in my book.
>
> BTW, I share David's pain here - I have spent some time in the past
> explaining *my* requirements, and have found that I repeatedly come up
> against either blank "I don't understand why you want that" responses,
> or suggestions that I change how I work to match what's proposed.
>
> Nobody seems to be attempting to collect requirements here.
>

I do, in the wiki. I am trying to synchronize the work done at Pycon, and
in the future. I am trying to synthethize the needs there.

http://wiki.python.org/moin/DistutilsVersionFight
http://wiki.python.org/moin/Distutils/ManifestPluginSystem
http://wiki.python.org/moin/Distutils/Metadata
http://wiki.python.org/moin/Distutils/StaticMetadata
http://wiki.python.org/moin/Distutils/StandardizeEggInfo

What will happen in any case, is that I am going to move Distutils
forward and make decisions for Python 2.7, based on those pages that will
become PEPs at some point.

Some people will complain here as usual, but nevertheless.

So, if you and anyone wants to help, read those wiki pages
and join, by commenting them or completing them.

Cheers
Tarek

From ziade.tarek at gmail.com  Wed Apr  8 12:03:39 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 12:03:39 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>
Message-ID: <94bdd2610904080303h602868d0w2af540a2317a97b@mail.gmail.com>

On Wed, Apr 8, 2009 at 10:30 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Wed, Apr 8, 2009 at 10:21, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> I agree with this. Remember: I wanted to deprecate MANIFEST.in in favor of
>> a pure python description. Then people strongly objected.
>
> How many people objected? Would it be OK to only have support for this
> in an external plugin, and not in stdlib?

These changes will go into Python current Distutils. You are welcome to help
on enhancing this proposal.

>
> Again, this is of course only possible if the new distutils is called
> something else, as we need backwards compatibility for a Very Long
> Time if it continues to be called distutils.

We are going to work as we said during the Summit :

- make distutils lighter, and provide better/simpler ways to use it
- include the good bits of setuptools in it
- provide a backport for 2.5/2.6 as soon as 2.7 is out.

I am changing my strategy. Since I am the current Distutils
maintainer, I will take the role of
'packaging dictator' for Python 2.7.

If you think I am missing some points (I am probably), You will need
to complete the wiki pages
or write a new page there explaining what you would like to see
included in Distutils.

But I will not work in flame-threads anymore on my side.

Cheers
Tarek

From regebro at gmail.com  Wed Apr  8 12:06:11 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 12:06:11 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp> <grfqo8$695$1@ger.gmane.org>
	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
	<87iqlf8v0p.fsf@benfinney.id.au>
	<79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
Message-ID: <319e029f0904080306u24678475x8fdcf614e5233e37@mail.gmail.com>

Meta discussions are generally a complete waste of time. This is the
only thing I intend to say on the issue.

On Wed, Apr 8, 2009 at 11:44, Paul Moore <p.f.moore at gmail.com> wrote:
> As a bystander, what *I* care most about is that Python ends up with a
> single approach to creating and distributing extensions. What concerns
> me most about this whole discussion, is that it seems like no-one is
> attempting to compromise, and participants are pulling further apart
> rather than converging on a solution.

It's bad if it seems like that to you, because I would say that this
is completely incorrect. There is no pulling apart, because there are
no standpoints that can pull apart and there are no diverging
proposals.

There has been two related proposals afaik. Tareks talk about plugins,
and mine where I said I would like the default behavior to be:

1. The file list is gotten from a specification in setup.py.
2. If no such specification, the file list is gotten from the VCS.
3. If no VCS, include all files except well known temporary file extensions.

This is completely compatible with Tareks proposal for plugins, as
this simply could be the three default plugins, if no other plugins
are specified. And there has not come up any real arguments against
either mine or Tareks proposals. So how could this be "pulling apart"?

> PS No, I won't state my requirements again. That'll only restart the
> flames. Maybe later, if someone starts a genuine, unbiased, attempt to
> collect requirements...

Tarek has collected requirements and are still doing so, and everybody
is completely open for new requirements, or explanations of why
proposed solutions doesn't work.

Regards
-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Wed Apr  8 12:11:37 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 12:11:37 +0200
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904080306u24678475x8fdcf614e5233e37@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>
	<grfqo8$695$1@ger.gmane.org> <49DC12EC.7060008@ar.media.kyoto-u.ac.jp>
	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>
	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>
	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>
	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>
	<87iqlf8v0p.fsf@benfinney.id.au>
	<79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
	<319e029f0904080306u24678475x8fdcf614e5233e37@mail.gmail.com>
Message-ID: <94bdd2610904080311y73e198dbi88d4c1f6821a3001@mail.gmail.com>

On Wed, Apr 8, 2009 at 12:06 PM, Lennart Regebro <regebro at gmail.com> wrote:
> There has been two related proposals afaik. Tareks talk about plugins,
> and mine where I said I would like the default behavior to be:
>
> 1. The file list is gotten from a specification in setup.py.
> 2. If no such specification, the file list is gotten from the VCS.
> 3. If no VCS, include all files except well known temporary file extensions.
>
> This is completely compatible with Tareks proposal for plugins, as
> this simply could be the three default plugins, if no other plugins
> are specified. And there has not come up any real arguments against
> either mine or Tareks proposals. So how could this be "pulling apart"?

Right, notice that my proposal here on plugins is intended to be compatible
with how everyone do collect files. It's just a generic tool.

And maybe we will eventually reach a consensus later on the other proposal.

++
Tarek

From regebro at gmail.com  Wed Apr  8 12:11:46 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 12:11:46 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <94bdd2610904080303h602868d0w2af540a2317a97b@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>
	<94bdd2610904080303h602868d0w2af540a2317a97b@mail.gmail.com>
Message-ID: <319e029f0904080311s5eca4730lc8df4df14eb2e212@mail.gmail.com>

On Wed, Apr 8, 2009 at 12:03, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> But I will not work in flame-threads anymore on my side.

Probably a good idea.

The only thing I'm worried about is the fact that you are proposing
both to remove and add things to distutils. Although backports will be
available, how will this work? If I use a new version of distutils,
and somebody uses 2.5 to install my package, how will he know how to
fix the errors that appear? And how will my 2.6 package work under 2.7
if you remove backwards compatibility?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Wed Apr  8 12:27:41 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Apr 2009 11:27:41 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
Message-ID: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>

2009/4/8 Tarek Ziad? <ziade.tarek at gmail.com>:
>> Nobody seems to be attempting to collect requirements here.
>>
>
> I do, in the wiki. I am trying to synchronize the work done at Pycon, and
> in the future. I am trying to synthethize the needs there.
>
> http://wiki.python.org/moin/DistutilsVersionFight
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem
> http://wiki.python.org/moin/Distutils/Metadata
> http://wiki.python.org/moin/Distutils/StaticMetadata
> http://wiki.python.org/moin/Distutils/StandardizeEggInfo
>
> What will happen in any case, is that I am going to move Distutils
> forward and make decisions for Python 2.7, based on those pages that will
> become PEPs at some point.

I'm aware of this, and sorry if my comments seemed to ignore this (but
see below)

> Some people will complain here as usual, but nevertheless.

:-)

> So, if you and anyone wants to help, read those wiki pages
> and join, by commenting them or completing them.

Hmm, I'm not sure where I'd put my requirements in that lot. So I'll
try putting them here (and risk the flames :-)) Please feel free to
incorporate them in the Wiki in any form you feel is appropriate.

1. (Meta-requirement) I want to be able to download a Windows
installer[1] for *every* package I need.
1a. This means that the barrier for packagers building Windows
installers should be as low as possible.
1b. It also means that other formats (e.g. eggs) should offer no
benefit over Windows installers

I'm willing to concede that not all developers can be expected to
build distributions for Windows (but if they are building *anything*
for Windows, it should be "obvious" that a Windows installer is the
only sane choice[2]). Given that, the following requirements are what
I'd want for stuff I'm expected to build myself:

2. For pure Python extensions, I can take a "standard" source
tarball[3] and simply run the "standard" distutils command to build a
Windows installer.
2a. I can do this on any machine, even if it has no network connection
(obviously I'd get the source tarball on disk or equivalent in that
case).

3. For C extensions with no dependency on external libraries, I could
do the same assuming I had the "correct" C compiler present (mingw or
the right version of MSVC).

4. For C extensions with external dependencies, it gets more complex -
but essentially it should be "easy" for the developer to write
setup.py in such a way that I can download the external dependencies
in some well-defined form, configure a few file locations, then build
a binary without further interaction.
4a. This is certainly possible - it's how Python itself builds
4b. This is the only case, really, where pure distutils currently
fails to satisfy *my* needs. But as I'd much rather have binary builds
in this case, and as I currently simply don't use packages in this
category that don't have binary distributions, I don't have any
specific requirements here.

[1] In current terms, "Windows installer" means either bdist_wininst
or bdist_msi - something that integrates with Windows "Add/Remove
programs".
[2] That's the problem (for me) that setuptools/easy_install has
introduced. There are now packages which build Windows binaries in egg
format only. Clearly there is some advantage to be had currently in
distributing eggs, which is *not* present in Windows installers. That
advantage is not relevant to me, so I lose out because there's no
Windows installer, even though the developer clearly has the means to
produce one.
[3] I don't distribute packages, so I am not 100% sure what a
"standard" is, but I guess sdist is what I mean here, in current
terms.

I hope this helps. It's intended to. I tried very hard to avoid it
being an anti-setuptools comment, but I have to say that before
setuptools/easy_install came along, EVERY SINGLE ONE of my
requirements was satisfied by core distutils (even 4 - as the fact
that building binaries in this context was difficult enough that all
projects in this category supplied bdist_wininst installers as a
matter of course). As a user, it feels to me as if all the benefits of
setuptools (and by association, of this discussion) are for the
developer, at the expense of the end user (well, end users in my
situation, at least). I'd like to be proved wrong over this, of course
:-)

Paul.

From ziade.tarek at gmail.com  Wed Apr  8 13:01:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 13:01:32 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <319e029f0904080311s5eca4730lc8df4df14eb2e212@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>
	<94bdd2610904080303h602868d0w2af540a2317a97b@mail.gmail.com>
	<319e029f0904080311s5eca4730lc8df4df14eb2e212@mail.gmail.com>
Message-ID: <94bdd2610904080401s2cfaa723o2d01eb9e3f5328ce@mail.gmail.com>

On Wed, Apr 8, 2009 at 12:11 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Wed, Apr 8, 2009 at 12:03, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> But I will not work in flame-threads anymore on my side.
>
> Probably a good idea.
>
> The only thing I'm worried about is the fact that you are proposing
> both to remove and add things to distutils. Although backports will be
> available, how will this work? If I use a new version of distutils,
> and somebody uses 2.5 to install my package, how will he know how to
> fix the errors that appear?
>And how will my 2.6 package work under 2.7
> if you remove backwards compatibility?

I don't think we want to break any current feature in 2.7, we will
probably just display
deprecation warnings and keep them, then wait until the next python version.

And if someone want to use your package, that uses a new feature
that is not present in 2.5/2.6, he will need to install the backport.

Some good practice will be needed for package developers
for this (maybe a kind error message)

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From regebro at gmail.com  Wed Apr  8 13:06:47 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 13:06:47 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <94bdd2610904080401s2cfaa723o2d01eb9e3f5328ce@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<319e029f0904080130w57295c0h2cc215e511dfbebd@mail.gmail.com>
	<94bdd2610904080303h602868d0w2af540a2317a97b@mail.gmail.com>
	<319e029f0904080311s5eca4730lc8df4df14eb2e212@mail.gmail.com>
	<94bdd2610904080401s2cfaa723o2d01eb9e3f5328ce@mail.gmail.com>
Message-ID: <319e029f0904080406r19690656i59c3e86b3ea858c4@mail.gmail.com>

On Wed, Apr 8, 2009 at 13:01, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> I don't think we want to break any current feature in 2.7, we will
> probably just display
> deprecation warnings and keep them, then wait until the next python version.
>
> And if someone want to use your package, that uses a new feature
> that is not present in 2.5/2.6, he will need to install the backport.
>
> Some good practice will be needed for package developers
> for this (maybe a kind error message)

Well, that's survivable, if not ideal.
I would push hard for a rename if I could just come up with a good
name. But I can't. :-)

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From tseaver at palladion.com  Wed Apr  8 14:15:53 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 08 Apr 2009 08:15:53 -0400
Subject: [Distutils] Deprecate MANIFEST.in
In-Reply-To: <319e029f0904080306u24678475x8fdcf614e5233e37@mail.gmail.com>
References: <94bdd2610904050945y66dfc982uac56c333601921d7@mail.gmail.com>	<49DAD3D9.7070400@ar.media.kyoto-u.ac.jp>
	<grfqo8$695$1@ger.gmane.org>	<49DC12EC.7060008@ar.media.kyoto-u.ac.jp>	<319e029f0904072109ue74d752ybca6ce64fd8d8c31@mail.gmail.com>	<49DC2256.2090707@ar.media.kyoto-u.ac.jp>	<319e029f0904072145m2e30c2b5pddb5c0baaafdc6e8@mail.gmail.com>	<49DC4481.9080902@ar.media.kyoto-u.ac.jp>	<87iqlf8v0p.fsf@benfinney.id.au>	<79990c6b0904080244g344e7b13w4f1b390c1a29a7cd@mail.gmail.com>
	<319e029f0904080306u24678475x8fdcf614e5233e37@mail.gmail.com>
Message-ID: <gri4hu$ttr$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lennart Regebro wrote:

> There has been two related proposals afaik. Tareks talk about plugins,
> and mine where I said I would like the default behavior to be:
> 
> 1. The file list is gotten from a specification in setup.py.
> 2. If no such specification, the file list is gotten from the VCS.

  2a. There should be a way to specify deltas from the list generated
      via the VCS (to include particular derived files, or exclude
      some sources).

> 3. If no VCS, include all files except well known temporary file extensions.

I would never want #3, myself, but I have no problem with it being
available as an option for the VCS-averse.  It should probably use the
same "deltas" model as in #2a.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3JV5+gerLs4ltQ4RAn42AKC2TbBtwPNeYKKLJO2UZiXk4vUCuACgtHho
sMemhM5bb2E8kjT9ELg44rw=
=Pq4q
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Wed Apr  8 14:18:41 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 08 Apr 2009 08:18:41 -0400
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <e91cc0270904080048n118bc0d9x9620aa64dd3dae5b@mail.gmail.com>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>	<20090407132546.E202E3A4063@sparrow.telecommunity.com>	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>	<20090408074426.GC25110@fridge.pov.lt>
	<e91cc0270904080048n118bc0d9x9620aa64dd3dae5b@mail.gmail.com>
Message-ID: <gri4n6$ttr$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Noah Gift wrote:
> On Wed, Apr 8, 2009 at 7:44 PM, Marius Gedminas <marius at pov.lt> wrote:
>> On Tue, Apr 07, 2009 at 02:23:50PM -0400, Jim Fulton wrote:
>>> On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>>>> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>>>> 1.  In the case of entry points for setuptools, it actually recurses
>>>>> into EVERY egg directory in your path, not just the egg you
>>>>> requested, adds them to your sys.path and additionally looks for
>>>>> four files inside of every egg.  On a laptop on local storage, this
>>>>> doesn't matter, but when thousands of machines hit the same filer,
>>>>> with many python processes, bad things happen...
>>>> Install your eggs with --multi-version, and then only the eggs that
>>>> are required for the running script will be added to sys.path or have
>>>> their contents opened.  (Installing them as zip files rather than
>>>> directories may also speed this up.)
>>> My experience on Linux is that installing eggs as Zip files slows
>>> imports.
>> Was that in the presence of NFS?
> 
> yes.

Jim's report, which matches my experience, is not related to NFS:
multiple, zipped bdist_eggs are slower to import / use thant the same
eggs unzipped.

I think the "zip is faster" meme comes from the case of a single
monolithic zip archive (e.g., the whole stdlib, or all the Zope eggs,
in one file).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3JYh+gerLs4ltQ4RAsYKAKDUEqv2+kVmHPnkLjp+mKNKsxv1SgCdF0OC
UjkqmPNfjYz5/ScBjG9ELN0=
=d5+b
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Wed Apr  8 14:23:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 08 Apr 2009 08:23:49 -0400
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
Message-ID: <gri50q$ttr$3@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Tue, Apr 7, 2009 at 8:47 PM, Lennart Regebro <regebro at gmail.com> wrote:
>> On Tue, Apr 7, 2009 at 20:18, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> If anyone wants to work on it, or comment,
>> Some comments:
>>
>>> "MANIFEST.in is fine"
>> Except that it introduces Yet Another Domain Specific Language. Wtf
>> does "prune" mean? It's not obvious. The docs indicate that it
>> actually excludes, but why it's not called "exclude" is then strange.
>> We already have a language, Python. We don't need more. ;-)
>>
>> And the fact that is uses a separate file doesn't help.
>>
>> Yes, I know I'm just whining. But I don't agree that MANIFEST.in is
>> fine. I think it is unfine. :)
> 
> I agree with this. Remember: I wanted to deprecate MANIFEST.in in favor of
> a pure python description. Then people strongly objected.

I want *less* stuff (ideally nothing) spelled in imperative Python, with
some common declarative file replacing both the information currently in
setup.py and MANIFEST.in.  I thought we were in agreement that
non-introspectable metadata was a Bad Thing(TM)?

  http://wiki.python.org/moin/Distutils/StaticMetadata



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3JdV+gerLs4ltQ4RAhGbAJ4in8+UOgM5GJ/3ISGfU2VHLVh4tgCg13O1
qx4oVMxLRJXHUz794mW2UyM=
=pgtS
-----END PGP SIGNATURE-----


From regebro at gmail.com  Wed Apr  8 14:33:31 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 14:33:31 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <gri50q$ttr$3@ger.gmane.org>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<gri50q$ttr$3@ger.gmane.org>
Message-ID: <319e029f0904080533k2189ec56i71f1f44c37370ef0@mail.gmail.com>

On Wed, Apr 8, 2009 at 14:23, Tres Seaver <tseaver at palladion.com> wrote:
> I want *less* stuff (ideally nothing) spelled in imperative Python, with
> some common declarative file replacing both the information currently in
> setup.py and MANIFEST.in. ?I thought we were in agreement that
> non-introspectable metadata was a Bad Thing(TM)?
>
> ?http://wiki.python.org/moin/Distutils/StaticMetadata

For me, the important part is that it's not spread around many files.
I would also tend to prefer some INI-type file format than python
code. On the other hand, python code offers benefits such as being
able to dynamically change the data depending on things like module
availability and python version, etc.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Wed Apr  8 14:58:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 8 Apr 2009 14:58:07 +0200
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <gri50q$ttr$3@ger.gmane.org>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<gri50q$ttr$3@ger.gmane.org>
Message-ID: <94bdd2610904080558t2eb59319j44d35ac65ff7157d@mail.gmail.com>

On Wed, Apr 8, 2009 at 2:23 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Tarek Ziad? wrote:
>> On Tue, Apr 7, 2009 at 8:47 PM, Lennart Regebro <regebro at gmail.com> wrote:
>>> On Tue, Apr 7, 2009 at 20:18, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>> If anyone wants to work on it, or comment,
>>> Some comments:
>>>
>>>> "MANIFEST.in is fine"
>>> Except that it introduces Yet Another Domain Specific Language. Wtf
>>> does "prune" mean? It's not obvious. The docs indicate that it
>>> actually excludes, but why it's not called "exclude" is then strange.
>>> We already have a language, Python. We don't need more. ;-)
>>>
>>> And the fact that is uses a separate file doesn't help.
>>>
>>> Yes, I know I'm just whining. But I don't agree that MANIFEST.in is
>>> fine. I think it is unfine. :)
>>
>> I agree with this. Remember: I wanted to deprecate MANIFEST.in in favor of
>> a pure python description. Then people strongly objected.
>
> I want *less* stuff (ideally nothing) spelled in imperative Python, with
> some common declarative file replacing both the information currently in
> setup.py and MANIFEST.in. ?I thought we were in agreement that
> non-introspectable metadata was a Bad Thing(TM)?
>
> ?http://wiki.python.org/moin/Distutils/StaticMetadata
>

I consider the MANIFEST static file itself to be part of the metadata.
and the discussions we had were related to the way to *build* this file.

which is not incompatible to what you have proposed I think.

I can imagine a declaration in setup.cfg to provide the commands the options
to use to build metadata, like:

metadata-builder= LIST OF PLUGINS TO USE

for example, a Mercurial script + a MANIFEST.in reader would write in setup.cfg:

metadata-builder=hg,template

Or someone could write another plugin that  builds the file in
imperative Python, but that is
not part of setup.py.

Maybe we could start to write down examples of such setup.cfg files
for a few project out there, in this wiki page.

Tarek

From david at ar.media.kyoto-u.ac.jp  Wed Apr  8 14:42:17 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Apr 2009 21:42:17 +0900
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <gri50q$ttr$3@ger.gmane.org>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<gri50q$ttr$3@ger.gmane.org>
Message-ID: <49DC9BA9.9050202@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> Tarek Ziad? wrote:
> > On Tue, Apr 7, 2009 at 8:47 PM, Lennart Regebro <regebro at gmail.com>
> wrote:
> >> On Tue, Apr 7, 2009 at 20:18, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
> >>> If anyone wants to work on it, or comment,
> >> Some comments:
> >>
> >>> "MANIFEST.in is fine"
> >> Except that it introduces Yet Another Domain Specific Language. Wtf
> >> does "prune" mean? It's not obvious. The docs indicate that it
> >> actually excludes, but why it's not called "exclude" is then strange.
> >> We already have a language, Python. We don't need more. ;-)
> >>
> >> And the fact that is uses a separate file doesn't help.
> >>
> >> Yes, I know I'm just whining. But I don't agree that MANIFEST.in is
> >> fine. I think it is unfine. :)
> > I agree with this. Remember: I wanted to deprecate MANIFEST.in in
> favor of
> > a pure python description. Then people strongly objected.
>
> I want *less* stuff (ideally nothing) spelled in imperative Python, with
> some common declarative file replacing both the information currently in
> setup.py and MANIFEST.in.  I thought we were in agreement that
> non-introspectable metadata was a Bad Thing(TM)?

It is possible to use a single, static format which is a bit more
powerful than .ini (to allow conditional and the likes), like haskell
packaging tool does [1] (or rpm .spec file does as well [2]). I am
convinced that such a format would cover many needs without even needing
any python code except a dummy setup file. It also has the advantages
that it can be made compatible with distutils and a potential successor.

This problem has been solved by other people in other communities. It
may not be perfect, but it is certainly better than the current
distutils situation, and it has been widely used,

cheers,

David

[1]
http://www.haskell.org/cabal/release/cabal-1.6.0.2/doc/users-guide/authors.html#pkg-descr
[2] http://www.rpm.org/max-rpm/s1-rpm-inside-conditionals.html

From pje at telecommunity.com  Wed Apr  8 15:06:29 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 08 Apr 2009 09:06:29 -0400
Subject: [Distutils] Plugins for the MANIFEST file
In-Reply-To: <319e029f0904080533k2189ec56i71f1f44c37370ef0@mail.gmail.co
 m>
References: <94bdd2610904071118k2fab2a3ydfb66d563dcc31c4@mail.gmail.com>
	<319e029f0904071147m54b1bff7hda435fc2a31229d@mail.gmail.com>
	<94bdd2610904080121u211c4ab2p47160bd3e2e46bf1@mail.gmail.com>
	<gri50q$ttr$3@ger.gmane.org>
	<319e029f0904080533k2189ec56i71f1f44c37370ef0@mail.gmail.com>
Message-ID: <20090408130402.6FBA53A4063@sparrow.telecommunity.com>

At 02:33 PM 4/8/2009 +0200, Lennart Regebro wrote:
>On Wed, Apr 8, 2009 at 14:23, Tres Seaver <tseaver at palladion.com> wrote:
> > I want *less* stuff (ideally nothing) spelled in imperative Python, with
> > some common declarative file replacing both the information currently in
> > setup.py and MANIFEST.in.  I thought we were in agreement that
> > non-introspectable metadata was a Bad Thing(TM)?
> >
> >  http://wiki.python.org/moin/Distutils/StaticMetadata
>
>For me, the important part is that it's not spread around many files.
>I would also tend to prefer some INI-type file format than python
>code. On the other hand, python code offers benefits such as being
>able to dynamically change the data depending on things like module
>availability and python version, etc.

That's why my preference is to make the static format oriented for 
machine-readability rather than human read/write ability.  It should 
then also be possible to generate the static format from Python or 
from other formats, and to define a standard around either setup.py 
commands or something else to let an installer tool request 
generation of the static data.

Like WSGI, the lingua franca needn't be pretty, just 
well-defined.  Heck, I'd almost be willing to use XML for the file 
format, since XML would make it easy to tag files with multiple 
pieces of information, such as "this is generated documentation that 
should be installed" vs. "this is documentation source *and* should 
also be installed" (e.g. a README.txt that's also reSTified to 
HTML).  XML also allows the markup vocabulary to be extended, and 
unrecognized markup to be ignored, so that tagging files as 
"installable" in the general case could allow a dumber installer to 
just install them without needing to know if they're docs or data or 
message catalogs or what.

That being said, if there was a better format than XML (for 
appropriate values of "better"; i.e. YAML and JSON don't count), I'd 
be happy with that too.


From erik.labianca at gmail.com  Wed Apr  8 17:07:04 2009
From: erik.labianca at gmail.com (Erik S. LaBianca)
Date: Wed, 08 Apr 2009 11:07:04 -0400
Subject: [Distutils] Distutils changes - end user requirements (Was:
 Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
Message-ID: <49DCBD98.2060407@gmail.com>


On 4/8/09 6:27 AM, Paul Moore wrote:
> 2009/4/8 Tarek Ziad? <ziade.tarek at gmail.com>:
>>> Nobody seems to be attempting to collect requirements here.
>>>
>> I do, in the wiki. I am trying to synchronize the work done at Pycon, and
>> in the future. I am trying to synthethize the needs there.
>>
>> http://wiki.python.org/moin/DistutilsVersionFight
>> http://wiki.python.org/moin/Distutils/ManifestPluginSystem
>> http://wiki.python.org/moin/Distutils/Metadata
>> http://wiki.python.org/moin/Distutils/StaticMetadata
>> http://wiki.python.org/moin/Distutils/StandardizeEggInfo
>>

Perhaps it would be a worthwhile effort to create a
DistUtilsRequirements page on the Wiki? I think we all agree that the
current situation needs help, but what is in scope for distutils and
what needs to be supplied via plugins and third-party tools? I'm sure
for many of you this seems obvious, but I think the flame wars are in
part due to differing understandings of the goal.

Off hand, here are some of the requirements I perceive for the packaging
ecosystem on the whole. I do not think all of them should be met by
distutils proper, but the overall ecosystem of python packaging tools
needs to provide solutions for all of them.

For users:

1. Simple package installation following the conventions of their
platform. IE apt-get, aptitude, yum, synaptic, ports, msi.

2. Simple package installation following python conventions. IE
easy_install, pip or whatever activestates thingy was called.

3. Simple installation including necessary dependencies. IE easy_install
or apt-get. Raw rpm, deb, or msi installs don't follow this.

4. Simple application installation or deployment following both the
"system managed" and "application managed" models Tarek alluded to.

For distribution packagers:

1. Dependency meta-data. I can't make packages for appA that work
correctly without knowing that it depends on libB version 1.0 or greater.

2. Declarative meta-data in the sdist. I need to be able to unpack an
sdist file and pull out all the information i need to build packages
without connecting to your dvcs or installing a bunch of third party stuff.

3. Rational versioning. I need to know how to express python package
versions in a way I can map it's version number to the versioning scheme
I use. IE RPM, Deb, PyPi.

4. Build tools. I don't want to become an expert on cython, swig,
setuptools, or anything else. I want a standard mechanism to build
packages regardless of their complexity. Distutils bdist sorta fits the
bill here.

5. Reproducible sdists. I want to be able to take your package, make a
small change, and rebuild the package without having to connect up to
your vcs to do it.

For package authors:

1. Easy builds for my own use and distribution. IE create setup.py,
./setup.py bdist_rpm, use the package! I get the impression setuptools
is the more modern way to do this, but I must claim ignorance, personally.

2. Easy integration with VCS so I don't forget to include newly added
files etc. Nobody wants to manually update MANIFEST, or MANIFEST.in, if
the information needed is already in VCS.

3. Support for building C extensions and the like. Distutils covers this
pretty well now, I think.

4. Support for including "run scripts", hacking documentation, user
documentation, support data files, etc. These need to be flagged in the
metadata so distribution packages know what's what.

I'm sure there's a lot more. If there's interest, I'll be happy to try
to consolidate others contributions and clarifications and put it on the
Wiki.

Thanks

--erik

From marius at pov.lt  Wed Apr  8 19:54:29 2009
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 8 Apr 2009 20:54:29 +0300
Subject: [Distutils] short circuiting module lookups
In-Reply-To: <gri4n6$ttr$2@ger.gmane.org>
References: <e91cc0270904070454x45d81240m10c2fa25aa4d96be@mail.gmail.com>
	<20090407132546.E202E3A4063@sparrow.telecommunity.com>
	<A67843DA-B177-4B75-9B8B-648A88BB646D@zope.com>
	<20090408074426.GC25110@fridge.pov.lt>
	<e91cc0270904080048n118bc0d9x9620aa64dd3dae5b@mail.gmail.com>
	<gri4n6$ttr$2@ger.gmane.org>
Message-ID: <20090408175429.GA11049@fridge.pov.lt>

On Wed, Apr 08, 2009 at 08:18:41AM -0400, Tres Seaver wrote:
> Noah Gift wrote:
> > On Wed, Apr 8, 2009 at 7:44 PM, Marius Gedminas <marius at pov.lt> wrote:
> >> On Tue, Apr 07, 2009 at 02:23:50PM -0400, Jim Fulton wrote:
> >>> My experience on Linux is that installing eggs as Zip files slows
> >>> imports.
> >> Was that in the presence of NFS?
> > 
> > yes.
> 
> Jim's report, which matches my experience, is not related to NFS:
> multiple, zipped bdist_eggs are slower to import / use thant the same
> eggs unzipped.

Right, and I was wondering whether the presence of NFS changes that or
not.  (It's plausible -- if every file access requires a network
roundtrip, maybe accessing one big file is faster than accessing many
small files.  I'm afraid I've exposed my total ignorance of NFS here ;)

> I think the "zip is faster" meme comes from the case of a single
> monolithic zip archive (e.g., the whole stdlib, or all the Zope eggs,
> in one file).

Yes, good point.

Marius Gedminas
-- 
I code in vi because I don't want to learn another OS. :)
                -- Robert Love
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090408/dc5d3a4c/attachment-0001.pgp>

From jaraco at jaraco.com  Wed Apr  8 21:12:36 2009
From: jaraco at jaraco.com (Jason R. Coombs)
Date: Wed, 8 Apr 2009 15:12:36 -0400
Subject: [Distutils] setuptools 0.7a for Python 3 fails to build on Windows
Message-ID: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C4439@hornigold.jaraco.com>

Thanks to Lennart Regebro for putting together a build of setuptools for
Python 3 here:
http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-pyth
on-3/

Unfortunately, the package fails to build on Windows.  Refer to the blog
post (and comments) for details.

The problem occurs when get_script_args calls resource_string for the
'cli.exe' launcher.

It turns out that in Python 3, the following command yields the same error:

>>> open('setuptools/cli.exe', 'rt').read()

resource_string ultimately executes the same command, which is why the build
fails.

The correct thing to do in this particular case is to open the file in
binary mode, which returns `bytes` instead of a Unicode `str`.
Unfortunately, I'm not sure what negative impact would come from altering
pkg_resources.DefaultProvider._get to always read bytes, and that doesn't
strike me as the correct solution.

So, I'm writing here to ask: what should be done about pkg_resources in
Python 3 to support getting a package resource that's binary data?

As I see it, there are a few options,
- always have pkg_resources providers return bytes.
- read the bytes and try converting to str, falling back to bytes on
failure.
- require a parameter to indicate what type of content is expected.

There are probably others.  Any suggestions are appreciated.

Regards,
Jason
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6998 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090408/2aaae567/attachment.bin>

From regebro at gmail.com  Wed Apr  8 21:52:49 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 8 Apr 2009 21:52:49 +0200
Subject: [Distutils] setuptools 0.7a for Python 3 fails to build on
	Windows
In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C4439@hornigold.jaraco.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C4439@hornigold.jaraco.com>
Message-ID: <319e029f0904081252o3ef3278at30a1e1f2b2ca7e5e@mail.gmail.com>

On Wed, Apr 8, 2009 at 21:12, Jason R. Coombs <jaraco at jaraco.com> wrote:
> Unfortunately, I'm not sure what negative impact would come from altering
> pkg_resources.DefaultProvider._get to always read bytes, and that doesn't
> strike me as the correct solution.

No, if you do that everything explodes, and you can't even run
python3.0 setup.py anymore. So that's definitely not right. :-)

> So, I'm writing here to ask: what should be done about pkg_resources in
> Python 3 to support getting a package resource that's binary data?
>
> As I see it, there are a few options,
> - always have pkg_resources providers return bytes.
> - read the bytes and try converting to str, falling back to bytes on
> failure.

That change does still run all the tests at least, so it's worth a
try. I changed DefaultProvider._get to:

    def _get(self, path):
        stream = open(path, 'rb')
        try:
            data = stream.read()
            return data.decode()
        except UnicodeDecodeError:
            return data
        finally:
            stream.close()

Try if that makes a difference.

> - require a parameter to indicate what type of content is expected.

Tricky, since _get is called from somewhere else than where the fact
that it's cli.exe thet should be opened is set. There is too much
iterators and indirection in pkg_resources for my small brain. I
usually get a headache. :-)

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From sridharr at activestate.com  Wed Apr  8 22:28:24 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 08 Apr 2009 13:28:24 -0700
Subject: [Distutils] setup.py --install-requires?
Message-ID: <49DD08E8.4040601@activestate.com>

Is it possible to get the value of install_requires for an arbitrary 
package without having to parse setup.py?

I see that --name, --version, --description, --provides, and so on are 
available as an argument to setup.py, but --install-requires is missing. 
Why?

For example, zc.catalog declares these dependencies in its setup.py

       install_requires=['ZODB3',
                         'pytz',
                         'setuptools',
                         'zope.catalog',
                         'zope.container',
                         'zope.component',
                         'zope.i18nmessageid',
                         'zope.index>=3.5.1',
                         'zope.interface',
                         'zope.publisher',
                         'zope.schema',
                         'zope.security',
                         ],

Is it possible to reliably get this list in a custom Python code without 
having to parse setup.py?

From strawman at astraw.com  Wed Apr  8 22:35:23 2009
From: strawman at astraw.com (Andrew Straw)
Date: Wed, 08 Apr 2009 13:35:23 -0700
Subject: [Distutils] setup.py --install-requires?
In-Reply-To: <49DD08E8.4040601@activestate.com>
References: <49DD08E8.4040601@activestate.com>
Message-ID: <49DD0A8B.5050801@astraw.com>

Sridhar Ratnakumar wrote:
> Is it possible to get the value of install_requires for an arbitrary
> package without having to parse setup.py?
> 
> I see that --name, --version, --description, --provides, and so on are
> available as an argument to setup.py, but --install-requires is missing.
> Why?
> 
> For example, zc.catalog declares these dependencies in its setup.py
> 
>       install_requires=['ZODB3',
>                         'pytz',
>                         'setuptools',
>                         'zope.catalog',
>                         'zope.container',
>                         'zope.component',
>                         'zope.i18nmessageid',
>                         'zope.index>=3.5.1',
>                         'zope.interface',
>                         'zope.publisher',
>                         'zope.schema',
>                         'zope.security',
>                         ],
> 
> Is it possible to reliably get this list in a custom Python code without
> having to parse setup.py?

stdeb does it like this:

install_requires = \
  open(os.path.join(egg_info_dirname,'requires.txt'),'rU').read()


From jaraco at jaraco.com  Wed Apr  8 23:37:37 2009
From: jaraco at jaraco.com (Jason R. Coombs)
Date: Wed, 8 Apr 2009 17:37:37 -0400
Subject: [Distutils] setuptools 0.7a for Python 3 fails to build on
 Windows
In-Reply-To: <319e029f0904081252o3ef3278at30a1e1f2b2ca7e5e@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C4439@hornigold.jaraco.com>
	<319e029f0904081252o3ef3278at30a1e1f2b2ca7e5e@mail.gmail.com>
Message-ID: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C443A@hornigold.jaraco.com>

Lennart,
	That seems to have done the trick.  Between that fix and using the
64-bit cli.exe and gui.exe from http://bugs.python.org/setuptools/issue2, I
was able to install setuptools on Python 3.0.1 64-bit on Windows Vista SP1
and run easy_install.  For that build in particular, a Windows installer is
available at
http://dl.getdropbox.com/u/54081/setuptools-0.7a1.win-amd64.exe. I'll try to
put together a 32-bit build also later.

	Thanks so much for doing the Python 3 port.  I'll be sure to report
back if I encounter any further problems.

	Now I can get started porting my namespace packages to Python 3.

Jason

> -----Original Message-----
> From: Lennart Regebro [mailto:regebro at gmail.com]
> Sent: Wednesday, 08 April, 2009 15:53
> To: Jason R. Coombs
> Cc: distutils-sig at python.org
> Subject: Re: [Distutils] setuptools 0.7a for Python 3 fails to build on
> Windows
> 
> On Wed, Apr 8, 2009 at 21:12, Jason R. Coombs <jaraco at jaraco.com>
> wrote:
> > Unfortunately, I'm not sure what negative impact would come from
> altering
> > pkg_resources.DefaultProvider._get to always read bytes, and that
> doesn't
> > strike me as the correct solution.
> 
> No, if you do that everything explodes, and you can't even run
> python3.0 setup.py anymore. So that's definitely not right. :-)
> 
> > So, I'm writing here to ask: what should be done about pkg_resources
> in
> > Python 3 to support getting a package resource that's binary data?
> >
> > As I see it, there are a few options,
> > - always have pkg_resources providers return bytes.
> > - read the bytes and try converting to str, falling back to bytes on
> > failure.
> 
> That change does still run all the tests at least, so it's worth a
> try. I changed DefaultProvider._get to:
> 
>     def _get(self, path):
>         stream = open(path, 'rb')
>         try:
>             data = stream.read()
>             return data.decode()
>         except UnicodeDecodeError:
>             return data
>         finally:
>             stream.close()
> 
> Try if that makes a difference.
> 
> > - require a parameter to indicate what type of content is expected.
> 
> Tricky, since _get is called from somewhere else than where the fact
> that it's cli.exe thet should be opened is set. There is too much
> iterators and indirection in pkg_resources for my small brain. I
> usually get a headache. :-)
> 
> --
> Lennart Regebro: Pythonista, Barista, Notsotrista.
> http://regebro.wordpress.com/
> +33 661 58 14 64
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6998 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090408/d3d55e38/attachment-0001.bin>

From pje at telecommunity.com  Wed Apr  8 23:53:07 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 08 Apr 2009 17:53:07 -0400
Subject: [Distutils] setup.py --install-requires?
In-Reply-To: <49DD08E8.4040601@activestate.com>
References: <49DD08E8.4040601@activestate.com>
Message-ID: <20090408215039.E10623A4063@sparrow.telecommunity.com>

At 01:28 PM 4/8/2009 -0700, Sridhar Ratnakumar wrote:
>For example, zc.catalog declares these dependencies in its setup.py
>
>       install_requires=['ZODB3',
>                         'pytz',
>                         'setuptools',
>                         'zope.catalog',
>                         'zope.container',
>                         'zope.component',
>                         'zope.i18nmessageid',
>                         'zope.index>=3.5.1',
>                         'zope.interface',
>                         'zope.publisher',
>                         'zope.schema',
>                         'zope.security',
>                         ],
>
>Is it possible to reliably get this list in a custom Python code 
>without having to parse setup.py?

Something like this should do the trick:

import tempfile, os.path
from setuptools.sandbox import run_setup

def get_requires(setup_dir, empty_tmpdir):
     tmpdir = tempfile.mkdtemp(prefix="egginfotmp-")
     run_setup(os.path.join(setup_dir,'setup.py'), ['-e', tmpdir])
     for dist in pkg_resources.find_distributions(tmpdir, True):
         return dist.requires()
     else:
         raise RuntimeError("egg_info didn't work")

You'll get back a list of pkg_resources.Requirement objects rather 
than strings, but you can turn them back into strings if you like.


From sridharr at activestate.com  Thu Apr  9 01:48:49 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 08 Apr 2009 16:48:49 -0700
Subject: [Distutils] setup.py --install-requires?
In-Reply-To: <20090408215039.E10623A4063@sparrow.telecommunity.com>
References: <49DD08E8.4040601@activestate.com>
	<20090408215039.E10623A4063@sparrow.telecommunity.com>
Message-ID: <49DD37E1.9060307@activestate.com>

On 08/04/09 02:53 PM, P.J. Eby wrote:
> Something like this should do the trick:
>
> import tempfile, os.path
> from setuptools.sandbox import run_setup
>
> def get_requires(setup_dir, empty_tmpdir):
>      tmpdir = tempfile.mkdtemp(prefix="egginfotmp-")
>      run_setup(os.path.join(setup_dir,'setup.py'), ['-e', tmpdir])
>      for dist in pkg_resources.find_distributions(tmpdir, True):
>          return dist.requires()
>      else:
>          raise RuntimeError("egg_info didn't work")

That doesn't look like very reliable. It failed with both zc.catalog[1] 
and pip[2].


[1] error: option -e not recognized
[2] IOError: No such file or directory: 'pip-0.3.1/docs/index.txt'

From zooko at zooko.com  Thu Apr  9 04:55:46 2009
From: zooko at zooko.com (zooko)
Date: Wed, 8 Apr 2009 20:55:46 -0600
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
Message-ID: <42656BBB-226F-4732-BB1A-BBCB1830F435@zooko.com>

On Apr 8, 2009, at 4:27 AM, Paul Moore wrote:

> 1. (Meta-requirement) I want to be able to download a Windows  
> installer[1] for *every* package I need.
> 1a. This means that the barrier for packagers building Windows  
> installers should be as low as possible.
> 1b. It also means that other formats (e.g. eggs) should offer no  
> benefit over Windows installers

I personally, as a consumer of other people's software, prefer to  
acquire their packages as sdist .tar.gz's or as .eggs on all  
platforms, including Windows, which I use regularly.  So your and my  
preferences as a consumer of packages differ on this.  Fortunately,  
as far as the distutils is concerned "Windows installers" vs.  
"sdists" vs. "eggs" are not mutually exclusive.  The distutils will  
provide a standard for metadata so that all of those distribution  
formats can have the same metadata, and it will allow automation such  
as bdist_msi and sdist_dsc (from stdeb) so that it can become easier  
for developers to produce these things.  By the way, the exact same  
sorts of preference-among-consumers issues arises on the Linux side  
(which I also use regularly), where some people mistakenly think that  
they want distutils to support eggs worse, when what they really want  
is for it to support debs better (which I hope stdeb will ultimately  
do).

Regards,

Zooko


From ben+python at benfinney.id.au  Thu Apr  9 05:28:24 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 09 Apr 2009 13:28:24 +1000
Subject: [Distutils] Distutils changes - end user requirements
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
Message-ID: <87vdpe7bfr.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> 1. (Meta-requirement) I want to be able to download a Windows
> installer[1] for *every* package I need.

I presume, by ?meta-requirement?, you're showing awareness that this
is something very much dependent on motivating people to actually
create those installers for each and every package.

> 1a. This means that the barrier for packagers building Windows
> installers should be as low as possible.

This, too, is dependent in part on motivating people to actually get
an environment capable of producing such installers. The selection of
metadata in PYthon packages is only one, IMO relatively minor, input
to that.

> 1b. It also means that other formats (e.g. eggs) should offer no
> benefit over Windows installers

I can't see how this is reasonable at all. Windows installers are
inherently outside the bounds of free-software developers to improve;
yet your requirement is that no alternative be allowed to do better? I
hope I'm misunderstanding your requirement and that you've got
something more reasonable in mind.

> 2. For pure Python extensions, I can take a "standard" source
> tarball[3] and simply run the "standard" distutils command to build a
> Windows installer.
> 2a. I can do this on any machine, even if it has no network connection
> (obviously I'd get the source tarball on disk or equivalent in that
> case).

Does ?any machine? include those without MSI-creation tools? I imagine
(but would love to know otherwise) that the creation of an MSI package
cannot be done on any arbitrary Windows machine, but needs special
(external to distutils) programs and/or libraries installed first.

-- 
 \          ?Instead of a trap door, what about a trap window? The guy |
  `\      looks out it, and if he leans too far, he falls out. Wait. I |
_o__)                guess that's like a regular window.? ?Jack Handey |
Ben Finney


From regebro at gmail.com  Thu Apr  9 06:03:01 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 06:03:01 +0200
Subject: [Distutils] setuptools 0.7a for Python 3 fails to build on
	Windows
In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C443A@hornigold.jaraco.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B662DF3C4439@hornigold.jaraco.com>
	<319e029f0904081252o3ef3278at30a1e1f2b2ca7e5e@mail.gmail.com>
	<8B473FAE8A08C34C9F5666FD4B0A87B662DF3C443A@hornigold.jaraco.com>
Message-ID: <319e029f0904082103tef2c49aq306c7495c8acf78a@mail.gmail.com>

On Wed, Apr 8, 2009 at 23:37, Jason R. Coombs <jaraco at jaraco.com> wrote:
> Lennart,
> ? ? ? ?That seems to have done the trick. ?Between that fix and using the
> 64-bit cli.exe and gui.exe from http://bugs.python.org/setuptools/issue2, I
> was able to install setuptools on Python 3.0.1 64-bit on Windows Vista SP1
> and run easy_install. ?For that build in particular, a Windows installer is
> available at
> http://dl.getdropbox.com/u/54081/setuptools-0.7a1.win-amd64.exe. I'll try to
> put together a 32-bit build also later.
>
> ? ? ? ?Thanks so much for doing the Python 3 port. ?I'll be sure to report
> back if I encounter any further problems.
>
> ? ? ? ?Now I can get started porting my namespace packages to Python 3.

OK, Cool.

I'll try to put in the 2to3 support that I have made for
zope.interfaces soon. That should easy porting as well.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Thu Apr  9 06:20:24 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 06:20:24 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
Message-ID: <319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>

On Wed, Apr 8, 2009 at 12:27, Paul Moore <p.f.moore at gmail.com> wrote:
> 1. (Meta-requirement) I want to be able to download a Windows
> installer[1] for *every* package I need.

I would like to know why you have this requirement. Some sort of
Windows binary, absolutely. But an installer? Would a package
management GUI be enough? A Windows software that can install (and
remove) eggs?

> 1a. This means that the barrier for packagers building Windows
> installers should be as low as possible.

Agreed.

> 1b. It also means that other formats (e.g. eggs) should offer no
> benefit over Windows installers

Well... they don't, do they? Except that they can be easy_installed, right?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Thu Apr  9 06:12:09 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Apr 2009 13:12:09 +0900
Subject: [Distutils] Distutils changes - end user requirements
In-Reply-To: <87vdpe7bfr.fsf@benfinney.id.au>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<87vdpe7bfr.fsf@benfinney.id.au>
Message-ID: <49DD7599.5090805@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
>
> I presume, by ?meta-requirement?, you're showing awareness that this
> is something very much dependent on motivating people to actually
> create those installers for each and every package.
>   

Yes, and eggs decreases this motivation, as eggs are cross-platform (if
only python code of course), so people do not build .exe anymore. With a
wininst/msi-based installation, you get more features, which fit more
with what many windows users expect (software shown in install/remove
panel, can be removed). So those windows users have a worse install
experience than before eggs were widespread.

>
> Does ?any machine? include those without MSI-creation tools? I imagine
> (but would love to know otherwise) that the creation of an MSI package
> cannot be done on any arbitrary Windows machine, but needs special
> (external to distutils) programs and/or libraries installed first.
>   

No, it doesn't need anything special, as it is included in python
(msilib is available at least in python >= 2.5 stdlib). Actually, you
can build almost any extension from wine as long as  mingw compilers are
an option, msi or wininst-based. Msi in particular have features that
eggs will most likely never be able to support well (group policy
support, which is something that some numpy/scipy users have regularly
requested).

cheers,

David


From pje at telecommunity.com  Thu Apr  9 07:01:12 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 09 Apr 2009 01:01:12 -0400
Subject: [Distutils] setup.py --install-requires?
In-Reply-To: <49DD37E1.9060307@activestate.com>
References: <49DD08E8.4040601@activestate.com>
	<20090408215039.E10623A4063@sparrow.telecommunity.com>
	<49DD37E1.9060307@activestate.com>
Message-ID: <20090409045845.40B7D3A4063@sparrow.telecommunity.com>

At 04:48 PM 4/8/2009 -0700, Sridhar Ratnakumar wrote:
>On 08/04/09 02:53 PM, P.J. Eby wrote:
>>Something like this should do the trick:
>>
>>import tempfile, os.path
>>from setuptools.sandbox import run_setup
>>
>>def get_requires(setup_dir, empty_tmpdir):
>>      tmpdir = tempfile.mkdtemp(prefix="egginfotmp-")
>>      run_setup(os.path.join(setup_dir,'setup.py'), ['-e', tmpdir])
>>      for dist in pkg_resources.find_distributions(tmpdir, True):
>>          return dist.requires()
>>      else:
>>          raise RuntimeError("egg_info didn't work")
>
>That doesn't look like very reliable. It failed with both 
>zc.catalog[1] and pip[2].
>
>
>[1] error: option -e not recognized
>[2] IOError: No such file or directory: 'pip-0.3.1/docs/index.txt'

Whoops...  that should be ['egg_info', '-e', tmpdir]


From regebro at gmail.com  Thu Apr  9 07:43:59 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 07:43:59 +0200
Subject: [Distutils] setuptools still supported on Python 2.3?
Message-ID: <319e029f0904082243r18f54e8ene5eeebb9501d0e19@mail.gmail.com>

I noticed when installing setuptools on Python 2.3 that easy_install
fails because it tries to import subprocess, which is a new module in
2.4. I personally do not need Python 2.3 support any longer, but the
docs say it should be supported, which seems to be only partly true.

The rest of setuptools seems to work on Python 2.3 still, although I
haven't tried it in practice. But the tests run.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From chris at simplistix.co.uk  Thu Apr  9 10:49:35 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 09 Apr 2009 09:49:35 +0100
Subject: [Distutils] buildout complains of unused options that are used...
Message-ID: <49DDB69F.10002@simplistix.co.uk>

Hi All,

I have the following buildout:

[buildout]
...
zeo_address = 127.0.0.1:2000
...

...
[zeo.conf]
recipe = collective.recipe.template
input = etc/zeo.conf.in
output = etc/zeo.conf

...and buildout complains that the zeo_address line isn't used, except 
that it is, but collective.recipe.template.

What's causing this whine and what should be doing what to mark that 
option as used?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From p.f.moore at gmail.com  Thu Apr  9 11:24:18 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 10:24:18 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
Message-ID: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>

2009/4/9 Lennart Regebro <regebro at gmail.com>:
> On Wed, Apr 8, 2009 at 12:27, Paul Moore <p.f.moore at gmail.com> wrote:
>> 1. (Meta-requirement) I want to be able to download a Windows
>> installer[1] for *every* package I need.
>
> I would like to know why you have this requirement. Some sort of
> Windows binary, absolutely. But an installer? Would a package
> management GUI be enough? A Windows software that can install (and
> remove) eggs?

Because I like to see my installed Python packages in Windows'
"Add/Remove Programs" list.

Think of it as an application of "There should be one-- and preferably
only one --obvious way to do it" (in this case, install and remove
software on my PC). Although maybe because I'm not Dutch, that doesn't
apply :-)

>> 1a. This means that the barrier for packagers building Windows
>> installers should be as low as possible.
>
> Agreed.
>
>> 1b. It also means that other formats (e.g. eggs) should offer no
>> benefit over Windows installers
>
> Well... they don't, do they? Except that they can be easy_installed, right?

Don't they? I have to admit that I'm baffled by how the features in
setuptools/eggs/easy_install all hang together. What about the magic
that creates executables from scripts? Entry points? Stuff like that.
Don't you need to use eggs to make them work?

If not, why do people distribute eggs rather than bdist_wininst
installers? From what I understand, easy_install can convert and
install a bdist_wininst if an egg doesn't exist, but there's no path
the other way. So by what you're saying, eggs are a strict subset of
bdist_wininst, and so people should be distributing bdist_wininst
installers. But they aren't, so what gives?

Can any packager step up and explain why they execute "python setup.py
bdist_egg" rather than "python setup.py bdist_wininst" when creating
distributions for their Windows users?

Paul.

From p.f.moore at gmail.com  Thu Apr  9 11:37:53 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 10:37:53 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <42656BBB-226F-4732-BB1A-BBCB1830F435@zooko.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<42656BBB-226F-4732-BB1A-BBCB1830F435@zooko.com>
Message-ID: <79990c6b0904090237q1b5f1c62le9039d95203362d5@mail.gmail.com>

2009/4/9 zooko <zooko at zooko.com>:
> On Apr 8, 2009, at 4:27 AM, Paul Moore wrote:
>
>> 1. (Meta-requirement) I want to be able to download a Windows installer[1]
>> for *every* package I need.
>> 1a. This means that the barrier for packagers building Windows installers
>> should be as low as possible.
>> 1b. It also means that other formats (e.g. eggs) should offer no benefit
>> over Windows installers
>
> I personally, as a consumer of other people's software, prefer to acquire
> their packages as sdist .tar.gz's or as .eggs on all platforms, including
> Windows, which I use regularly. ?So your and my preferences as a consumer of
> packages differ on this.

Are you able to explain *why* you prefer this? (I'm not trying to
change your mind at all, just trying to get a wider understanding of
how people's experiences differ).

For me (using Windows only) the benefits of bdist_wininst/bdist_msi are:
- Uses the system package manager (limited as it may be)
- Means I don't need to have a compiler installed (even though I do on
many of my machines)

If I have to guess at your reasons, my assumption would be:
- Consistent means of installing on all platforms
- ???

How far off the mark am I? Written like that, my reasons (obviously,
as I'm biased!) look stronger, so I am pretty sure I'm not
understanding your situation properly.

> ?Fortunately, as far as the distutils is concerned
> "Windows installers" vs. "sdists" vs. "eggs" are not mutually exclusive.
> ?The distutils will provide a standard for metadata so that all of those
> distribution formats can have the same metadata, and it will allow
> automation such as bdist_msi and sdist_dsc (from stdeb) so that it can
> become easier for developers to produce these things. ?By the way, the exact
> same sorts of preference-among-consumers issues arises on the Linux side
> (which I also use regularly), where some people mistakenly think that they
> want distutils to support eggs worse, when what they really want is for it
> to support debs better (which I hope stdeb will ultimately do).

The key thing for me would be if it was easy to convert any given
binary format for a platform into another. This stems from the fact
that compiling extensions is hard on Windows. So being able to convert
eggs to and from bdist_wininst or bdist_msi installers would be
sufficient. But standard metadata isn't quite enough by itself, as it
still requires someone with a compiler and the relevant libraries
configured to create the DLLs, etc.

Given that all the *files* contained in any given binary distribution
are identical (if they aren't that's a different problem!) then
creating converters should be possible - although I'm not aware of
anyone looking at this. Maybe it's a useful area to spend some effort
on?

Paul.

From p.f.moore at gmail.com  Thu Apr  9 11:46:36 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 10:46:36 +0100
Subject: [Distutils] Distutils changes - end user requirements
In-Reply-To: <49DD7599.5090805@ar.media.kyoto-u.ac.jp>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<87vdpe7bfr.fsf@benfinney.id.au>
	<49DD7599.5090805@ar.media.kyoto-u.ac.jp>
Message-ID: <79990c6b0904090246k687c5393jad14e510497ee5f3@mail.gmail.com>

2009/4/9 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> Ben Finney wrote:
>>
>> I presume, by ?meta-requirement?, you're showing awareness that this
>> is something very much dependent on motivating people to actually
>> create those installers for each and every package.
>>
>
> Yes, and eggs decreases this motivation, as eggs are cross-platform (if
> only python code of course), so people do not build .exe anymore. With a
> wininst/msi-based installation, you get more features, which fit more
> with what many windows users expect (software shown in install/remove
> panel, can be removed). So those windows users have a worse install
> experience than before eggs were widespread.
>
>>
>> Does ?any machine? include those without MSI-creation tools? I imagine
>> (but would love to know otherwise) that the creation of an MSI package
>> cannot be done on any arbitrary Windows machine, but needs special
>> (external to distutils) programs and/or libraries installed first.
>>
>
> No, it doesn't need anything special, as it is included in python
> (msilib is available at least in python >= 2.5 stdlib). Actually, you
> can build almost any extension from wine as long as ?mingw compilers are
> an option, msi or wininst-based. Msi in particular have features that
> eggs will most likely never be able to support well (group policy
> support, which is something that some numpy/scipy users have regularly
> requested).

Yes, I am very much aware (hence, as you noted, my use of the term
"meta-requirement") that this is more a social issue than a technical
one - although I believe that technical decisions can influence the
social side.

Thanks to David for answering the rest of your queries much better
than I would have managed!

Paul.

From david at ar.media.kyoto-u.ac.jp  Thu Apr  9 11:28:55 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Apr 2009 18:28:55 +0900
Subject: [Distutils] Distutils changes - end user requirements
 (Was:	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
Message-ID: <49DDBFD7.10604@ar.media.kyoto-u.ac.jp>

Paul Moore wrote:
> If not, why do people distribute eggs rather than bdist_wininst
> installers? 

For python only code, I can see one obvious reason: you can build one
egg on your platform, and it is supposed to work everywhere. I may be
wrong, but I don't think packagers who care about windows a lot would
distribute eggs only. For numpy/scipy, some windows users do request
eggs for numpy/scipy (which we do not provide for numpy/scipy).

cheers,

David

From eric at trueblade.com  Thu Apr  9 11:49:12 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 09 Apr 2009 05:49:12 -0400
Subject: [Distutils] Distutils changes - end user requirements
 (Was:	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
Message-ID: <49DDC498.4010207@trueblade.com>

Paul Moore wrote:
> Can any packager step up and explain why they execute "python setup.py
> bdist_egg" rather than "python setup.py bdist_wininst" when creating
> distributions for their Windows users?

I use both. If I'm installing a single package into a site-wide python, 
I use bdist_wininst. If I'm installing a package where I want to have 
multiple versions (in different directories, often with different 
python.exe's), I run bdist_egg. The latter case is usually installed 
with buildout.

From regebro at gmail.com  Thu Apr  9 12:08:54 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 12:08:54 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
Message-ID: <319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>

2009/4/9 Paul Moore <p.f.moore at gmail.com>:
> Don't they? I have to admit that I'm baffled by how the features in
> setuptools/eggs/easy_install all hang together. What about the magic
> that creates executables from scripts? Entry points? Stuff like that.
> Don't you need to use eggs to make them work?

No....? Entry points work even if you have the source code in a tgz
format and run setup.py install. The distribution format is not
magical for that afaik.

> So by what you're saying, eggs are a strict subset of
> bdist_wininst, and so people should be distributing bdist_wininst
> installers. But they aren't, so what gives?

Nobody knows about it?

But in any case, even if it would be a good idea to have every single
Python package on the system listed in the Add/Remove programs list
(Which I don't think it is, but that's a matter of taste, no logical
arguments behind that), that would in practice mean that each and
every package on PyPI must have a wininstaller, even if it is a
pure-python package. That doesn't seem realistic to me.


When I used Windows I would have preferred to have installed packages
listed as "parts" in the Python installer, so that you can go in an
modify the setup and add and remove parts. As with WinG or MS Office
or whatever. That would also remove the need for a separate installer
exe for each file. This is then basically the same requirements as the
uninstall requirements from Linux people, except that you would have a
GUI.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Thu Apr  9 12:13:44 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 11:13:44 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <49DDBFD7.10604@ar.media.kyoto-u.ac.jp>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<49DDBFD7.10604@ar.media.kyoto-u.ac.jp>
Message-ID: <79990c6b0904090313v3c34955pec913f1a126ca293@mail.gmail.com>

2009/4/9 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> Paul Moore wrote:
>> If not, why do people distribute eggs rather than bdist_wininst
>> installers?
>
> For python only code, I can see one obvious reason: you can build one
> egg on your platform, and it is supposed to work everywhere. I may be
> wrong, but I don't think packagers who care about windows a lot would
> distribute eggs only. For numpy/scipy, some windows users do request
> eggs for numpy/scipy (which we do not provide for numpy/scipy).

The other reason I see is "easy_install handles the dependencies for
you". Which has the other irritating side effect that developers no
longer document dependencies on the package website...

I'm fairly sure I've seen packages which distribute Windows-specific
eggs, but no Windows installers. But offhand, I can't recall which.

Actually, a case in point (sort of) - setuptools itself provides
bdist_wininst installers for Python 2.3, 2.4 and 2.5. It does NOT
provide one for Python 2.6, just an egg file. Now, setuptools is a
pure-Python package, as far as I know, but this is certainly something
I've seen confusing people on this list ("When will there be a Python
2.6 installer for setuptools?") Actually, if setuptools *is*
pure-Python, why are any of these version-specific at all? Standard
bdist_wininst installers for Python packages are version independent.

Executive summary: It's a confusing mess :-(

Paul.

From p.f.moore at gmail.com  Thu Apr  9 12:30:47 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 11:30:47 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
Message-ID: <79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>

2009/4/9 Lennart Regebro <regebro at gmail.com>:
> 2009/4/9 Paul Moore <p.f.moore at gmail.com>:
>> Don't they? I have to admit that I'm baffled by how the features in
>> setuptools/eggs/easy_install all hang together. What about the magic
>> that creates executables from scripts? Entry points? Stuff like that.
>> Don't you need to use eggs to make them work?
>
> No....? Entry points work even if you have the source code in a tgz
> format and run setup.py install. The distribution format is not
> magical for that afaik.
>
>> So by what you're saying, eggs are a strict subset of
>> bdist_wininst, and so people should be distributing bdist_wininst
>> installers. But they aren't, so what gives?
>
> Nobody knows about it?

Possibly :-(

> But in any case, even if it would be a good idea to have every single
> Python package on the system listed in the Add/Remove programs list
> (Which I don't think it is, but that's a matter of taste, no logical
> arguments behind that), that would in practice mean that each and
> every package on PyPI must have a wininstaller, even if it is a
> pure-python package. That doesn't seem realistic to me.

Personally, I'd be happy if every package that currently distributes
any form of Windows binaries, distributed a Windows installer. That's
about the same level of coverage as existed before setuptools
appeared, so I don't think that's impossible to achieve. I agree that
expecting *everything* to have a Windows installer is unreasonable.

As regards your other points regarding Windows installers, I don't
disagree entirely. But my personal preference is to work with the
system packager, even if it's less functional than I'd like.

Paul.

From regebro at gmail.com  Thu Apr  9 13:45:52 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 13:45:52 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
Message-ID: <319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>

2009/4/9 Paul Moore <p.f.moore at gmail.com>:
> Personally, I'd be happy if every package that currently distributes
> any form of Windows binaries, distributed a Windows installer. That's
> about the same level of coverage as existed before setuptools
> appeared, so I don't think that's impossible to achieve. I agree that
> expecting *everything* to have a Windows installer is unreasonable.

I don't see the benefits of a windows installer in that situation. You
are going to have a large set of python libraries, with a small subset
of them in the Installed Software list, and the rest not. What did
that achieve?

> As regards your other points regarding Windows installers, I don't
> disagree entirely. But my personal preference is to work with the
> system packager, even if it's less functional than I'd like.

But the suggestion of having packages managed from the python setup
program is working with the system packager just as much as selecting
what parts of Office you want is. That's what Windows users would
expect. I don't think they, when installing Plone expect to get a
hundred Python packages listed in the Installed Software list, as an
extreme example. And how would it handle multiple installations into
Python, etc.?

I just don't see the benefit. While having a Python package manager, I
*can* see the benefit.

But as always, I don't use Windows much nowadays, so I don't really
care. I just want understand the thinking.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Thu Apr  9 15:18:00 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 14:18:00 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
Message-ID: <79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>

2009/4/9 Lennart Regebro <regebro at gmail.com>:
> 2009/4/9 Paul Moore <p.f.moore at gmail.com>:
>> Personally, I'd be happy if every package that currently distributes
>> any form of Windows binaries, distributed a Windows installer. That's
>> about the same level of coverage as existed before setuptools
>> appeared, so I don't think that's impossible to achieve. I agree that
>> expecting *everything* to have a Windows installer is unreasonable.
>
> I don't see the benefits of a windows installer in that situation. You
> are going to have a large set of python libraries, with a small subset
> of them in the Installed Software list, and the rest not. What did
> that achieve?

Sorry, I don't follow. Maybe I was unclear. One more try, and if I
still manage to confuse you, don't worry about it. It's not a major
point.

As far as I am concerned, all I need is for all the packages *I* use
to be available as Windows installers, which I'll download and
install. All of the Python packages I use are in Windows' Add/Remove
Programs.

Given that "the packages I need" isn't a particularly helpful
definition, and given that I'm a terrible dabbler and like to try out
new packages at an alarming rate (:-)), I'd like it if any package
that is distributed in a Windows binary form at all, to be in
available in Windows installer form (ie, I don't want to be forced
into a situation where, in order to use a package, I have to use an
installation method other than my preferred method).

That sounds horribly dogmatic - "everyone should work to make my life
easier" :-( It wasn't meant like that at all. For context, I'm
thinking of the days before setuptools, when everything either had a
Windows installer (bdist_wininst) or no Windows binaries at all. What
frustrates me about the current situation is seeing cases where the
developer has clearly gone to the effort to build Windows binaries,
but they aren't in a form I can (or rather, want to) use. An
egg->bdist_wininst converter would fix this issue. As would everyone
standardising on bdist_wininst (which, as per the previous message,
appears to be prefectly feasible given that bdist_wininst seems to be
a strict superset of egg...)

>> As regards your other points regarding Windows installers, I don't
>> disagree entirely. But my personal preference is to work with the
>> system packager, even if it's less functional than I'd like.
>
> But the suggestion of having packages managed from the python setup
> program is working with the system packager just as much as selecting
> what parts of Office you want is. That's what Windows users would
> expect. I don't think they, when installing Plone expect to get a
> hundred Python packages listed in the Installed Software list, as an
> extreme example. And how would it handle multiple installations into
> Python, etc.?

Ah! I see what you're getting at. If it is indeed possible to modify
the Python installer (which I believe is a MSI installer) to manage
extensions as "subpackages", then that would be a fine option, indeed.
I wasn't aware that was even possible - and I certainly am not aware
of anyone with expertise in how the standard Pytjhon binaries are
packaged having said they could take such a project on.

But if it could be done, then I'm all in favour.

> I just don't see the benefit. While having a Python package manager, I
> *can* see the benefit.
>
> But as always, I don't use Windows much nowadays, so I don't really
> care. I just want understand the thinking.

Comparing the present with package management integrated into the
Python installer, it's mainly a presumption that the latter isn't
going to happen. I was assuming the choice was between what we have
now and a standalone Python package management system which doesn't
integrate into Add/Remove programs at all. That's a much clearer
"follow the system standard" vs "roll your own" type of choice (which
is still a personal choice, of course...)

On that note, didn't ActiveState distribute their version of Python
with a package manager (PPM)? As far as I know, that was only ever
supported by ActiveState themselves, no-one else ever built PPM
packages for their extensions, and it's been quietly dropped in recent
versions (Google tells me that it vanished around Python 2.3). So that
gives some real-world evidence of how non-standard Python package
managers fare (albeit not very representative, given the limited use
of ActiveState's Python).

Paul.

From eric at trueblade.com  Thu Apr  9 15:50:08 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 09 Apr 2009 09:50:08 -0400
Subject: [Distutils] Distutils changes - end user requirements
 (Was:	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
	<79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
Message-ID: <49DDFD10.6080609@trueblade.com>

Paul Moore wrote:
> An
> egg->bdist_wininst converter would fix this issue. As would everyone
> standardising on bdist_wininst (which, as per the previous message,
> appears to be prefectly feasible given that bdist_wininst seems to be
> a strict superset of egg...)

I don't think this is true. I don't think bdist_wininst supports the 
buildout use case where I want to install multiple versions of multiple 
packages in different (or even the same) directory. This is a critical 
use case for Zope/Plone apps.


From p.f.moore at gmail.com  Thu Apr  9 16:03:45 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 15:03:45 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <49DDFD10.6080609@trueblade.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
	<79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
	<49DDFD10.6080609@trueblade.com>
Message-ID: <79990c6b0904090703y4f65968er1f0f224c138e005d@mail.gmail.com>

2009/4/9 Eric Smith <eric at trueblade.com>:
> Paul Moore wrote:
>>
>> An
>> egg->bdist_wininst converter would fix this issue. As would everyone
>> standardising on bdist_wininst (which, as per the previous message,
>> appears to be prefectly feasible given that bdist_wininst seems to be
>> a strict superset of egg...)
>
> I don't think this is true. I don't think bdist_wininst supports the
> buildout use case where I want to install multiple versions of multiple
> packages in different (or even the same) directory. This is a critical use
> case for Zope/Plone apps.

OK, I'm making that statement based on some earlier comments. I've
probably misunderstood them.

But I thought easy_install could take a bdist_wininst installer and
use it to make an egg? Am I wrong? Or is that not relevant here? Or is
this another one of those "eggs are not easy_install is not
setuptools" distinctions that confuses me so much? :-(

Paul.

From jim at zope.com  Thu Apr  9 16:30:44 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 9 Apr 2009 10:30:44 -0400
Subject: [Distutils] buildout complains of unused options that are
	used...
In-Reply-To: <49DDB69F.10002@simplistix.co.uk>
References: <49DDB69F.10002@simplistix.co.uk>
Message-ID: <300CA49C-665F-4D8D-BD72-EED8975DC84F@zope.com>


On Apr 9, 2009, at 4:49 AM, Chris Withers wrote:

> Hi All,
>
> I have the following buildout:
>
> [buildout]
> ...
> zeo_address = 127.0.0.1:2000
> ...
>
> ...
> [zeo.conf]
> recipe = collective.recipe.template
> input = etc/zeo.conf.in
> output = etc/zeo.conf
>
> ...and buildout complains that the zeo_address line isn't used,  
> except that it is, but collective.recipe.template.

I see no evidence that it is being used. It's not being used by the  
configuration code you shared and the recipe source doesnt' seem to  
use it.

Jim

--
Jim Fulton
Zope Corporation



From pje at telecommunity.com  Thu Apr  9 17:15:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 09 Apr 2009 11:15:31 -0400
Subject: [Distutils] setuptools still supported on Python 2.3?
In-Reply-To: <319e029f0904082243r18f54e8ene5eeebb9501d0e19@mail.gmail.co
 m>
References: <319e029f0904082243r18f54e8ene5eeebb9501d0e19@mail.gmail.com>
Message-ID: <20090409151306.2D3963A4063@sparrow.telecommunity.com>

At 07:43 AM 4/9/2009 +0200, Lennart Regebro wrote:
>I noticed when installing setuptools on Python 2.3 that easy_install
>fails because it tries to import subprocess, which is a new module in
>2.4. I personally do not need Python 2.3 support any longer, but the
>docs say it should be supported, which seems to be only partly true.
>
>The rest of setuptools seems to work on Python 2.3 still, although I
>haven't tried it in practice. But the tests run.

There is no code in setuptools itself that uses subprocess.  Are you 
sure it's not a plugin, or the package you're installing?


From pje at telecommunity.com  Thu Apr  9 17:18:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 09 Apr 2009 11:18:51 -0400
Subject: [Distutils] Distutils changes - end user requirements (Was:
 Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090703y4f65968er1f0f224c138e005d@mail.gmail.co
 m>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
	<79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
	<49DDFD10.6080609@trueblade.com>
	<79990c6b0904090703y4f65968er1f0f224c138e005d@mail.gmail.com>
Message-ID: <20090409151623.1ED873A4063@sparrow.telecommunity.com>

At 03:03 PM 4/9/2009 +0100, Paul Moore wrote:
>2009/4/9 Eric Smith <eric at trueblade.com>:
> > Paul Moore wrote:
> >>
> >> An
> >> egg->bdist_wininst converter would fix this issue. As would everyone
> >> standardising on bdist_wininst (which, as per the previous message,
> >> appears to be prefectly feasible given that bdist_wininst seems to be
> >> a strict superset of egg...)
> >
> > I don't think this is true. I don't think bdist_wininst supports the
> > buildout use case where I want to install multiple versions of multiple
> > packages in different (or even the same) directory. This is a critical use
> > case for Zope/Plone apps.
>
>OK, I'm making that statement based on some earlier comments. I've
>probably misunderstood them.
>
>But I thought easy_install could take a bdist_wininst installer and
>use it to make an egg?

Yes, it can.

You could also, in principle, reverse the process.  However, the 
resulting bdist_wininst wouldn't support multiple versions, because 
the bdist_wininst infrastructure doesn't support that.


From chris at simplistix.co.uk  Thu Apr  9 17:34:47 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 09 Apr 2009 16:34:47 +0100
Subject: [Distutils] buildout complains of unused options that are
	used...
In-Reply-To: <300CA49C-665F-4D8D-BD72-EED8975DC84F@zope.com>
References: <49DDB69F.10002@simplistix.co.uk>
	<300CA49C-665F-4D8D-BD72-EED8975DC84F@zope.com>
Message-ID: <49DE1597.8040802@simplistix.co.uk>

Jim Fulton wrote:
> 
>> [buildout]
>> ...
>> zeo_address = 127.0.0.1:2000
>> ...
>>
>> ...
>> [zeo.conf]
>> recipe = collective.recipe.template
>> input = etc/zeo.conf.in
>> output = etc/zeo.conf
>>
>> ...and buildout complains that the zeo_address line isn't used, except 
>> that it is, but collective.recipe.template.
> 
> I see no evidence that it is being used. It's not being used by the 
> configuration code you shared and the recipe source doesnt' seem to use it.

Sorry, from zeo.conf.in:

%define INSTANCE ${buildout:directory}

<zeo>
    address ${buildout:zeo_address}
</zeo>
...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From tseaver at palladion.com  Thu Apr  9 18:55:37 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 12:55:37 -0400
Subject: [Distutils] RFC:  Updating PEP 345
Message-ID: <49DE2889.8070807@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I haven't heard back from Richard Jones, who is the PEP owner, but have
perpared to update PEP 345[1] in line with what to be the majority view
(at PyCon) for adding "distribution-level" dependency metadata (as
opposed to importable package / module dependencies) to the PKG-INFO file.

I have backed off on the notion of overloading 'Requires:' / 'Provides:'
/ 'Obsoletes:', following Jim's notion of deprecating them in favor of
new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
'Obsoletes-Dist'.

"Stock" distutils should probably spell the arguments to
distutils.core.setup predictably:  'requires_dist', 'provides_dist',
'obsoletes_dist'.  setuptools can treat 'install_requires' as an
undeprecated alias for 'requires_dist'.


[1] http://svn.python.org/view/peps/trunk/pep-0345.txt?view=markup


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3iiI+gerLs4ltQ4RAtVLAKCZsXVoS9a0f0B0o8eAErsPmZP6OACePvJ7
8nx3i8Z4LEOsj0SsNczDtAk=
=l7Kk
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Apr  9 19:05:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 19:05:55 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <49DE2889.8070807@palladion.com>
References: <49DE2889.8070807@palladion.com>
Message-ID: <94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>

Jim started a branch during Pycon, maybe you can push your work over
there for review here ?

Also, I am wondering if we shouldn't merge it with PEP 376, where we
might want to add a REQUIRES file
into the egg.info structure, besides the PKG-INFO file.

Cheers
Tarek


On Thu, Apr 9, 2009 at 6:55 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I haven't heard back from Richard Jones, who is the PEP owner, but have
> perpared to update PEP 345[1] in line with what to be the majority view
> (at PyCon) for adding "distribution-level" dependency metadata (as
> opposed to importable package / module dependencies) to the PKG-INFO file.
>
> I have backed off on the notion of overloading 'Requires:' / 'Provides:'
> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
> new fields. ?I named them 'Requires-Dist:', 'Provides-Dist:', and
> 'Obsoletes-Dist'.
>
> "Stock" distutils should probably spell the arguments to
> distutils.core.setup predictably: ?'requires_dist', 'provides_dist',
> 'obsoletes_dist'. ?setuptools can treat 'install_requires' as an
> undeprecated alias for 'requires_dist'.
>
>
> [1] http://svn.python.org/view/peps/trunk/pep-0345.txt?view=markup
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver ? ? ? ? ?+1 540-429-0999 ? ? ? ? ?tseaver at palladion.com
> Palladion Software ? "Excellence by Design" ? ?http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFJ3iiI+gerLs4ltQ4RAtVLAKCZsXVoS9a0f0B0o8eAErsPmZP6OACePvJ7
> 8nx3i8Z4LEOsj0SsNczDtAk=
> =l7Kk
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From tseaver at palladion.com  Thu Apr  9 19:19:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 13:19:49 -0400
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
Message-ID: <grlans$h9q$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/4/9 Lennart Regebro <regebro at gmail.com>:
>> 2009/4/9 Paul Moore <p.f.moore at gmail.com>:
>>> Don't they? I have to admit that I'm baffled by how the features in
>>> setuptools/eggs/easy_install all hang together. What about the magic
>>> that creates executables from scripts? Entry points? Stuff like that.
>>> Don't you need to use eggs to make them work?
>> No....? Entry points work even if you have the source code in a tgz
>> format and run setup.py install. The distribution format is not
>> magical for that afaik.
>>
>>> So by what you're saying, eggs are a strict subset of
>>> bdist_wininst, and so people should be distributing bdist_wininst
>>> installers. But they aren't, so what gives?
>> Nobody knows about it?
> 
> Possibly :-(
> 
>> But in any case, even if it would be a good idea to have every single
>> Python package on the system listed in the Add/Remove programs list
>> (Which I don't think it is, but that's a matter of taste, no logical
>> arguments behind that), that would in practice mean that each and
>> every package on PyPI must have a wininstaller, even if it is a
>> pure-python package. That doesn't seem realistic to me.
> 
> Personally, I'd be happy if every package that currently distributes
> any form of Windows binaries, distributed a Windows installer. That's
> about the same level of coverage as existed before setuptools
> appeared, so I don't think that's impossible to achieve. I agree that
> expecting *everything* to have a Windows installer is unreasonable.

Is there a technical reason why Windows users cannot build the
installers themselves from "pure Python" sdists?  I would rather
distribute *no* binaries at all, myself, especially if "self-help"
works.  Stuff which requires a compiler is obviously a barrier for many
Windows users:  such packages normally need a Windows-savvy contributor
to do the installer build, which often lags the 'sdist' release by a
noticeable period.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3i41+gerLs4ltQ4RAkhhAJ9zG5t8gSxVZcMzTCLe3ecNeVzRrwCffAGP
f7TDGM1CMxXJtgPIihaZWyw=
=zxkr
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Apr  9 19:20:25 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 19:20:25 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
Message-ID: <94bdd2610904091020mc784974ybb45753361039263@mail.gmail.com>

Btw we also need to add some metadata that should be described in PEP
345 and added in PKG-INFO:

- maintainer and maintainer_email

see: http://bugs.python.org/issue3686


On Thu, Apr 9, 2009 at 7:05 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Jim started a branch during Pycon, maybe you can push your work over
> there for review here ?
>
> Also, I am wondering if we shouldn't merge it with PEP 376, where we
> might want to add a REQUIRES file
> into the egg.info structure, besides the PKG-INFO file.
>
> Cheers
> Tarek
>
>
> On Thu, Apr 9, 2009 at 6:55 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> I haven't heard back from Richard Jones, who is the PEP owner, but have
>> perpared to update PEP 345[1] in line with what to be the majority view
>> (at PyCon) for adding "distribution-level" dependency metadata (as
>> opposed to importable package / module dependencies) to the PKG-INFO file.
>>
>> I have backed off on the notion of overloading 'Requires:' / 'Provides:'
>> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
>> new fields. ?I named them 'Requires-Dist:', 'Provides-Dist:', and
>> 'Obsoletes-Dist'.
>>
>> "Stock" distutils should probably spell the arguments to
>> distutils.core.setup predictably: ?'requires_dist', 'provides_dist',
>> 'obsoletes_dist'. ?setuptools can treat 'install_requires' as an
>> undeprecated alias for 'requires_dist'.
>>
>>
>> [1] http://svn.python.org/view/peps/trunk/pep-0345.txt?view=markup
>>
>>
>> Tres.
>> - --
>> ===================================================================
>> Tres Seaver ? ? ? ? ?+1 540-429-0999 ? ? ? ? ?tseaver at palladion.com
>> Palladion Software ? "Excellence by Design" ? ?http://palladion.com
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.6 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>>
>> iD8DBQFJ3iiI+gerLs4ltQ4RAtVLAKCZsXVoS9a0f0B0o8eAErsPmZP6OACePvJ7
>> 8nx3i8Z4LEOsj0SsNczDtAk=
>> =l7Kk
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From tseaver at palladion.com  Thu Apr  9 19:31:36 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 13:31:36 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
Message-ID: <49DE30F8.8080504@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> Jim started a branch during Pycon, maybe you can push your work over
> there for review here ?

AFAICT, Jim hasn't checked anything into that branch:

> $ svndiff http://svn.python.org/projects/peps/{trunk,branches/jim-update-345}/pep-0345.txt
> $ svn log --stop-on-copy http://svn.python.org/projects/peps/branches/jim-update-345
> ------------------------------------------------------------------------
> r70636 | jim.fulton | 2009-03-27 15:19:28 -0400 (Fri, 27 Mar 2009) | 1 line
> 
> Update meta data reflecting work in setuptools

I am not (yet) a Python committer;  if I were so blessed, I would be
glad to commit this change on that branch.

> Also, I am wondering if we shouldn't merge it with PEP 376, where we
> might want to add a REQUIRES file
> into the egg.info structure, besides the PKG-INFO file.

I wouldn't chunk those changes:  the PEP which specifies the PKG-INFO
format needs to have a clear history.

If PKG-INFO is extended to hold distribution-level requires / provides /
obsoletes, then I can see no reason to add another generated file inside
egg-info (which already contains PKG-INFO).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3jD4+gerLs4ltQ4RArsXAJsHcW/cI5hRiuzOQ/zgvIF4SbGoWACeMzMV
HWBIMloJDE4OE+UQI3zxaWo=
=i/wZ
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Apr  9 20:06:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 20:06:40 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <49DE30F8.8080504@palladion.com>
References: <49DE2889.8070807@palladion.com>
	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
	<49DE30F8.8080504@palladion.com>
Message-ID: <94bdd2610904091106y7dc307f3g2edc32d3194ae9b5@mail.gmail.com>

On Thu, Apr 9, 2009 at 7:31 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Tarek Ziad? wrote:
>> Jim started a branch during Pycon, maybe you can push your work over
>> there for review here ?
>
> AFAICT, Jim hasn't checked anything into that branch:
>
>> $ svndiff http://svn.python.org/projects/peps/{trunk,branches/jim-update-345}/pep-0345.txt
>> $ svn log --stop-on-copy http://svn.python.org/projects/peps/branches/jim-update-345
>> ------------------------------------------------------------------------
>> r70636 | jim.fulton | 2009-03-27 15:19:28 -0400 (Fri, 27 Mar 2009) | 1 line
>>
>> Update meta data reflecting work in setuptools
>
> I am not (yet) a Python committer; ?if I were so blessed, I would be
> glad to commit this change on that branch.

ok, maybe you can push it to the wiki then, and I'll commit it when it's ready ?

>
>> Also, I am wondering if we shouldn't merge it with PEP 376, where we
>> might want to add a REQUIRES file
>> into the egg.info structure, besides the PKG-INFO file.
>
> I wouldn't chunk those changes: ?the PEP which specifies the PKG-INFO
> format needs to have a clear history.
>
> If PKG-INFO is extended to hold distribution-level requires / provides /
> obsoletes, then I can see no reason to add another generated file inside
> egg-info (which already contains PKG-INFO).

Because setuptools does it. It adds a requires.txt file separated from PKG-INFO.
But it would make sense not to have a second file.

Tarek

From tseaver at palladion.com  Thu Apr  9 20:16:57 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 14:16:57 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904091020mc784974ybb45753361039263@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
	<94bdd2610904091020mc784974ybb45753361039263@mail.gmail.com>
Message-ID: <49DE3B99.9010201@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> Btw we also need to add some metadata that should be described in PEP
> 345 and added in PKG-INFO:
> 
> - maintainer and maintainer_email
> 
> see: http://bugs.python.org/issue3686

Revised patch attached.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3juY+gerLs4ltQ4RAnRQAKCiUkNgR7wNDol8xQKEKYVWRvQsxgCgyrrI
OWhYOK7TGE4NyWpn0d+/0Rg=
=nhuG
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pep-0345-update_Pycon2009-2.patch
Type: text/x-patch
Size: 5679 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090409/db044a28/attachment-0001.bin>

From a.cavallo at acm.org  Thu Apr  9 20:23:12 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Thu, 9 Apr 2009 19:23:12 +0100
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904091106y7dc307f3g2edc32d3194ae9b5@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
	<49DE30F8.8080504@palladion.com>
	<94bdd2610904091106y7dc307f3g2edc32d3194ae9b5@mail.gmail.com>
Message-ID: <a71153f0904091123j701f3910kd8e0087ad6b619d0@mail.gmail.com>

Hi,
I might be late and I haven't been to pycon but I've noted in this pep 345:

It is an attempt to re-invent the rpm spec file?

In this case how would you plan to integrate with the rpms dependecies?

Let me provide an usage scenario for packaging a generic rpm ("foobar"
from now on)
depending on another python module (let's call it "depend"):

 1. user "build" his/her install a *local* version of "depend" in
order to build "foobar"
 2. the building of the "foobar" rpm succeed because the "depend" is
installed (although locally)
 3. "build" gives his rpm generated (and possibly the srpms) to the
network administrator
     for deployment

At this point I see two problems:
 1. the administrator installs "foobar" assuming that won't fail (runtime error)
 2. the administrator tries to rebuild "foobar" from the srpm (build time error)

In both case the administrator will blame "build" because:
 1. The developer relying on "foobar" cannot start it (missing
dependency "depend")
 2. The administrator cannot rebuild "foobar" because "depends" is not installed

The problem is excacerbated by the fact that rpm has already a dependecy checker
(as yum/yast/smart and other tools make use of this) and it won't see
the "PKG_INFO"
dependencies.

Regards,
Antonio

On Thu, Apr 9, 2009 at 7:06 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Thu, Apr 9, 2009 at 7:31 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Tarek Ziad? wrote:
>>> Jim started a branch during Pycon, maybe you can push your work over
>>> there for review here ?
>>
>> AFAICT, Jim hasn't checked anything into that branch:
>>
>>> $ svndiff http://svn.python.org/projects/peps/{trunk,branches/jim-update-345}/pep-0345.txt
>>> $ svn log --stop-on-copy http://svn.python.org/projects/peps/branches/jim-update-345
>>> ------------------------------------------------------------------------
>>> r70636 | jim.fulton | 2009-03-27 15:19:28 -0400 (Fri, 27 Mar 2009) | 1 line
>>>
>>> Update meta data reflecting work in setuptools
>>
>> I am not (yet) a Python committer; ?if I were so blessed, I would be
>> glad to commit this change on that branch.
>
> ok, maybe you can push it to the wiki then, and I'll commit it when it's ready ?
>
>>
>>> Also, I am wondering if we shouldn't merge it with PEP 376, where we
>>> might want to add a REQUIRES file
>>> into the egg.info structure, besides the PKG-INFO file.
>>
>> I wouldn't chunk those changes: ?the PEP which specifies the PKG-INFO
>> format needs to have a clear history.
>>
>> If PKG-INFO is extended to hold distribution-level requires / provides /
>> obsoletes, then I can see no reason to add another generated file inside
>> egg-info (which already contains PKG-INFO).
>
> Because setuptools does it. It adds a requires.txt file separated from PKG-INFO.
> But it would make sense not to have a second file.
>
> Tarek
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From tseaver at palladion.com  Thu Apr  9 20:41:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 14:41:43 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <a71153f0904091123j701f3910kd8e0087ad6b619d0@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>	<49DE30F8.8080504@palladion.com>	<94bdd2610904091106y7dc307f3g2edc32d3194ae9b5@mail.gmail.com>
	<a71153f0904091123j701f3910kd8e0087ad6b619d0@mail.gmail.com>
Message-ID: <grlfhf$49l$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antonio Cavallo wrote:

> I might be late and I haven't been to pycon but I've noted in this
> pep 345:
> 
> It is an attempt to re-invent the rpm spec file?

No, it is an attempt to provide Python-specific information about a
package in a machine-readable format, which doesn't require running
'setup()'.  This information will be present in 'sdist' and 'bdist*'
archives, as well as in installed packages.

Once landed, we can look at how layers like setuptools, as well as the
catalog (PyPI) can exploit this information.

> In this case how would you plan to integrate with the rpms dependecies?

Not at all.  There is no guarantee that we know about or can satisfy the
policies of arbitrary distributions.   We hope that adding dependency
information which is targeted at distributions rather than importable
packages / modules will make life easier for the folks who package our
distributions as .rpm, .deb, etc.  E.g., packagers might be able to
write tools which query this data and then generate the files which
match their policies, mapping distribution names, etc.

Note that PEP 314, which is already landed, added dependency-related
fields ('Requires', 'Provides', 'Obsoletes') but specified a semantic
for them which is not as useful (the names in those fields are supposed
to be names of importable packages or modules, rather than distutils
project names).  There are some packages in PyPI already which spell
these fields, but there is no practical way to *use* them in any
automated way.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3kFn+gerLs4ltQ4RAlZQAJ9SVqFlxwpO5yUA7sOYyaWkvHxUAACggX8e
s+JEFb51SYLRt9JFzBRQuG8=
=XI4H
-----END PGP SIGNATURE-----


From theller at ctypes.org  Thu Apr  9 21:36:37 2009
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 09 Apr 2009 21:36:37 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <grlans$h9q$2@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
Message-ID: <grlio5$ejp$1@ger.gmane.org>

Tres Seaver schrieb:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Paul Moore wrote:
>> 2009/4/9 Lennart Regebro <regebro at gmail.com>:
>>> 2009/4/9 Paul Moore <p.f.moore at gmail.com>:
>>>> Don't they? I have to admit that I'm baffled by how the features in
>>>> setuptools/eggs/easy_install all hang together. What about the magic
>>>> that creates executables from scripts? Entry points? Stuff like that.
>>>> Don't you need to use eggs to make them work?
>>> No....? Entry points work even if you have the source code in a tgz
>>> format and run setup.py install. The distribution format is not
>>> magical for that afaik.
>>>
>>>> So by what you're saying, eggs are a strict subset of
>>>> bdist_wininst, and so people should be distributing bdist_wininst
>>>> installers. But they aren't, so what gives?
>>> Nobody knows about it?
>> 
>> Possibly :-(
>> 
>>> But in any case, even if it would be a good idea to have every single
>>> Python package on the system listed in the Add/Remove programs list
>>> (Which I don't think it is, but that's a matter of taste, no logical
>>> arguments behind that), that would in practice mean that each and
>>> every package on PyPI must have a wininstaller, even if it is a
>>> pure-python package. That doesn't seem realistic to me.
>> 
>> Personally, I'd be happy if every package that currently distributes
>> any form of Windows binaries, distributed a Windows installer. That's
>> about the same level of coverage as existed before setuptools
>> appeared, so I don't think that's impossible to achieve. I agree that
>> expecting *everything* to have a Windows installer is unreasonable.
> 
> Is there a technical reason why Windows users cannot build the
> installers themselves from "pure Python" sdists?

No. There's even a script that automates the process completely.  It allows
to build bdist_wininst installers by drag and drop.

http://code.activestate.com/recipes/117248/

"""
Recipe 117248: installing source distributions on windows

Distutil's bdist_wininst installers offer uninstallation support
for Python extensions, many developers however only distribute sources
in zip or tar.gz format. The typical steps to install such a distribution
are:
- download the file
- unpack with winzip into a temporary directory
- open a command prompt and type 'python setup.py install'
- remove the temporary directory

This script unpacks a source distribution into a temporary directory,
builds a windows installer on the fly, executes it, and cleans everything up afterward.
"""


Not that it has gained much interest, afaik.

-- 
Thanks,
Thomas


From ktenney at gmail.com  Thu Apr  9 21:37:09 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Thu, 9 Apr 2009 14:37:09 -0500
Subject: [Distutils] Buildout recipe for managing .deb packages?
Message-ID: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>

Howdy,

Is there a recommended way to manage Debian system
packages with a buildout?

Thanks,
Kent

From jim at zope.com  Thu Apr  9 21:38:31 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 9 Apr 2009 15:38:31 -0400
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
Message-ID: <372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>


On Apr 9, 2009, at 3:37 PM, Kent Tenney wrote:

> Howdy,
>
> Is there a recommended way to manage Debian system
> packages with a buildout?


I can't imagine why you would want to. I probably don't know what  
you're asking.

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Thu Apr  9 21:43:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 21:43:49 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <49DE3B99.9010201@palladion.com>
References: <49DE2889.8070807@palladion.com>
	<94bdd2610904091005v4cf5b5b7t243426b580845c4f@mail.gmail.com>
	<94bdd2610904091020mc784974ybb45753361039263@mail.gmail.com>
	<49DE3B99.9010201@palladion.com>
Message-ID: <94bdd2610904091243k3af145ebh1a153d54984a8a42@mail.gmail.com>

On Thu, Apr 9, 2009 at 8:16 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Tarek Ziad? wrote:
>> Btw we also need to add some metadata that should be described in PEP
>> 345 and added in PKG-INFO:
>>
>> - maintainer and maintainer_email
>>
>> see: http://bugs.python.org/issue3686
>
> Revised patch attached.

Added :
http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt

I'll update the wiki accordingly, and update it everytime we need it.

From p.f.moore at gmail.com  Thu Apr  9 21:45:26 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 20:45:26 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <grlans$h9q$2@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
Message-ID: <79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>

2009/4/9 Tres Seaver <tseaver at palladion.com>:
> Is there a technical reason why Windows users cannot build the
> installers themselves from "pure Python" sdists? ?I would rather
> distribute *no* binaries at all, myself, especially if "self-help"
> works. ?Stuff which requires a compiler is obviously a barrier for many
> Windows users: ?such packages normally need a Windows-savvy contributor
> to do the installer build, which often lags the 'sdist' release by a
> noticeable period.

No technical reason, no. It's as simple as "python setup.py
bdist_wininst" or "python setup.py mdist_msi". Personally, I'm happy
doing that for any pure python package that doesn't provide an
installer.

The only downside is that not all packages document whether they are
pure Python. It can be frustrating to download a package, unpack it,
and try to build it only to find out that it has C code that won't
build. Or even more subtle, it builds fine, but ignores important
speedup code written in C...

But the main reason is social - Windows users expect to download
installers, and have a low tolerance for projects that don't provide
such. And a low tolerance for anything involving a command line, in
many cases. Call us bone idle if you must, but it's a fact you need to
deal with in considering a Windows audience.

However, it's equally true (I believe) that "python setup.py
bdist_wininst" works fine on a Linux box. So it's not as if building
Windows installers is a huge chore for developers, either. (I accept
that there are other tasks, like distribution). It's a trade-off of
developer time vs user time (and I fully accept that this trade-off
comes out differently in an open source/volunteer environment).

Paul.

From jim at zope.com  Thu Apr  9 21:48:01 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 9 Apr 2009 15:48:01 -0400
Subject: [Distutils] buildout complains of unused options that are
	used...
In-Reply-To: <49DE1597.8040802@simplistix.co.uk>
References: <49DDB69F.10002@simplistix.co.uk>
	<300CA49C-665F-4D8D-BD72-EED8975DC84F@zope.com>
	<49DE1597.8040802@simplistix.co.uk>
Message-ID: <039FB43B-1688-4E76-A524-7E0C532EF505@zope.com>


On Apr 9, 2009, at 11:34 AM, Chris Withers wrote:

> Jim Fulton wrote:
>>> [buildout]
>>> ...
>>> zeo_address = 127.0.0.1:2000
>>> ...
>>>
>>> ...
>>> [zeo.conf]
>>> recipe = collective.recipe.template
>>> input = etc/zeo.conf.in
>>> output = etc/zeo.conf
>>>
>>> ...and buildout complains that the zeo_address line isn't used,  
>>> except that it is, but collective.recipe.template.
>> I see no evidence that it is being used. It's not being used by the  
>> configuration code you shared and the recipe source doesnt' seem to  
>> use it.
>
> Sorry, from zeo.conf.in:
>
> %define INSTANCE ${buildout:directory}
>
> <zeo>
>   address ${buildout:zeo_address}
> </zeo>
> ...


OK.  I see what's going on here. The recipe is using a private  
buildout API to perform substitutions at install time, rather than at  
initialization time. A buildout recipe should only read data from  
other sections in the initialization phase, otherwise, buildout  
doesn't see that the option has been read. That's why buildout is  
complaining about an unused option.  To avoid this, the recipe should  
perform the substitution in the initialization phase (__init__ call).

Jim

--
Jim Fulton
Zope Corporation



From p.f.moore at gmail.com  Thu Apr  9 21:51:40 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 20:51:40 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <grlio5$ejp$1@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org> <grlio5$ejp$1@ger.gmane.org>
Message-ID: <79990c6b0904091251k73da3015h698bc86601e7e906@mail.gmail.com>

2009/4/9 Thomas Heller <theller at ctypes.org>:
>> Is there a technical reason why Windows users cannot build the
>> installers themselves from "pure Python" sdists?
>
> No. There's even a script that automates the process completely. ?It allows
> to build bdist_wininst installers by drag and drop.
>
> http://code.activestate.com/recipes/117248/
[...]
> Not that it has gained much interest, afaik.

I, for one, never even knew it existed. That's pretty neat.

Paul.

From trentm at activestate.com  Thu Apr  9 21:54:22 2009
From: trentm at activestate.com (Trent Mick)
Date: Thu, 09 Apr 2009 12:54:22 -0700
Subject: [Distutils] Distutils changes - end user requirements
 (Was:	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
	<79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
Message-ID: <49DE526E.6030400@activestate.com>

Paul Moore wrote:
> On that note, didn't ActiveState distribute their version of Python
> with a package manager (PPM)? As far as I know, that was only ever
> supported by ActiveState themselves, no-one else ever built PPM
> packages for their extensions, and it's been quietly dropped in recent
> versions (Google tells me that it vanished around Python 2.3).

Yes, it was there a long time ago. It was dropped around ActivePython 
2.3, as you say, because it wasn't well architected and, at the time, we 
didn't have the resources to maintain it.

However, we've just recently hired a full time employee to (in part) 
start looking at a "PyPM" package manager again -- this time with an 
appropriate architecture. The hope is to have something to show in the 
fall (perhaps sooner).

Trent

-- 
Trent Mick
trentm at activestate.com

From tseaver at palladion.com  Thu Apr  9 22:02:39 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 16:02:39 -0400
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
Message-ID: <grlk90$j5v$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/4/9 Tres Seaver <tseaver at palladion.com>:
>> Is there a technical reason why Windows users cannot build the
>> installers themselves from "pure Python" sdists?  I would rather
>> distribute *no* binaries at all, myself, especially if "self-help"
>> works.  Stuff which requires a compiler is obviously a barrier for many
>> Windows users:  such packages normally need a Windows-savvy contributor
>> to do the installer build, which often lags the 'sdist' release by a
>> noticeable period.
> 
> No technical reason, no. It's as simple as "python setup.py
> bdist_wininst" or "python setup.py mdist_msi". Personally, I'm happy
> doing that for any pure python package that doesn't provide an
> installer.
> 
> The only downside is that not all packages document whether they are
> pure Python. It can be frustrating to download a package, unpack it,
> and try to build it only to find out that it has C code that won't
> build. Or even more subtle, it builds fine, but ignores important
> speedup code written in C...
> 
> But the main reason is social - Windows users expect to download
> installers, and have a low tolerance for projects that don't provide
> such. And a low tolerance for anything involving a command line, in
> many cases. Call us bone idle if you must, but it's a fact you need to
> deal with in considering a Windows audience.

*Shrug*.  I mostly don't care about donating extra effort to help users
who refuse to help themselves, or to contribute to the product. *That*
isn't going to change, either.  I've been willing to make exceptoins in
the past for "crucial" packages with C extensions, but anything that
requires booting to Windows is mostly not going to happen.

> However, it's equally true (I believe) that "python setup.py
> bdist_wininst" works fine on a Linux box. So it's not as if building
> Windows installers is a huge chore for developers, either. (I accept
> that there are other tasks, like distribution). It's a trade-off of
> developer time vs user time (and I fully accept that this trade-off
> comes out differently in an open source/volunteer environment).

I don't think that works:

$ ../../bin/python setup.py bdist_wininst
running bdist_wininst
running build
installing to build/bdist.linux-i686/wininst
running install_egg_info
running egg_info
writing pkginfo.egg-info/PKG-INFO
writing top-level names to pkginfo.egg-info/top_level.txt
writing dependency_links to pkginfo.egg-info/dependency_links.txt
reading manifest file 'pkginfo.egg-info/SOURCES.txt'
writing manifest file 'pkginfo.egg-info/SOURCES.txt'
Copying pkginfo.egg-info to
build/bdist.linux-i686/wininst/PURELIB/pkginfo-0.1-py2.6.egg-info
running install_scripts
creating '/tmp/tmpJ5kgq8.zip' and adding '.' to it
adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/dependency_links.txt'
adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/SOURCES.txt'
adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/top_level.txt'
adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/PKG-INFO'
creating dist
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules _winreg, win32api or win32con are installed.
error:
/home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
No such file or directory

Note that this package is pure python.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3lRf+gerLs4ltQ4RAjwkAJ4kSb++3dwISpZWoFTWy3ymPfHdMACeMZ42
BzuguG6IxSvFV4Q8Lvh8I98=
=dbEz
-----END PGP SIGNATURE-----


From regebro at gmail.com  Thu Apr  9 22:17:18 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 22:17:18 +0200
Subject: [Distutils] setuptools still supported on Python 2.3?
In-Reply-To: <20090409151306.2D3963A4063@sparrow.telecommunity.com>
References: <319e029f0904082243r18f54e8ene5eeebb9501d0e19@mail.gmail.com>
	<20090409151306.2D3963A4063@sparrow.telecommunity.com>
Message-ID: <319e029f0904091317x24e53c53mc735138acfce4e19@mail.gmail.com>

On Thu, Apr 9, 2009 at 17:15, P.J. Eby <pje at telecommunity.com> wrote:
> There is no code in setuptools itself that uses subprocess. ?Are you sure
> it's not a plugin, or the package you're installing?

Yes, you are right, I got confused by the error message.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Thu Apr  9 22:24:21 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 9 Apr 2009 22:24:21 +0200
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
Message-ID: <319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>

On Thu, Apr 9, 2009 at 21:37, Kent Tenney <ktenney at gmail.com> wrote:
> Howdy,
>
> Is there a recommended way to manage Debian system
> packages with a buildout?

Do you mean make the buildout install .deb packages? You could build a
recipe, but it would only work on .deb systems, partly negating the
point of a buildout...

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ktenney at gmail.com  Thu Apr  9 22:25:24 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Thu, 9 Apr 2009 15:25:24 -0500
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
Message-ID: <ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>

On Thu, Apr 9, 2009 at 2:38 PM, Jim Fulton <jim at zope.com> wrote:
>
> On Apr 9, 2009, at 3:37 PM, Kent Tenney wrote:
>
>> Howdy,
>>
>> Is there a recommended way to manage Debian system
>> packages with a buildout?
>
>
> I can't imagine why you would want to. I probably don't know what you're
> asking.

to get an app to run or compile I do a bunch of
sudo apt-get install xxxx

This can involve tedious trial and error.

Maybe I like the app, maybe I don't.

If I like it, I want a convenient format in which to remember
the required packages, automate their installation,
provide convenience when building machines.

If I don't like it, I'd like to remove the packages I installed.

Buildout does this for eggs, and tarballs via cmmi, I want the
same convenience for system packages.

>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
>

From ktenney at gmail.com  Thu Apr  9 22:26:30 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Thu, 9 Apr 2009 15:26:30 -0500
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
Message-ID: <ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>

On Thu, Apr 9, 2009 at 3:24 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Thu, Apr 9, 2009 at 21:37, Kent Tenney <ktenney at gmail.com> wrote:
>> Howdy,
>>
>> Is there a recommended way to manage Debian system
>> packages with a buildout?
>
> Do you mean make the buildout install .deb packages? You could build a
> recipe, but it would only work on .deb systems, partly negating the
> point of a buildout...

It would help me, and I'm all about me.

>
> --
> Lennart Regebro: Python, Zope, Plone, Grok
> http://regebro.wordpress.com/
> +33 661 58 14 64
>

From pje at telecommunity.com  Thu Apr  9 22:29:38 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 09 Apr 2009 16:29:38 -0400
Subject: [Distutils] bdist_wininst on Linux (was Distutils changes - end
 user requirements (Was: Deprecate MANIFEST.in))
In-Reply-To: <grlk90$j5v$1@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
Message-ID: <20090409202711.E750C3A4063@sparrow.telecommunity.com>

At 04:02 PM 4/9/2009 -0400, Tres Seaver wrote:
>Warning: Can't read registry to find the necessary compiler setting
>Make sure that Python modules _winreg, win32api or win32con are installed.
>error:
>/home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
>No such file or directory
>
>Note that this package is pure python.

Looks like somebody broke it in 2.6 then...  I build the 2.3-2.5 
bdist_wininst installers for setuptools on a Linux box.


From jim at zope.com  Thu Apr  9 22:30:11 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 9 Apr 2009 16:30:11 -0400
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
	<ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
Message-ID: <796D0F24-1927-41D8-8287-801F7930ECBA@zope.com>


On Apr 9, 2009, at 4:25 PM, Kent Tenney wrote:

> On Thu, Apr 9, 2009 at 2:38 PM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Apr 9, 2009, at 3:37 PM, Kent Tenney wrote:
>>
>>> Howdy,
>>>
>>> Is there a recommended way to manage Debian system
>>> packages with a buildout?
>>
>>
>> I can't imagine why you would want to. I probably don't know what  
>> you're
>> asking.
>
> to get an app to run or compile I do a bunch of
> sudo apt-get install xxxx
>
> This can involve tedious trial and error.
>
> Maybe I like the app, maybe I don't.
>
> If I like it, I want a convenient format in which to remember
> the required packages, automate their installation,
> provide convenience when building machines.
>
> If I don't like it, I'd like to remove the packages I installed.
>
> Buildout does this for eggs, and tarballs via cmmi, I want the
> same convenience for system packages.


That's a fascinating (and legitimate) goal. :)  I'm curious if any  
debian experts have other suggestions about how to achive it.

As Lennart mentioned, you'd need to create a recipe that installed  
(and uninstalled) the packages.

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Thu Apr  9 22:30:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 22:30:20 +0200
Subject: [Distutils] Making commands extensible by default
Message-ID: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>

Hello,

This is a side discussion but quiet important ihmo.

== Problem ==

Some people complained about the fact that is was hard to extend
Distutils commands.
You end up rewriting the whole command most of the time.

So what's a command ? It's a class that is used by the distribution
instance when you run Distutils.

roughly:

cmd = Command(distribution)
cmd.initialize_options()
cmd.finalize_options()        <---  allows to check the options if
subcommands where run
cmd.run()                            <---  runs the code

each command can define sub commands, but most of the time it's a
harcoded list, so you need to inherit the command
if you want to add a new behavior.

== work in progress, ==

What we want to do here is being able to define subsets in run(),
sharing the same options environment.

so basically, a rough, generic run() method could be:

def run():
   for func in some_funcs:
      func(self, options)

If "some_funcs" could be defined by a registery with simple names,
anyone could provide new functions
and configure the registery to run a sequence of function.

Given a command name, Distutils can get this list of function, through
a registery.
Each function could register itself into Distutils, like in what I
have started to work here for the manifest file:
see http://wiki.python.org/moin/Distutils/ManifestPluginSystem

The ordering would be configurable through the setup.cfg file.

Any opinion, idea for this part ?

Cheers
Tarek

From p.f.moore at gmail.com  Thu Apr  9 22:30:21 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 21:30:21 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <grlk90$j5v$1@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
Message-ID: <79990c6b0904091330h439eadd1u30a09519598163ea@mail.gmail.com>

2009/4/9 Tres Seaver <tseaver at palladion.com>:
>> However, it's equally true (I believe) that "python setup.py
>> bdist_wininst" works fine on a Linux box. So it's not as if building
>> Windows installers is a huge chore for developers, either. (I accept
>> that there are other tasks, like distribution). It's a trade-off of
>> developer time vs user time (and I fully accept that this trade-off
>> comes out differently in an open source/volunteer environment).
>
> I don't think that works:
>
> $ ../../bin/python setup.py bdist_wininst
[...]
> Warning: Can't read registry to find the necessary compiler setting
> Make sure that Python modules _winreg, win32api or win32con are installed.
> error:
> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
> No such file or directory
>
> Note that this package is pure python.

Hmm, I thought that worked. I'll see if I can build a suitable Linux
VM to give it a try. Just in case it matters, what OS are you running?

Paul.

From ziade.tarek at gmail.com  Thu Apr  9 22:36:14 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 22:36:14 +0200
Subject: [Distutils] bdist_wininst on Linux (was Distutils changes - end
	user requirements (Was: Deprecate MANIFEST.in))
In-Reply-To: <20090409202711.E750C3A4063@sparrow.telecommunity.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
	<20090409202711.E750C3A4063@sparrow.telecommunity.com>
Message-ID: <94bdd2610904091336n2d549f30qc132b7b1b0706030@mail.gmail.com>

Please fill an issue with this bug asap, so we can try to fix it
before Python 2.6.2 final is out


On Thu, Apr 9, 2009 at 10:29 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 04:02 PM 4/9/2009 -0400, Tres Seaver wrote:
>>
>> Warning: Can't read registry to find the necessary compiler setting
>> Make sure that Python modules _winreg, win32api or win32con are installed.
>> error:
>>
>> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
>> No such file or directory
>>
>> Note that this package is pure python.
>
> Looks like somebody broke it in 2.6 then... ?I build the 2.3-2.5
> bdist_wininst installers for setuptools on a Linux box.
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From strawman at astraw.com  Thu Apr  9 22:39:16 2009
From: strawman at astraw.com (Andrew Straw)
Date: Thu, 09 Apr 2009 13:39:16 -0700
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
	<ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
Message-ID: <49DE5CF4.1050603@astraw.com>

Kent Tenney wrote:
> On Thu, Apr 9, 2009 at 3:24 PM, Lennart Regebro <regebro at gmail.com> wrote:
>> On Thu, Apr 9, 2009 at 21:37, Kent Tenney <ktenney at gmail.com> wrote:
>>> Howdy,
>>>
>>> Is there a recommended way to manage Debian system
>>> packages with a buildout?
>> Do you mean make the buildout install .deb packages? You could build a
>> recipe, but it would only work on .deb systems, partly negating the
>> point of a buildout...
> 
> It would help me, and I'm all about me.

To do it properly, I think you would need to maintain a database mapping
buildout distribution names to Debian binary package names.

Zooko wrote something for stdeb that tries to automatically find the
dependencies, to some degree of success, using apt-file. The
implementation is in the function
get_deb_depends_from_setuptools_requires() in
http://github.com/astraw/stdeb/blob/master/stdeb/util.py

From benji at zope.com  Thu Apr  9 22:45:03 2009
From: benji at zope.com (Benji York)
Date: Thu, 9 Apr 2009 16:45:03 -0400
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com> 
	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com> 
	<ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
Message-ID: <e5fff6640904091345r2e0e18f5pcdbb197c168760c1@mail.gmail.com>

On Thu, Apr 9, 2009 at 4:26 PM, Kent Tenney <ktenney at gmail.com> wrote:
> On Thu, Apr 9, 2009 at 3:24 PM, Lennart Regebro <regebro at gmail.com> wrote:
>> On Thu, Apr 9, 2009 at 21:37, Kent Tenney <ktenney at gmail.com> wrote:
>>> Howdy,
>>>
>>> Is there a recommended way to manage Debian system
>>> packages with a buildout?
>>
>> Do you mean make the buildout install .deb packages? You could build a
>> recipe, but it would only work on .deb systems, partly negating the
>> point of a buildout...
>
> It would help me, and I'm all about me.

I approve of your perspective.

Take a look at iw.recipe.cmd (http://pypi.python.org/pypi/iw.recipe.cmd),
it lets you run arbitrary commands during a buildout.

[install-some-debs]
recipe = iw.recipe.cmd
on_install = true
on_update = true
cmds = apt-get install ...
--
Benji York
Senior Software Engineer
Zope Corporation

From p.f.moore at gmail.com  Thu Apr  9 22:52:20 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Apr 2009 21:52:20 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <grlk90$j5v$1@ger.gmane.org>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
Message-ID: <79990c6b0904091352v66834126kefcea89bebe026bb@mail.gmail.com>

2009/4/9 Tres Seaver <tseaver at palladion.com>:
> creating dist
> Warning: Can't read registry to find the necessary compiler setting
> Make sure that Python modules _winreg, win32api or win32con are installed.

This is a warning, so can be ignored.

> error:
> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
> No such file or directory

This is a bug. That "ux-i686" part of the filename looks wrong.

Yes, it's a bug in bdist_wininst, introduced in revision 62197, Mon
Apr 7 01:53:39 2008 UTC.

If I file a bug and it gets fixed, will you build installers for me? :-)

(Seriously, thanks for trying this and locating the bug. I've raised
it as http://bugs.python.org/issue5731).

Paul.

From strawman at astraw.com  Thu Apr  9 22:54:54 2009
From: strawman at astraw.com (Andrew Straw)
Date: Thu, 09 Apr 2009 13:54:54 -0700
Subject: [Distutils] bdist_wininst on Linux (was Distutils changes - end
 user requirements (Was: Deprecate MANIFEST.in))
In-Reply-To: <20090409202711.E750C3A4063@sparrow.telecommunity.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>	<grlans$h9q$2@ger.gmane.org>	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>	<grlk90$j5v$1@ger.gmane.org>
	<20090409202711.E750C3A4063@sparrow.telecommunity.com>
Message-ID: <49DE609E.7040000@astraw.com>

P.J. Eby wrote:
> At 04:02 PM 4/9/2009 -0400, Tres Seaver wrote:
>> Warning: Can't read registry to find the necessary compiler setting
>> Make sure that Python modules _winreg, win32api or win32con are
>> installed.
>> error:
>> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
>>
>> No such file or directory
>>
>> Note that this package is pure python.
> 
> Looks like somebody broke it in 2.6 then...  I build the 2.3-2.5
> bdist_wininst installers for setuptools on a Linux box.
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


I just reconfigured and remade a clean Python 2.5.4 and also get a
failure. What did you need to get this to work with 2.3-2.5?

My setup.py is:

from distutils.core import setup
setup(name="simplepack",
      version="1.0",
      author="Andrew Straw <strawman at astraw.com>",
      packages = ['simplepack']
     )

And the other files are:

./simplepack/__init__.py (empty)
./simplepack/simplepack.py (has a single print statement)

The error I got was:

$ ~/py2.5/bin/python setup.py bdist_wininst
running bdist_wininst
running build
running build_py
creating build
creating build/lib
creating build/lib/simplepack
copying simplepack/__init__.py -> build/lib/simplepack
copying simplepack/simplepack.py -> build/lib/simplepack
installing to build/bdist.linux-x86_64/wininst
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wininst
creating build/bdist.linux-x86_64/wininst/PURELIB
creating build/bdist.linux-x86_64/wininst/PURELIB/simplepack
copying build/lib/simplepack/__init__.py ->
build/bdist.linux-x86_64/wininst/PURELIB/simplepack
copying build/lib/simplepack/simplepack.py ->
build/bdist.linux-x86_64/wininst/PURELIB/simplepack
running install_egg_info
Writing
build/bdist.linux-x86_64/wininst/PURELIB/simplepack-1.0-py2.5.egg-info
creating '/tmp/tmpUytyOq.zip' and adding '.' to it
adding 'PURELIB/simplepack-1.0-py2.5.egg-info'
adding 'PURELIB/simplepack/__init__.py'
adding 'PURELIB/simplepack/simplepack.py'
creating dist
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules _winreg, win32api or win32con are installed.
removing 'build/bdist.linux-x86_64/wininst' (and everything under it)


From ziade.tarek at gmail.com  Thu Apr  9 22:55:53 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 22:55:53 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904091352v66834126kefcea89bebe026bb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
	<79990c6b0904091352v66834126kefcea89bebe026bb@mail.gmail.com>
Message-ID: <94bdd2610904091355w99656e4p76081f6613eb4893@mail.gmail.com>

On Thu, Apr 9, 2009 at 10:52 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/4/9 Tres Seaver <tseaver at palladion.com>:
>> creating dist
>> Warning: Can't read registry to find the necessary compiler setting
>> Make sure that Python modules _winreg, win32api or win32con are installed.
>
> This is a warning, so can be ignored.
>
>> error:
>> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
>> No such file or directory
>
> This is a bug. That "ux-i686" part of the filename looks wrong.
>
> Yes, it's a bug in bdist_wininst, introduced in revision 62197, Mon
> Apr 7 01:53:39 2008 UTC.
>
> If I file a bug and it gets fixed, will you build installers for me? :-)
>
> (Seriously, thanks for trying this and locating the bug. I've raised
> it as http://bugs.python.org/issue5731).

Great thanks, I'll add a test and fix it right now, so this discussion
can continue, and distutils trunk will
get some beta testers ;)


>
> Paul.
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From zooko at zooko.com  Thu Apr  9 22:15:48 2009
From: zooko at zooko.com (zooko)
Date: Thu, 9 Apr 2009 14:15:48 -0600
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090237q1b5f1c62le9039d95203362d5@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<42656BBB-226F-4732-BB1A-BBCB1830F435@zooko.com>
	<79990c6b0904090237q1b5f1c62le9039d95203362d5@mail.gmail.com>
Message-ID: <A54AB186-AE58-494F-B1C2-1B4E50099E01@zooko.com>

On Apr 9, 2009, at 3:37 AM, Paul Moore wrote:
>>
>> I personally, as a consumer of other people's software, prefer to  
>> acquire their packages as sdist .tar.gz's or as .eggs on all  
>> platforms, including Windows, which I use regularly.  So your and  
>> my preferences as a consumer of packages differ on this.
...
>
> If I have to guess at your reasons, my assumption would be:
> - Consistent means of installing on all platforms

This is a big one.  Another is that try to use command-line  
interfaces instead of GUIs whenever possible.  I can, for example,  
script installation and run the same script on all my platforms, or  
schedule the script on all my platforms using buildbot.  Another is  
that I can uninstall with "rm", which I greatly prefer over the  
Windows Add/Remove programs interface.  Another is that if I  
easy_install a *different* package which requires that package, then  
the .egg can be automatically installed with no manual effort on my  
part.

Regards,

Zooko

From ziade.tarek at gmail.com  Thu Apr  9 23:49:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 9 Apr 2009 23:49:44 +0200
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <94bdd2610904091355w99656e4p76081f6613eb4893@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
	<79990c6b0904091352v66834126kefcea89bebe026bb@mail.gmail.com>
	<94bdd2610904091355w99656e4p76081f6613eb4893@mail.gmail.com>
Message-ID: <94bdd2610904091449j3ddda027r2ce14d3cfba2f5e0@mail.gmail.com>

>
> If I file a bug and it gets fixed, will you build installers for me? :-)
>
> (Seriously, thanks for trying this and locating the bug. I've raised
> it as http://bugs.python.org/issue5731).

fixed in the trunk, r71413,

I am now backporting it in the 2.6 maintenance branch, but I'd
appreciate any feeback
on Distutils current trunk

Cheers
Tarek

From pje at telecommunity.com  Fri Apr 10 00:06:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 09 Apr 2009 18:06:02 -0400
Subject: [Distutils] bdist_wininst on Linux (was Distutils changes - end
 user requirements (Was: Deprecate MANIFEST.in))
In-Reply-To: <49DE609E.7040000@astraw.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<grlans$h9q$2@ger.gmane.org>
	<79990c6b0904091245x4cd9e354t67a526609c7ffae9@mail.gmail.com>
	<grlk90$j5v$1@ger.gmane.org>
	<20090409202711.E750C3A4063@sparrow.telecommunity.com>
	<49DE609E.7040000@astraw.com>
Message-ID: <20090409220334.890943A4063@sparrow.telecommunity.com>

At 01:54 PM 4/9/2009 -0700, Andrew Straw wrote:
>P.J. Eby wrote:
> > At 04:02 PM 4/9/2009 -0400, Tres Seaver wrote:
> >> Warning: Can't read registry to find the necessary compiler setting
> >> Make sure that Python modules _winreg, win32api or win32con are
> >> installed.
> >> error:
> >> 
> /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe:
> >>
> >> No such file or directory
> >>
> >> Note that this package is pure python.
> >
> > Looks like somebody broke it in 2.6 then...  I build the 2.3-2.5
> > bdist_wininst installers for setuptools on a Linux box.
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > http://mail.python.org/mailman/listinfo/distutils-sig
>
>
>I just reconfigured and remade a clean Python 2.5.4 and also get a
>failure. What did you need to get this to work with 2.3-2.5?

Nothing.  I get the same warning message, but the executable gets 
built and is found in the dist/ directory.  Notice that Tres's error 
message is for the absence of a 'wininst-6.0ux-i686.exe'...  which 
suggests that it may simply be a problem with his local Python 
installation, rather than a 2.6-specific isssue.


>My setup.py is:
>
>from distutils.core import setup
>setup(name="simplepack",
>       version="1.0",
>       author="Andrew Straw <strawman at astraw.com>",
>       packages = ['simplepack']
>      )
>
>And the other files are:
>
>./simplepack/__init__.py (empty)
>./simplepack/simplepack.py (has a single print statement)
>
>The error I got was:
>
>$ ~/py2.5/bin/python setup.py bdist_wininst
>running bdist_wininst
>running build
>running build_py
>creating build
>creating build/lib
>creating build/lib/simplepack
>copying simplepack/__init__.py -> build/lib/simplepack
>copying simplepack/simplepack.py -> build/lib/simplepack
>installing to build/bdist.linux-x86_64/wininst
>running install_lib
>creating build/bdist.linux-x86_64
>creating build/bdist.linux-x86_64/wininst
>creating build/bdist.linux-x86_64/wininst/PURELIB
>creating build/bdist.linux-x86_64/wininst/PURELIB/simplepack
>copying build/lib/simplepack/__init__.py ->
>build/bdist.linux-x86_64/wininst/PURELIB/simplepack
>copying build/lib/simplepack/simplepack.py ->
>build/bdist.linux-x86_64/wininst/PURELIB/simplepack
>running install_egg_info
>Writing
>build/bdist.linux-x86_64/wininst/PURELIB/simplepack-1.0-py2.5.egg-info
>creating '/tmp/tmpUytyOq.zip' and adding '.' to it
>adding 'PURELIB/simplepack-1.0-py2.5.egg-info'
>adding 'PURELIB/simplepack/__init__.py'
>adding 'PURELIB/simplepack/simplepack.py'
>creating dist
>Warning: Can't read registry to find the necessary compiler setting
>Make sure that Python modules _winreg, win32api or win32con are installed.
>removing 'build/bdist.linux-x86_64/wininst' (and everything under it)


From marius at pov.lt  Fri Apr 10 00:19:52 2009
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 10 Apr 2009 01:19:52 +0300
Subject: [Distutils] Distutils changes - end user requirements
	(Was:	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
Message-ID: <20090409221952.GD16762@fridge.pov.lt>

On Thu, Apr 09, 2009 at 10:24:18AM +0100, Paul Moore wrote:
> 2009/4/9 Lennart Regebro <regebro at gmail.com>:
> > On Wed, Apr 8, 2009 at 12:27, Paul Moore <p.f.moore at gmail.com> wrote:
> >> 1. (Meta-requirement) I want to be able to download a Windows
> >> installer[1] for *every* package I need.
> >
> > I would like to know why you have this requirement. Some sort of
> > Windows binary, absolutely. But an installer? Would a package
> > management GUI be enough? A Windows software that can install (and
> > remove) eggs?
> 
> Because I like to see my installed Python packages in Windows'
> "Add/Remove Programs" list.

But that list is for *programs*, not *libraries*.

I'd understand a wish that a Python app (that also depends on twenty
other Python libraries) be bundled as a single Windows installer (that
contains all those twenty libraries inside) for an easy
next->next->next->finish installation.

I don't understand a desire of downloading twenty .exe files and
clicking on them all to get an app running.  Did I misunderstand
something?

Marius Gedminas
-- 
Added mysterious, undocumented --scanflags and --fuzzy options.
        -- nmap 3.0 announcement
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090410/52e23a1f/attachment.pgp>

From jim at zope.com  Fri Apr 10 00:25:06 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 9 Apr 2009 18:25:06 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <49DE2889.8070807@palladion.com>
References: <49DE2889.8070807@palladion.com>
Message-ID: <74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>


On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
...
> I have backed off on the notion of overloading 'Requires:' /  
> 'Provides:'
> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
> 'Obsoletes-Dist'.
>
> "Stock" distutils should probably spell the arguments to
> distutils.core.setup predictably:  'requires_dist', 'provides_dist',
> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
> undeprecated alias for 'requires_dist'.


What is the rational for this?  I'd strongly prefer the "requires"  
argument name to be compatible with setuptools.  Otherwise, I think  
we'll introduce needless confusion.

Jim

--
Jim Fulton
Zope Corporation



From tseaver at palladion.com  Fri Apr 10 00:40:22 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 09 Apr 2009 18:40:22 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
Message-ID: <49DE7956.6080507@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
> ...
>> I have backed off on the notion of overloading 'Requires:' /  
>> 'Provides:'
>> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
>> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
>> 'Obsoletes-Dist'.
>>
>> "Stock" distutils should probably spell the arguments to
>> distutils.core.setup predictably:  'requires_dist', 'provides_dist',
>> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
>> undeprecated alias for 'requires_dist'.
> 
> 
> What is the rational for this?  I'd strongly prefer the "requires"  
> argument name to be compatible with setuptools.  Otherwise, I think  
> we'll introduce needless confusion.

I'm aiming for self-consistency within the 'PKG-INFO' field names:

 - 'Requires'
 - 'Requires-Python'
 - 'Requires-External'

The 'Obsoletes' and 'Provides' fields also need
distutils-project-oriented versions, so picking a suffix ('-Dist') which
matched for them seemed cleanest.

Add that to the fact that setuptools has no way (yet) to spell
'provides' or 'obsoletes', and it seemed to me clearer to just make
setuptools current argument an alias for the "consistent" version to be
landed in distutils.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3nlW+gerLs4ltQ4RAqnHAKCn033qJCe7iuV0uzBWVXlnAQiqfwCgzX2F
L0rqU3Z7jC4ftYrSHnToLH4=
=umR3
-----END PGP SIGNATURE-----


From greg.ewing at canterbury.ac.nz  Fri Apr 10 02:03:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Apr 2009 12:03:29 +1200
Subject: [Distutils] Distutils changes - end user requirements (Was:
 Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090237q1b5f1c62le9039d95203362d5@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<42656BBB-226F-4732-BB1A-BBCB1830F435@zooko.com>
	<79990c6b0904090237q1b5f1c62le9039d95203362d5@mail.gmail.com>
Message-ID: <49DE8CD1.6060806@canterbury.ac.nz>

Paul Moore wrote:

> If I have to guess at your reasons, my assumption would be:
> - Consistent means of installing on all platforms
> - ???

I can imagine that people for whom Windows is not their
primary platform might like to be able to install Python
packages the same way they do on other platforms. There's
less mental gear switching involved that way when moving
from one platform to another.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Apr 10 02:12:33 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Apr 2009 12:12:33 +1200
Subject: [Distutils] Distutils changes - end user requirements (Was:
 Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
Message-ID: <49DE8EF1.4090808@canterbury.ac.nz>

Paul Moore wrote:

> Personally, I'd be happy if every package that currently distributes
> any form of Windows binaries, distributed a Windows installer. ...
 > I agree that
> expecting *everything* to have a Windows installer is unreasonable.

Would it be feasible to have a generic installer for
.egg files that does whatever a real Windows installer
would have done? Distribute it with Python and make it
launch when you double-click a .egg file.

-- 
Greg

From david at ar.media.kyoto-u.ac.jp  Fri Apr 10 06:54:16 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 10 Apr 2009 13:54:16 +0900
Subject: [Distutils] Distutils changes - end user
 requirements	(Was:	Deprecate MANIFEST.in)
In-Reply-To: <20090409221952.GD16762@fridge.pov.lt>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<20090409221952.GD16762@fridge.pov.lt>
Message-ID: <49DED0F8.1030602@ar.media.kyoto-u.ac.jp>

Marius Gedminas wrote:
>
> But that list is for *programs*, not *libraries*.
>   

The difference starts to be blurry for python (is numpy an application
or a library ?), and it does not make much sense for windows users, I think.

> I don't understand a desire of downloading twenty .exe files and
> clicking on them all to get an app running.  Did I misunderstand
> something?
>   

I think it is related to several issues:
    - no uninstall with eggs
    - .exe is a known thing for any windows user, .egg isn't
    - you can't easily get the dependencies, download them separately,
and install this on another computer

One thing which would be nice too is something to bundle several python
packages together, a bit like pip freeze does, but to build one single
installer.

For all those reasons, having a commonly accepted, low-level set of data
that can be fed/obtained from any tool is crucial. I don't see any other
way to allow different tools to be built reliably, depending on the
platform and/or goal.

cheers,

David

From a.cavallo at acm.org  Fri Apr 10 11:21:59 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Fri, 10 Apr 2009 10:21:59 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <49DE526E.6030400@activestate.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<319e029f0904090308u2259dd65na9113b1a339fd114@mail.gmail.com>
	<79990c6b0904090330g7cfd0471u82a15e48221785ae@mail.gmail.com>
	<319e029f0904090445h15cf6c95m16d73952aa70bc2a@mail.gmail.com>
	<79990c6b0904090618m7f947073n49320b8b9fb41f3a@mail.gmail.com>
	<49DE526E.6030400@activestate.com>
Message-ID: <a71153f0904100221l3971b34ag6e049355ba2ff5d2@mail.gmail.com>

Hi,
I've been working on something similar with various approaches since a
long while.

I've finally got someting running on
http://download.opensuse.org/repositories/home:/cavallo71:/python-opt

It compiles on across different distribution but at the moment the
suse rpmlint checks are
too stringent and the compile fails (actually it works fine but it
doesn't release the packages).

Basically what I needed was (still is) a rpm integrated package to
install on any linux distribution
with a precise set of requirements:

1. The administrator should see the distro installed python (transparency)
2. The new "interpreter" should set on request the environment (explicit)
3. The installation should leverage the rpm infrastructure
(consistency in a running system)

These requirements are mandatory in order to garantee a smooth operation
on across systems.
Moreover a user shoudl use the distuils tools to install additional packages
without any "magic" (so python setup.py bdist_rpm should generate a
working installable package).

Basically to "activate" the new python interpreter all a user have to do is
typing something like:
$> . /opt/cavallo-python-2.6.1/cavallo-python-env.sh

(yes the name has to be changed at some point;))

Let me know if this sounds a good idea,

Regards,
Antonio Cavallo





I'm packaging python for linux in a separate deirectory and I needed
to be completely "system" trasnparent,

On Thu, Apr 9, 2009 at 8:54 PM, Trent Mick <trentm at activestate.com> wrote:
> Paul Moore wrote:
>>
>> On that note, didn't ActiveState distribute their version of Python
>> with a package manager (PPM)? As far as I know, that was only ever
>> supported by ActiveState themselves, no-one else ever built PPM
>> packages for their extensions, and it's been quietly dropped in recent
>> versions (Google tells me that it vanished around Python 2.3).
>
> Yes, it was there a long time ago. It was dropped around ActivePython 2.3,
> as you say, because it wasn't well architected and, at the time, we didn't
> have the resources to maintain it.
>
> However, we've just recently hired a full time employee to (in part) start
> looking at a "PyPM" package manager again -- this time with an appropriate
> architecture. The hope is to have something to show in the fall (perhaps
> sooner).
>
> Trent
>
> --
> Trent Mick
> trentm at activestate.com
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From jim at zope.com  Fri Apr 10 14:30:33 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 10 Apr 2009 08:30:33 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <49DE7956.6080507@palladion.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
Message-ID: <B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>


On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jim Fulton wrote:
>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>> ...
>>> I have backed off on the notion of overloading 'Requires:' /
>>> 'Provides:'
>>> / 'Obsoletes:', following Jim's notion of deprecating them in  
>>> favor of
>>> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
>>> 'Obsoletes-Dist'.
>>>
>>> "Stock" distutils should probably spell the arguments to
>>> distutils.core.setup predictably:  'requires_dist', 'provides_dist',
>>> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
>>> undeprecated alias for 'requires_dist'.
>>
>>
>> What is the rational for this?  I'd strongly prefer the "requires"
>> argument name to be compatible with setuptools.  Otherwise, I think
>> we'll introduce needless confusion.
>
> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>
> - 'Requires'
> - 'Requires-Python'
> - 'Requires-External'
>
> The 'Obsoletes' and 'Provides' fields also need
> distutils-project-oriented versions, so picking a suffix ('-Dist')  
> which
> matched for them seemed cleanest.
>
> Add that to the fact that setuptools has no way (yet) to spell
> 'provides' or 'obsoletes', and it seemed to me clearer to just make
> setuptools current argument an alias for the "consistent" version to  
> be
> landed in distutils.


I get that. In fact, I already got that. :) I think backward  
compatibility with existing wide usage is more important and not  
incompatible.

So then I suggest we change the field and option names to (adjusting  
capitalization as necessary): install_requires, install_provides, and  
install_obsoletes.

I'm a very strong -1 to requires_dist.

Jim

--
Jim Fulton
Zope Corporation



From tseaver at palladion.com  Fri Apr 10 15:19:13 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 10 Apr 2009 09:19:13 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
Message-ID: <grnh0o$tnh$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Jim Fulton wrote:
>>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>> ...
>>>> I have backed off on the notion of overloading 'Requires:' /
>>>> 'Provides:'
>>>> / 'Obsoletes:', following Jim's notion of deprecating them in  
>>>> favor of
>>>> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
>>>> 'Obsoletes-Dist'.
>>>>
>>>> "Stock" distutils should probably spell the arguments to
>>>> distutils.core.setup predictably:  'requires_dist', 'provides_dist',
>>>> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
>>>> undeprecated alias for 'requires_dist'.
>>>
>>> What is the rational for this?  I'd strongly prefer the "requires"
>>> argument name to be compatible with setuptools.  Otherwise, I think
>>> we'll introduce needless confusion.
>> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>>
>> - 'Requires'
>> - 'Requires-Python'
>> - 'Requires-External'
>>
>> The 'Obsoletes' and 'Provides' fields also need
>> distutils-project-oriented versions, so picking a suffix ('-Dist')  
>> which
>> matched for them seemed cleanest.
>>
>> Add that to the fact that setuptools has no way (yet) to spell
>> 'provides' or 'obsoletes', and it seemed to me clearer to just make
>> setuptools current argument an alias for the "consistent" version to  
>> be
>> landed in distutils.
> 
> 
> I get that. In fact, I already got that. :) I think backward  
> compatibility with existing wide usage is more important and not  
> incompatible.
> 
> So then I suggest we change the field and option names to (adjusting  
> capitalization as necessary): install_requires, install_provides, and  
> install_obsoletes.
> 
> I'm a very strong -1 to requires_dist.

I think we need to distinguish the spelling of the arguments passed to
setup() from the fields written into PKG-INFO.  At the moment, there is
a very widespread usage of 'install_requires' among setuptools-based
packages.  How that argument maps onto PKG-INFO fields (which is what
PEP 345 is really about) is open to some debate.

I think consistency *within* PKG-INFO fieldnames is more important than
consistency between setup arguments and the corresponding fieldnames:
for instance, distutils.core.setup already defiens a 'url' argument,
which maps onto 'Homepage-URL' in PKG-INFO.

We could use the names you proposed for arguments to 'setup()', and
still use the names I proposed for PKG-INFO.  There is one more new
argument to setup() we should consider, which might be spelled
'build_requires', and which would map onto the 'Requires-External'
PKG-INFO field.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ30dR+gerLs4ltQ4RAhYHAKDY971Z0WOVGWMWUaGML42ByTNguwCfb5gX
a6/EZb+J/4T+XgtMBSoCgLs=
=rhm+
-----END PGP SIGNATURE-----


From marius at pov.lt  Fri Apr 10 15:48:25 2009
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 10 Apr 2009 16:48:25 +0300
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <grnh0o$tnh$1@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<grnh0o$tnh$1@ger.gmane.org>
Message-ID: <20090410134825.GA14535@fridge.pov.lt>

On Fri, Apr 10, 2009 at 09:19:13AM -0400, Tres Seaver wrote:
> There is one more new
> argument to setup() we should consider, which might be spelled
> 'build_requires', and which would map onto the 'Requires-External'
> PKG-INFO field.

Setuptools has a 'setup_requires' argument to setup().  I don't know if
the semantics of that one are similar to your proposed 'build_requires'.

Marius Gedminas
-- 
Truth does not demand belief. Scientists do not join hands every Sunday,
singing, 'Yes, gravity is real! I will have faith! I will be strong! I believe
in my heart that what goes up, up, up must come down, down, down. Amen!' If
they did, we would think they were pretty insecure about it. - Dan Barker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090410/7b533fb5/attachment.pgp>

From tseaver at palladion.com  Fri Apr 10 15:57:51 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 10 Apr 2009 09:57:51 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <20090410134825.GA14535@fridge.pov.lt>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<grnh0o$tnh$1@ger.gmane.org>
	<20090410134825.GA14535@fridge.pov.lt>
Message-ID: <grnj95$42k$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marius Gedminas wrote:
> On Fri, Apr 10, 2009 at 09:19:13AM -0400, Tres Seaver wrote:
>> There is one more new
>> argument to setup() we should consider, which might be spelled
>> 'build_requires', and which would map onto the 'Requires-External'
>> PKG-INFO field.
> 
> Setuptools has a 'setup_requires' argument to setup().  I don't know if
> the semantics of that one are similar to your proposed 'build_requires'.

Nope:  'build_requires' is something Matthias Klose discussed as being
useful for downstream packagers:  it doesn't refer to Python
distributions, but to external library / tool requriements.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ31Bf+gerLs4ltQ4RAmduAJwPAQftvIrNdSkImZgG469WJ45dGwCgm+Aj
4BXZ6Jo4HZDSlETKsYxrPAM=
=Jqm3
-----END PGP SIGNATURE-----


From david at ar.media.kyoto-u.ac.jp  Fri Apr 10 15:49:36 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 10 Apr 2009 22:49:36 +0900
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <grnj95$42k$1@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<grnh0o$tnh$1@ger.gmane.org>	<20090410134825.GA14535@fridge.pov.lt>
	<grnj95$42k$1@ger.gmane.org>
Message-ID: <49DF4E70.8010808@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> Marius Gedminas wrote:
> > On Fri, Apr 10, 2009 at 09:19:13AM -0400, Tres Seaver wrote:
> >> There is one more new
> >> argument to setup() we should consider, which might be spelled
> >> 'build_requires', and which would map onto the 'Requires-External'
> >> PKG-INFO field.
> > Setuptools has a 'setup_requires' argument to setup().  I don't know if
> > the semantics of that one are similar to your proposed 'build_requires'.
>
> Nope:  'build_requires' is something Matthias Klose discussed as being
> useful for downstream packagers:  it doesn't refer to Python
> distributions, but to external library / tool requriements.

It could still be useful to python as well, for a potential future tool.
That's very useful for bootstrapping packages, for example, so it is not
just for downstream packagers.

@Marius: build-requires spells out the dependencies necessary to *build*
the concerned package, but which are not necessary to *use* the package.
For example, a python package foo requires paver to be built correctly,
but you can use foo in your own code without paver.

cheers,

David

From jim at zope.com  Fri Apr 10 16:27:02 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 10 Apr 2009 10:27:02 -0400
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <grnh0o$tnh$1@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<grnh0o$tnh$1@ger.gmane.org>
Message-ID: <7D632AC3-0C2F-4612-B2C2-1C4C94BDFE34@zope.com>


On Apr 10, 2009, at 9:19 AM, Tres Seaver wrote:
...
> We could use the names you proposed for arguments to 'setup()', and
> still use the names I proposed for PKG-INFO.

I'm fine with that. I mainly care about not renaming  
install_requires. :)

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Fri Apr 10 16:30:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 10 Apr 2009 16:30:30 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
Message-ID: <94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>

On Fri, Apr 10, 2009 at 2:30 PM, Jim Fulton <jim at zope.com> wrote:
>
> On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Jim Fulton wrote:
>>>
>>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>> ...
>>>>
>>>> I have backed off on the notion of overloading 'Requires:' /
>>>> 'Provides:'
>>>> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
>>>> new fields. ?I named them 'Requires-Dist:', 'Provides-Dist:', and
>>>> 'Obsoletes-Dist'.
>>>>
>>>> "Stock" distutils should probably spell the arguments to
>>>> distutils.core.setup predictably: ?'requires_dist', 'provides_dist',
>>>> 'obsoletes_dist'. ?setuptools can treat 'install_requires' as an
>>>> undeprecated alias for 'requires_dist'.
>>>
>>>
>>> What is the rational for this? ?I'd strongly prefer the "requires"
>>> argument name to be compatible with setuptools. ?Otherwise, I think
>>> we'll introduce needless confusion.
>>
>> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>>
>> - 'Requires'
>> - 'Requires-Python'
>> - 'Requires-External'
>>
>> The 'Obsoletes' and 'Provides' fields also need
>> distutils-project-oriented versions, so picking a suffix ('-Dist') which
>> matched for them seemed cleanest.
>>
>> Add that to the fact that setuptools has no way (yet) to spell
>> 'provides' or 'obsoletes', and it seemed to me clearer to just make
>> setuptools current argument an alias for the "consistent" version to be
>> landed in distutils.
>
>
> I get that. In fact, I already got that. :) I think backward compatibility
> with existing wide usage is more important and not incompatible.

we could also support both spellings for one version, and deprecate
the old name with a warning,

Tarek

From jim at zope.com  Fri Apr 10 16:33:25 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 10 Apr 2009 10:33:25 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
Message-ID: <4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>


On Apr 10, 2009, at 10:30 AM, Tarek Ziad? wrote:

> On Fri, Apr 10, 2009 at 2:30 PM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Jim Fulton wrote:
>>>>
>>>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>>> ...
>>>>>
>>>>> I have backed off on the notion of overloading 'Requires:' /
>>>>> 'Provides:'
>>>>> / 'Obsoletes:', following Jim's notion of deprecating them in  
>>>>> favor of
>>>>> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:', and
>>>>> 'Obsoletes-Dist'.
>>>>>
>>>>> "Stock" distutils should probably spell the arguments to
>>>>> distutils.core.setup predictably:  'requires_dist',  
>>>>> 'provides_dist',
>>>>> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
>>>>> undeprecated alias for 'requires_dist'.
>>>>
>>>>
>>>> What is the rational for this?  I'd strongly prefer the "requires"
>>>> argument name to be compatible with setuptools.  Otherwise, I think
>>>> we'll introduce needless confusion.
>>>
>>> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>>>
>>> - 'Requires'
>>> - 'Requires-Python'
>>> - 'Requires-External'
>>>
>>> The 'Obsoletes' and 'Provides' fields also need
>>> distutils-project-oriented versions, so picking a suffix ('-Dist')  
>>> which
>>> matched for them seemed cleanest.
>>>
>>> Add that to the fact that setuptools has no way (yet) to spell
>>> 'provides' or 'obsoletes', and it seemed to me clearer to just make
>>> setuptools current argument an alias for the "consistent" version  
>>> to be
>>> landed in distutils.
>>
>>
>> I get that. In fact, I already got that. :) I think backward  
>> compatibility
>> with existing wide usage is more important and not incompatible.
>
> we could also support both spellings for one version, and deprecate
> the old name with a warning,


Strong: -1.

Why change the name? A different name isn't going to be better enough  
to be worth the hassle. Deprecation is waaaay overrated as a tool for  
reducing the pain of making people change their code or habits.

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Fri Apr 10 16:40:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 10 Apr 2009 16:40:16 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
Message-ID: <94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>

On Fri, Apr 10, 2009 at 4:33 PM, Jim Fulton <jim at zope.com> wrote:
>
> On Apr 10, 2009, at 10:30 AM, Tarek Ziad? wrote:
>
>> On Fri, Apr 10, 2009 at 2:30 PM, Jim Fulton <jim at zope.com> wrote:
>>>
>>> On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:
>>>
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>> Jim Fulton wrote:
>>>>>
>>>>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>>>> ...
>>>>>>
>>>>>> I have backed off on the notion of overloading 'Requires:' /
>>>>>> 'Provides:'
>>>>>> / 'Obsoletes:', following Jim's notion of deprecating them in favor of
>>>>>> new fields. ?I named them 'Requires-Dist:', 'Provides-Dist:', and
>>>>>> 'Obsoletes-Dist'.
>>>>>>
>>>>>> "Stock" distutils should probably spell the arguments to
>>>>>> distutils.core.setup predictably: ?'requires_dist', 'provides_dist',
>>>>>> 'obsoletes_dist'. ?setuptools can treat 'install_requires' as an
>>>>>> undeprecated alias for 'requires_dist'.
>>>>>
>>>>>
>>>>> What is the rational for this? ?I'd strongly prefer the "requires"
>>>>> argument name to be compatible with setuptools. ?Otherwise, I think
>>>>> we'll introduce needless confusion.
>>>>
>>>> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>>>>
>>>> - 'Requires'
>>>> - 'Requires-Python'
>>>> - 'Requires-External'
>>>>
>>>> The 'Obsoletes' and 'Provides' fields also need
>>>> distutils-project-oriented versions, so picking a suffix ('-Dist') which
>>>> matched for them seemed cleanest.
>>>>
>>>> Add that to the fact that setuptools has no way (yet) to spell
>>>> 'provides' or 'obsoletes', and it seemed to me clearer to just make
>>>> setuptools current argument an alias for the "consistent" version to be
>>>> landed in distutils.
>>>
>>>
>>> I get that. In fact, I already got that. :) I think backward
>>> compatibility
>>> with existing wide usage is more important and not incompatible.
>>
>> we could also support both spellings for one version, and deprecate
>> the old name with a warning,
>
>
> Strong: -1.
>
> Why change the name? A different name isn't going to be better enough to be
> worth the hassle. Deprecation is waaaay overrated as a tool for reducing the
> pain of making people change their code or habits.

I don't think it's a good idea to have a different name in PKG-INFO
and in the arguments to describe
the same element. we should have the same name everywhere for
consistency at the end.

I don't see anything wrong about adding a simple deprecation warning
here, It won't happen again
for quite a while.

Distutils already had some argument name changes like that for
backward compatiblity (licence<->license) but they were not
displaying any warning.

OTHO I'd be fine if the current setuptool name is used in PKG-INFO

From jim at zope.com  Fri Apr 10 16:47:08 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 10 Apr 2009 10:47:08 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
Message-ID: <2C9BE938-2E49-4217-BA27-4E0D2DD3EB2C@zope.com>


On Apr 10, 2009, at 10:40 AM, Tarek Ziad? wrote:

> On Fri, Apr 10, 2009 at 4:33 PM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Apr 10, 2009, at 10:30 AM, Tarek Ziad? wrote:
>>
>>> On Fri, Apr 10, 2009 at 2:30 PM, Jim Fulton <jim at zope.com> wrote:
>>>>
>>>> On Apr 9, 2009, at 6:40 PM, Tres Seaver wrote:
>>>>
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA1
>>>>>
>>>>> Jim Fulton wrote:
>>>>>>
>>>>>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>>>>> ...
>>>>>>>
>>>>>>> I have backed off on the notion of overloading 'Requires:' /
>>>>>>> 'Provides:'
>>>>>>> / 'Obsoletes:', following Jim's notion of deprecating them in  
>>>>>>> favor of
>>>>>>> new fields.  I named them 'Requires-Dist:', 'Provides-Dist:',  
>>>>>>> and
>>>>>>> 'Obsoletes-Dist'.
>>>>>>>
>>>>>>> "Stock" distutils should probably spell the arguments to
>>>>>>> distutils.core.setup predictably:  'requires_dist',  
>>>>>>> 'provides_dist',
>>>>>>> 'obsoletes_dist'.  setuptools can treat 'install_requires' as an
>>>>>>> undeprecated alias for 'requires_dist'.
>>>>>>
>>>>>>
>>>>>> What is the rational for this?  I'd strongly prefer the  
>>>>>> "requires"
>>>>>> argument name to be compatible with setuptools.  Otherwise, I  
>>>>>> think
>>>>>> we'll introduce needless confusion.
>>>>>
>>>>> I'm aiming for self-consistency within the 'PKG-INFO' field names:
>>>>>
>>>>> - 'Requires'
>>>>> - 'Requires-Python'
>>>>> - 'Requires-External'
>>>>>
>>>>> The 'Obsoletes' and 'Provides' fields also need
>>>>> distutils-project-oriented versions, so picking a suffix ('- 
>>>>> Dist') which
>>>>> matched for them seemed cleanest.
>>>>>
>>>>> Add that to the fact that setuptools has no way (yet) to spell
>>>>> 'provides' or 'obsoletes', and it seemed to me clearer to just  
>>>>> make
>>>>> setuptools current argument an alias for the "consistent"  
>>>>> version to be
>>>>> landed in distutils.
>>>>
>>>>
>>>> I get that. In fact, I already got that. :) I think backward
>>>> compatibility
>>>> with existing wide usage is more important and not incompatible.
>>>
>>> we could also support both spellings for one version, and deprecate
>>> the old name with a warning,
>>
>>
>> Strong: -1.
>>
>> Why change the name? A different name isn't going to be better  
>> enough to be
>> worth the hassle. Deprecation is waaaay overrated as a tool for  
>> reducing the
>> pain of making people change their code or habits.
>
> I don't think it's a good idea to have a different name in PKG-INFO
> and in the arguments to describe
> the same element. we should have the same name everywhere for
> consistency at the end.

Argue with Tres. :) He gave an example where setup argument names and  
meta-data field names were different.

> I don't see anything wrong about adding a simple deprecation warning
> here, It won't happen again
> for quite a while.

We disagree then. Too often people use deprecation as a way to argue  
that a change isn't too painful. Sometimes, change is necessary and  
deprecation helps to manage the change. The change we're talking about  
isn't necessary.

...

> OTHO I'd be fine if the current setuptool name is used in PKG-INFO


Me too. :)

Jim

--
Jim Fulton
Zope Corporation



From zooko at zooko.com  Fri Apr 10 17:02:01 2009
From: zooko at zooko.com (zooko)
Date: Fri, 10 Apr 2009 09:02:01 -0600
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
Message-ID: <6DA14050-CB85-4E97-9349-DD15BE24137B@zooko.com>

> Can any packager step up and explain why they execute "python  
> setup.py bdist_egg" rather than "python setup.py bdist_wininst"  
> when creating distributions for their Windows users?

For starters, the former works on all my development platforms.  I  
can script buildbot to do it the same on all platforms, for example.   
The latter I haven't tried.  Just a minute...  <Zooko tries "python  
setup.py bdist_wininst".>   (See appendix A.)

Secondly, the package can be automatically installed when the user  
easy_installs a *different* package which depends on my package.

The Tahoe project currently has fourteen such dependencies.  The  
Tahoe install process works on all platforms including Windows:

http://allmydata.org/source/tahoe/trunk/docs/install.html

It isn't perfect, but I think it is pretty cool that the same install  
document works for both Windows users and everyone else.

Thirdly, the entry_points mechanism for making executable scripts  
means I get executable scripts on all platforms with no extra effort  
on my part.

I'm not saying that this is perfect, or that users don't want other  
things, or that developers shouldn't want other things.  For Tahoe in  
specific, I really would like a normal installable package thing like  
Windows users expect for their apps.  But until someone offers to  
spend their time crafting such a thing for Tahoe (ideally by using  
the bbfreeze tool [1] to automatically generate one from the Tahoe  
source distribution), I'm happier with what I've got now than with  
the alternative, which is a long list of around sixteen -- sixteen!  
-- different packages that the user would have to manually install  
before they can install Tahoe, where each of them might have  
different special install instructions on any of a dozen different  
supported platforms.

Again, the setuptools-based system that Tahoe has now is not  
*alternative* to Windows-specific installers or Debian-specific apt- 
get'able packages.  In fact, hopefully they will become increasingly  
*complementary* as tools like stdeb and bbfreeze mature.

Regards,

Zooko

[1] http://allmydata.org/trac/tahoe/ticket/585 # make it work with  
bbfreeze

Appendix A:
Let's see, "python setup.py bdist_wininst" produced a file which I  
then uploaded to my web server: http://zooko.com/allmydata- 
tahoe-1.3.0-r3833.win32.exe.  When I run that file, it first pops up  
a security warning that the producer of this file (me, in fact) is  
untrusted.  Once I hit the button labelled "Leave me alone about your  
stupid security warnings" then it enters the normal installation  
wizardy thing.  It has all my package metadata nicely displayed on  
the first page.  Cool!  Then it asks me which python to use -- it  
found 2.5 and 2.6 on my system.  I choose, then click "install", then  
"finish" and it goes away.

Now what?  Let's see, it didn't add anything to my Start Menu.  I see  
that it installed it into c:/Python26/Lib/site-packages/allmydata ,  
(which is the name of the top-level module in the allmydata-tahoe  
distribution), as well as writing a Removeallmydat-tahoe.exe and a  
allmydata-tahoe-wininst.log file.  Oh!  I see that it made a c:/ 
Python26/Scripts/tahoe.exe file!  Cool.

Okay, but I can't actually use it unless I now go and install a dozen  
or more dependencies (as mentioned in the body of this letter).   
Okay, finally, uninstall.  Hm... It isn't there in the "Add/Remove  
Programs" list.  Oh -- it is named "Python 2.6 allmydata-tahoe-1.3.0- 
r3833" instead of "allmydata-tahoe".  And it worked to remove all the  
files as far as I can tell.

It works!  Cool!  My verdict: as a consumer of other people's  
software, I wouldn't want to use a GUI tool like this if I could help  
it (it takes about 10 times as long to install a package that way as  
on the command-line, you can't script it, and you can't easily  
capture an encoding of all the options which were chosen for later  
reproduction).  I'm sure that many other users would want to install  
applications this way, i.e. things which they actually care whether  
it is installed or not, but I suspect most users do not want to  
install dependencies this way, i.e. things that are there only  
because an application requires it.  As a producer of packages I  
would be happy to use bdist_wininst since it is fully scriptable and  
I can just program my buildbot to do it for me, however I won't do so  
until the resulting installer can either automatically satisfy its  
dependencies or come with its dependencies bundled inside it.

From p.f.moore at gmail.com  Fri Apr 10 18:18:59 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Apr 2009 17:18:59 +0100
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <6DA14050-CB85-4E97-9349-DD15BE24137B@zooko.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<6DA14050-CB85-4E97-9349-DD15BE24137B@zooko.com>
Message-ID: <79990c6b0904100918k4484e459hd11bb1690723b15@mail.gmail.com>

2009/4/10 zooko <zooko at zooko.com>:
> For starters, the former works on all my development platforms. ?I can
> script buildbot to do it the same on all platforms, for example. ?The latter
> I haven't tried. ?Just a minute... ?<Zooko tries "python setup.py
> bdist_wininst".> ? (See appendix A.)

First of all, thanks for actually trying this. It's interesting to see
a non-Windows user's reaction to the process.

> Secondly, the package can be automatically installed when the user
> easy_installs a *different* package which depends on my package.

Hmm, and this is *precisely* one of the things I dislike. I don't like
anything being installed except on my request. Consider the strawman
case of a hacker managing to add a dependency to some malicious
package into a package you depend on. Or more realistically, the audit
implications of having to manually inspect all the setup files of the
transitive closure of packages your package depends on - you have a
similar problem with manual dependency handling, but at least then you
simply have to read the documentation, and can rely on the fact that
nothing gets installed that you don't explicitly request.

But that's certainly a personal view.

> The Tahoe project currently has fourteen such dependencies. ?The Tahoe
> install process works on all platforms including Windows:
>
> http://allmydata.org/source/tahoe/trunk/docs/install.html
>
> It isn't perfect, but I think it is pretty cool that the same install
> document works for both Windows users and everyone else.

And yet, you'll get Debian users who say your install process doesn't
work for them because it doesn't use apt, and doesn't register the
packages in the apt database. And Windows users who'll point out that
it doesn't work behind their corporate firewall, etc.

One person's "works" is another's "irritating failure to conform to
local standards". Preference again :-)

> Thirdly, the entry_points mechanism for making executable scripts means I
> get executable scripts on all platforms with no extra effort on my part.

I have been told that bdist_wininst installers do all that eggs do.
The entry point mechanism is one area I wonder if that's true - but
I've yet to test it myself, largely because I find the documentation
for setuptools stunningly impenetrable :-( If eggs do support
executable scripts where bdist_wininst doesn't, that is indeed a win,
as script handling is a real wart with bdist_wininst. (But please
note, I'm talking eggs vs bdist_wininst, not easy_install vs
bdist_wininst). Actually, your experiment below shows that
bdist_wininst does this fine.

> Again, the setuptools-based system that Tahoe has now is not *alternative*
> to Windows-specific installers or Debian-specific apt-get'able packages. ?In
> fact, hopefully they will become increasingly *complementary* as tools like
> stdeb and bbfreeze mature.

Agreed. But as long as developers don't distribute bdist_wininst
installers as well as eggs, they are forcing the two to be
alternatives, rather than complementary.

[...]
> It works! ?Cool!

Glad it impressed you (at least somewhat :-))

> ?My verdict: as a consumer of other people's software, I
> wouldn't want to use a GUI tool like this if I could help it (it takes about
> 10 times as long to install a package that way as on the command-line, you
> can't script it, and you can't easily capture an encoding of all the options
> which were chosen for later reproduction). ?I'm sure that many other users
> would want to install applications this way, i.e. things which they actually
> care whether it is installed or not, but I suspect most users do not want to
> install dependencies this way, i.e. things that are there only because an
> application requires it.

I'm very wary of saying what other users want, but it does fit what
*I* want. But you need to understand that I only use it for installing
packages into my system Python (for use when writing standalone
scripts, or interactive use at the interpreter prompt).

For *applications*, on Windows I would expect a py2exe-packaged
standalone application, with all its dependencies (including Python)
bundled in. It sounds like that's what Tahoe is, and as such I'd not
be trying to install it into my site Python installation. If I was a
Tahoe user, and it had installed into my Python installation, then I'd
be hugely annoyed if (say) upgrading or uninstalling Twisted caused
Tahoe to break (I don't know if Tahoe uses Twisted, one of my gripes
is that it's not clear from the Tahoe web page precisely *what* it
depends on, and hence what I'd have to be careful to avoid fiddling
with...)

> ?As a producer of packages I would be happy to use
> bdist_wininst since it is fully scriptable and I can just program my
> buildbot to do it for me, however I won't do so until the resulting
> installer can either automatically satisfy its dependencies or come with its
> dependencies bundled inside it.

Why? If your users want it and/or it's simple for you to produce *in
case* a potential users wants it, then why do you refuse to distribute
it simply because you don't like the user experience it offers?

Anyway, thanks again for actually trying all this. I think I
understand your POV a bit better now. And you may want to investigate
py2exe as a distribution option more suited to your Windows users'
expectations (but py2exe does, I believe, only run on Windows, so it's
not a cross-platform option, sadly).

Paul.

From marius at pov.lt  Fri Apr 10 18:25:59 2009
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 10 Apr 2009 19:25:59 +0300
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <49DF4E70.8010808@ar.media.kyoto-u.ac.jp>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<grnh0o$tnh$1@ger.gmane.org> <20090410134825.GA14535@fridge.pov.lt>
	<grnj95$42k$1@ger.gmane.org>
	<49DF4E70.8010808@ar.media.kyoto-u.ac.jp>
Message-ID: <20090410162559.GA29851@fridge.pov.lt>

On Fri, Apr 10, 2009 at 10:49:36PM +0900, David Cournapeau wrote:
> Tres Seaver wrote:
> > Marius Gedminas wrote:
> > > On Fri, Apr 10, 2009 at 09:19:13AM -0400, Tres Seaver wrote:
> > >> There is one more new
> > >> argument to setup() we should consider, which might be spelled
> > >> 'build_requires', and which would map onto the 'Requires-External'
> > >> PKG-INFO field.
> > > Setuptools has a 'setup_requires' argument to setup().  I don't know if
> > > the semantics of that one are similar to your proposed 'build_requires'.
> >
> > Nope:  'build_requires' is something Matthias Klose discussed as being
> > useful for downstream packagers:  it doesn't refer to Python
> > distributions, but to external library / tool requriements.
> 
> It could still be useful to python as well, for a potential future tool.
> That's very useful for bootstrapping packages, for example, so it is not
> just for downstream packagers.
> 
> @Marius: build-requires spells out the dependencies necessary to *build*
> the concerned package, but which are not necessary to *use* the package.
> For example, a python package foo requires paver to be built correctly,
> but you can use foo in your own code without paver.

And that's exactly how setup_requires differs from install_requires,
which prompted my question. ;-)

Tres's answer explains it all (except for the syntax, which I should go
and look up if I'm interested, I suppose).

Marius Gedminas
-- 
This sentence contradicts itself -- no actually it doesn't.
                -- Douglas Hofstadter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090410/28e3518b/attachment.pgp>

From doko at ubuntu.com  Fri Apr 10 18:33:57 2009
From: doko at ubuntu.com (Matthias Klose)
Date: Fri, 10 Apr 2009 18:33:57 +0200
Subject: [Distutils] getting installation location from distutils without
 changing the filesystem?
Message-ID: <49DF74F5.1080201@ubuntu.com>

The current python.m4 macro in automake (aclocal) is a bit broken, because
'${prefix}' is passed literally to get_platform_lib(). currently trying to
replace this with something different.

Besides fixing the quoting, the implmentation seems to be questionable
(currently distutils.sysconfig.get_platform_lib() is used in this test), because
distutils.sysconfig.get_platform_lib() points to the platform library, not the
installation the module should be installed to, which is not the same in all
cases. Now trying not to use get_platform_lib(), but trying to run an setup
install command with --dry-run, and make this work with any distutils included
in 2.0 and up.

- $ python -c "from distutils.core import setup; setup()" -n install
running install
running build
running install_egg_info
Creating /usr/local/lib/python2.6/site-packages/
Writing /usr/local/lib/python2.6/site-packages/UNKNOWN-0.0.0.egg-info

and parsing the output only works for python versions providing egg-info files.

- not relying on .egg-info's:
 $ python -c "from distutils.core import setup; setup(py_modules=['conftest'])"
-n install
running install
running build
running build_py
creating build
creating build/lib.linux-i686-2.6
copying conftest.py -> build/lib.linux-i686-2.6
running install_lib
warning: install_lib: 'build/lib.linux-i686-2.6' does not exist -- no Python
modules to install
running install_egg_info
Creating /usr/local/lib/python2.6/dist-packages/
Writing /usr/local/lib/python2.6/dist-packages/UNKNOWN-0.0.0.egg-info

doesn't build and install the module. Building it before creates the `build'
directory. All the tests could be done in a temporary directory, just using the
build -b option doesn't work, because install doesn't know anything about -b.

running install_lib
copying build/lib.linux-i686-2.6/conftest.py ->
/usr/local/lib/python2.6/dist-packages
error: file '/usr/local/lib/python2.6/dist-packages/conftest.py' does not exist

finishes with an error although run with -n, needs again parsing of the output.

$ python -c "from distutils.core import setup; \
    print setup().get_command_obj('install', 0).install_purelib" -n -q install
/usr/local/lib/python2.6/dist-packages

looks short, but I'm unsure about the access to the install_purelib attribute of
the install command. Seems to work for python versions 2.3 and newer. Would be
glad if somebody still has 2.[012] and could check.

I would like to use the last option, but not yet sure about it should be
allowed, or if you have to fall back to the parsing of the second option.

  Matthias

From zooko at zooko.com  Fri Apr 10 19:16:04 2009
From: zooko at zooko.com (zooko)
Date: Fri, 10 Apr 2009 11:16:04 -0600
Subject: [Distutils] Distutils changes - end user requirements (Was:
	Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904100918k4484e459hd11bb1690723b15@mail.gmail.com>
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<6DA14050-CB85-4E97-9349-DD15BE24137B@zooko.com>
	<79990c6b0904100918k4484e459hd11bb1690723b15@mail.gmail.com>
Message-ID: <83B50E4B-A0CB-4724-B4C1-3037D53D8BFB@zooko.com>

> And yet, you'll get Debian users who say your install process  
> doesn't work for them because it doesn't use apt, and doesn't  
> register the packages in the apt database. And Windows users who'll  
> point out that it doesn't work behind their corporate firewall, etc.

Yes -- we already have lots of those!  That's why we also  
provide .deb's, and why the makers of commercial software products  
that use Tahoe provide Windows installers, etc.. The install.html  
page [1] are not for everyone. No single installation process can  
possibly satisfy everyone.

> (I don't know if Tahoe uses Twisted, one of my gripes is that it's  
> not clear from the Tahoe web page precisely *what* it depends on,  
> and hence what I'd have to be careful to avoid fiddling with...)

Hm.  That information is stored here:

http://allmydata.org/trac/tahoe/browser/_auto_deps.py

I've just added a hyperlink to there and an explanation on this page  
here:

http://allmydata.org/trac/tahoe/wiki/InstallDetails

> Why? If your users want it and/or it's simple for you to produce  
> *in case* a potential users wants it, then why do you refuse to  
> distribute it simply because you don't like the user experience it  
> offers?

Because I would need to spend time supporting it by explaining to  
users who try to use it how to install the dozen or so dependencies.   
I've done that in the past for the Tahoe project -- it is a huge  
timesink, especially if you want to give a similar level of support  
to the dozen or so *other* platforms which have their own preferred  
installation methods.  Switching to the setuptools-based install  
instructions [1] saves a great deal of effort on my part.  Alas that  
method of installation is, of course, not acceptable for many users,  
so it is but a beginning.

> Anyway, thanks again for actually trying all this. I think I  
> understand your POV a bit better now. And you may want to  
> investigate py2exe as a distribution option more suited to your  
> Windows users' expectations

Sure, the commercial product based on Tahoe which is made by  
allmydata.com uses py2exe for that purpose.  The "bbfreeze" tool that  
I mentioned will hopefully grow into a better replacement for py2exe.

Regards,

Zooko

[1] http://allmydata.org/source/tahoe/trunk/docs/install.html


From zooko at zooko.com  Fri Apr 10 19:27:07 2009
From: zooko at zooko.com (zooko)
Date: Fri, 10 Apr 2009 11:27:07 -0600
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
	<ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
Message-ID: <3092E027-8257-4B45-AB88-8F554827FE8F@zooko.com>

On Apr 9, 2009, at 14:25 PM, Kent Tenney wrote:

> to get an app to run or compile I do a bunch of
> sudo apt-get install xxxx
>
> This can involve tedious trial and error.
>
> Maybe I like the app, maybe I don't.
>
> If I like it, I want a convenient format in which to remember the  
> required packages, automate their installation, provide convenience  
> when building machines.
>
> If I don't like it, I'd like to remove the packages I installed.
>
> Buildout does this for eggs, and tarballs via cmmi, I want the same  
> convenience for system packages.

What I would do is turn the Python package into a .deb which declares  
its dependency on those other .deb's, using stdeb.  Then I do "sudo  
apt-get install $THAT_NEW_ONE" and it automatically brings in the  
others.  When I decide I don't like it, I do "sudo apt-get remove  
$THAT_NEW_ONE", and apt cleverly figures out that I don't need those  
other ones anymore.

Regards,

Zooko

From zooko at zooko.com  Fri Apr 10 19:36:36 2009
From: zooko at zooko.com (zooko)
Date: Fri, 10 Apr 2009 11:36:36 -0600
Subject: [Distutils] RFC:  Updating PEP 345
In-Reply-To: <49DE7956.6080507@palladion.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
Message-ID: <0FFEFFFE-C8F9-4A95-B010-6880C0884026@zooko.com>

On Apr 9, 2009, at 16:40 PM, Tres Seaver wrote:

> Jim Fulton wrote:
>> On Apr 9, 2009, at 12:55 PM, Tres Seaver wrote:
>>> setuptools can treat 'install_requires' as an undeprecated alias  
>>> for 'requires_dist'.
>>
>> What is the rational for this?  I'd strongly prefer the "requires"  
>> argument name to be compatible with setuptools.  Otherwise, I  
>> think we'll introduce needless confusion.
>
> I'm aiming for self-consistency within the 'PKG-INFO' field names:


The easier we can make it for The New Distutils to be compatible with  
existing and widely used [1] setuptools and easy_install the better.   
To the extent that you can suffer a little bit of ugliness or  
variation in the New Distutils in order to ease compatibility, I  
would be very grateful if you would do so.

For example, what if I want to run "easy_install http://example.com/ 
foo.tar.gz", where foo is packaged with the New Distutils?  This  
currently works with the Old Distutils, but foo is unable to declare  
its dependencies on the "bar" distribution using the Old Distutils.   
It also currently works if foo was packaged with setuptools. The  
biggest (to my mind) single improvement in the New Distutils is that  
if foo is packaged with the New Distutils then foo is able to declare  
its dependency on "bar".  Would it be nice if it spelled it in a way  
that was intelligible to the current crop widely-deployed tools?

It will be easier for people to make things like that work if you  
don't change the names of things unnecessarily.

Regards,

Zooko

[1] http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first- 
results

From doko at ubuntu.com  Fri Apr 10 19:42:08 2009
From: doko at ubuntu.com (Matthias Klose)
Date: Fri, 10 Apr 2009 19:42:08 +0200
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>	<372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
	<ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
Message-ID: <49DF84F0.70903@ubuntu.com>

Kent Tenney schrieb:
> On Thu, Apr 9, 2009 at 2:38 PM, Jim Fulton <jim at zope.com> wrote:
>> On Apr 9, 2009, at 3:37 PM, Kent Tenney wrote:
>>
>>> Howdy,
>>>
>>> Is there a recommended way to manage Debian system
>>> packages with a buildout?
>>
>> I can't imagine why you would want to. I probably don't know what you're
>> asking.
> 
> to get an app to run or compile I do a bunch of
> sudo apt-get install xxxx
> 
> This can involve tedious trial and error.
> 
> Maybe I like the app, maybe I don't.
> 
> If I like it, I want a convenient format in which to remember
> the required packages, automate their installation,
> provide convenience when building machines.
> 
> If I don't like it, I'd like to remove the packages I installed.

if you remove the packages manually, that you did install directly, you can get
rid of the dependencies by running apt-get autoremove. afaik there's currently
no way to install a package marked for autoremoval by hand.

  Matthias

From pje at telecommunity.com  Fri Apr 10 20:56:47 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 10 Apr 2009 14:56:47 -0400
Subject: [Distutils] Distutils changes - end user requirements (Was:
 Deprecate MANIFEST.in)
In-Reply-To: <79990c6b0904100918k4484e459hd11bb1690723b15@mail.gmail.com
 >
References: <79990c6b0904080327h7800a7cbk7ca680adbb4a417e@mail.gmail.com>
	<319e029f0904082120i63859844hc11575e31fc001f1@mail.gmail.com>
	<79990c6b0904090224j5c400bp6cb7956e82819dcb@mail.gmail.com>
	<6DA14050-CB85-4E97-9349-DD15BE24137B@zooko.com>
	<79990c6b0904100918k4484e459hd11bb1690723b15@mail.gmail.com>
Message-ID: <20090410185419.93D1B3A4063@sparrow.telecommunity.com>

At 05:18 PM 4/10/2009 +0100, Paul Moore wrote:
>Hmm, and this is *precisely* one of the things I dislike. I don't like
>anything being installed except on my request.

I know this won't affect your use case, but for the benefit of 
everyone else listening, I just want to mention that there is a 
--no-deps option to easy_install.  It should probably print what the 
dependencies are, but at least it exists.


From ktenney at gmail.com  Fri Apr 10 21:04:36 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Fri, 10 Apr 2009 14:04:36 -0500
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <3092E027-8257-4B45-AB88-8F554827FE8F@zooko.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<372D2540-A3A6-421B-8A73-ABA4C6DA740F@zope.com>
	<ca816810904091325g7c8b1216rf1cf9f0229820139@mail.gmail.com>
	<3092E027-8257-4B45-AB88-8F554827FE8F@zooko.com>
Message-ID: <ca816810904101204v1ccc2cefya6e449e55bbcc4a8@mail.gmail.com>

On Fri, Apr 10, 2009 at 12:27 PM, zooko <zooko at zooko.com> wrote:
> On Apr 9, 2009, at 14:25 PM, Kent Tenney wrote:
>
>> to get an app to run or compile I do a bunch of
>> sudo apt-get install xxxx
>>
>> This can involve tedious trial and error.
>>
>> Maybe I like the app, maybe I don't.
>>
>> If I like it, I want a convenient format in which to remember the required
>> packages, automate their installation, provide convenience when building
>> machines.
>>
>> If I don't like it, I'd like to remove the packages I installed.
>>
>> Buildout does this for eggs, and tarballs via cmmi, I want the same
>> convenience for system packages.
>
> What I would do is turn the Python package into a .deb which declares its
> dependency on those other .deb's, using stdeb. ?Then I do "sudo apt-get
> install $THAT_NEW_ONE" and it automatically brings in the others. ?When I
> decide I don't like it, I do "sudo apt-get remove $THAT_NEW_ONE", and apt
> cleverly figures out that I don't need those other ones anymore.
>

Interesting, building on existing clever, good peg_shape/hole_shape match.

Thanks,
Kent

> Regards,
>
> Zooko
>

From ianb at colorstudy.com  Fri Apr 10 21:08:59 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 10 Apr 2009 14:08:59 -0500
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com> 
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com> 
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com> 
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
Message-ID: <b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>

On Fri, Apr 10, 2009 at 9:40 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> > Why change the name? A different name isn't going to be better enough to
> be
> > worth the hassle. Deprecation is waaaay overrated as a tool for reducing
> the
> > pain of making people change their code or habits.
>
> I don't think it's a good idea to have a different name in PKG-INFO
> and in the arguments to describe
> the same element. we should have the same name everywhere for
> consistency at the end.
>
> I don't see anything wrong about adding a simple deprecation warning
> here, It won't happen again
> for quite a while.
>

People who install packages freak out over warnings.  If you could do a
warning during a PyPI upload, then someone who can actually make a change
might see it.  People installing a package should not see this warning.  I
feel very strongly about this as a general rule - putting messages intended
for packagers into the output presented during installation is distracting
and disconcerting and useless.

In the "check" command it would be entirely proper to issue a warning.  But
no one is going to re-release a project just to fix the spelling of this
argument in setup.py, and a lot of libraries just don't get updated often,
or people deliberately use old versions to avoid regression.  So outside of
the check command it should not cause any warning.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090410/d87205d2/attachment.htm>

From workitharder at gmail.com  Sat Apr 11 01:09:19 2009
From: workitharder at gmail.com (Buck)
Date: Fri, 10 Apr 2009 16:09:19 -0700 (PDT)
Subject: [Distutils] How to make easy_install handle platlibs?
Message-ID: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>

I'm trying to separate platform-specific and platform-independent
files on my Python installation. ./configure --prefix --exec-prefix
does this nicely for Python proper, but modules seem to be a whole
different story.

>From my testing, distutils handles this OK, putting "pure" python
modules into the --prefix directory and modules with binary .so's into
the --execprefix. I guess it defaults the install-platbase to be the
same as the --exec-prefix used in configure?

When running easy_install on the same packages (elementtree and
numpy), they all go into the platform-independant area.

Is this expected?

I tried settings some things in the distutils.cfg, but nothing I did
seemed to help.
--Buck

From workitharder at gmail.com  Sat Apr 11 01:09:19 2009
From: workitharder at gmail.com (Buck)
Date: Fri, 10 Apr 2009 16:09:19 -0700 (PDT)
Subject: [Distutils] How to make easy_install handle platlibs?
Message-ID: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>

I'm trying to separate platform-specific and platform-independent
files on my Python installation. ./configure --prefix --exec-prefix
does this nicely for Python proper, but modules seem to be a whole
different story.

>From my testing, distutils handles this OK, putting "pure" python
modules into the --prefix directory and modules with binary .so's into
the --execprefix. I guess it defaults the install-platbase to be the
same as the --exec-prefix used in configure?

When running easy_install on the same packages (elementtree and
numpy), they all go into the platform-independant area.

Is this expected?

I tried settings some things in the distutils.cfg, but nothing I did
seemed to help.
--Buck

From pje at telecommunity.com  Sat Apr 11 05:18:48 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 10 Apr 2009 23:18:48 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegro
	ups.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
Message-ID: <20090411031621.97A333A4063@sparrow.telecommunity.com>

At 04:09 PM 4/10/2009 -0700, Buck wrote:
>I'm trying to separate platform-specific and platform-independent
>files on my Python installation. ./configure --prefix --exec-prefix
>does this nicely for Python proper, but modules seem to be a whole
>different story.
>
> >From my testing, distutils handles this OK, putting "pure" python
>modules into the --prefix directory and modules with binary .so's into
>the --execprefix. I guess it defaults the install-platbase to be the
>same as the --exec-prefix used in configure?
>
>When running easy_install on the same packages (elementtree and
>numpy), they all go into the platform-independant area.
>
>Is this expected?

Eggs that are platform-dependent include the platform information in 
their filename; there isn't any support for putting them in different 
directories as well.


From workitharder at gmail.com  Sat Apr 11 05:53:40 2009
From: workitharder at gmail.com (Buck)
Date: Fri, 10 Apr 2009 20:53:40 -0700 (PDT)
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <20090411031621.97A333A4063@sparrow.telecommunity.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
Message-ID: <a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>


On Apr 10, 8:18?pm, "P.J. Eby" <p... at telecommunity.com> wrote:
> At 04:09 PM 4/10/2009 -0700, Buck wrote:
>
> >I'm trying to separate platform-specific and platform-independent
> >files on my Python installation. ./configure --prefix --exec-prefix
> >does this nicely for Python proper, but modules seem to be a whole
> >different story.
>
> > >From my testing, distutils handles this OK, putting "pure" python
> >modules into the --prefix directory and modules with binary .so's into
> >the --execprefix. I guess it defaults the install-platbase to be the
> >same as the --exec-prefix used in configure?
>
> >When running easy_install on the same packages (elementtree and
> >numpy), they all go into the platform-independant area.
>
> >Is this expected?
>
> Eggs that are platform-dependent include the platform information in
> their filename; there isn't any support for putting them in different
> directories as well.

I see the kernel version and architecture, but this is insufficient;
RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in
provided libraries are sufficient to make many (most?) "impure"
libraries stop working ( numpy, python-ldap, and hashlib for
examples).

easy_install apparently knows when packages are platform-dependant,
and the necessary directory is sys.exec_prefix (or maybe better:
distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems
like an easy change. Would a patch be accepted?



From pje at telecommunity.com  Sat Apr 11 06:15:46 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 11 Apr 2009 00:15:46 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegro
	ups.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
Message-ID: <20090411041320.3B27B3A4063@sparrow.telecommunity.com>

At 08:53 PM 4/10/2009 -0700, Buck wrote:
>I see the kernel version and architecture, but this is insufficient;
>RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in
>provided libraries are sufficient to make many (most?) "impure"
>libraries stop working ( numpy, python-ldap, and hashlib for
>examples).
>
>easy_install apparently knows when packages are platform-dependant,
>and the necessary directory is sys.exec_prefix (or maybe better:
>distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems
>like an easy change. Would a patch be accepted?

It would be a pretty major patch, since the distutils logic that's 
used for determining the installation directory is applied long 
before easy_install even knows what it's installing (and therefore, 
whether it's platform dependent).

It would probably be a lot easier to improve the platform string 
generation and comparison logic, as has been done for OS X.


From chris at simplistix.co.uk  Sat Apr 11 10:57:55 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 11 Apr 2009 09:57:55 +0100
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <e5fff6640904091345r2e0e18f5pcdbb197c168760c1@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
	<ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
	<e5fff6640904091345r2e0e18f5pcdbb197c168760c1@mail.gmail.com>
Message-ID: <49E05B93.9080602@simplistix.co.uk>

Benji York wrote:
> [install-some-debs]
> recipe = iw.recipe.cmd
> on_install = true
> on_update = true
> cmds = apt-get install ...

If I understand the OP, he's deploying on debian and wants to be able to do:

bin/buildout

...to set up a whole machine.
I love that idea, and in that case, I'd want a recipe that looked like:

[debian-packages]
recipe = ...recipe.aptitude
packages =
   logwatch
   postfix
   subversion
   ...

...which would basically do:

sudo aptitude install

...for each package in the list.

Have I missed something?
If not, lemme know when the recipe is written ;-)

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Sat Apr 11 14:20:25 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 11 Apr 2009 13:20:25 +0100
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
Message-ID: <49E08B09.9070307@simplistix.co.uk>

Tarek Ziad? wrote:
> I don't think it's a good idea to have a different name in PKG-INFO
> and in the arguments to describe
> the same element. we should have the same name everywhere for
> consistency at the end.
> 
> I don't see anything wrong about adding a simple deprecation warning
> here, It won't happen again
> for quite a while.
> 
> Distutils already had some argument name changes like that for
> backward compatiblity (licence<->license) but they were not
> displaying any warning.

What would be the problem with having both sets work and not deprecate 
either? (ie: make them synonyms of each other?)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From jim at zope.com  Sat Apr 11 14:27:59 2009
From: jim at zope.com (Jim Fulton)
Date: Sat, 11 Apr 2009 08:27:59 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <49E08B09.9070307@simplistix.co.uk>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<49E08B09.9070307@simplistix.co.uk>
Message-ID: <C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>


On Apr 11, 2009, at 8:20 AM, Chris Withers wrote:

> Tarek Ziad? wrote:
>> I don't think it's a good idea to have a different name in PKG-INFO
>> and in the arguments to describe
>> the same element. we should have the same name everywhere for
>> consistency at the end.
>> I don't see anything wrong about adding a simple deprecation warning
>> here, It won't happen again
>> for quite a while.
>> Distutils already had some argument name changes like that for
>> backward compatiblity (licence<->license) but they were not
>> displaying any warning.
>
> What would be the problem with having both sets work and not  
> deprecate either? (ie: make them synonyms of each other?)


It violates toowtdi.

Jim

--
Jim Fulton
Zope Corporation



From chris at simplistix.co.uk  Sat Apr 11 14:29:26 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 11 Apr 2009 13:29:26 +0100
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<49E08B09.9070307@simplistix.co.uk>
	<C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
Message-ID: <49E08D26.2020803@simplistix.co.uk>

Jim Fulton wrote:
> 
> On Apr 11, 2009, at 8:20 AM, Chris Withers wrote:
> 
>> Tarek Ziad? wrote:
>>> I don't think it's a good idea to have a different name in PKG-INFO
>>> and in the arguments to describe
>>> the same element. we should have the same name everywhere for
>>> consistency at the end.
>>> I don't see anything wrong about adding a simple deprecation warning
>>> here, It won't happen again
>>> for quite a while.
>>> Distutils already had some argument name changes like that for
>>> backward compatiblity (licence<->license) but they were not
>>> displaying any warning.
>>
>> What would be the problem with having both sets work and not deprecate 
>> either? (ie: make them synonyms of each other?)
> 
> It violates toowtdi.

*shrugs*

It's not duplicating code, and if it stops another 200 message thread on 
python-dev, I'm pretty sure we can live with it ;-)

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From ktenney at gmail.com  Sat Apr 11 15:11:16 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Sat, 11 Apr 2009 08:11:16 -0500
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <49E05B93.9080602@simplistix.co.uk>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
	<ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
	<e5fff6640904091345r2e0e18f5pcdbb197c168760c1@mail.gmail.com>
	<49E05B93.9080602@simplistix.co.uk>
Message-ID: <ca816810904110611y4675ad64m5af771eb39550595@mail.gmail.com>

On Sat, Apr 11, 2009 at 3:57 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> Benji York wrote:
>>
>> [install-some-debs]
>> recipe = iw.recipe.cmd
>> on_install = true
>> on_update = true
>> cmds = apt-get install ...
>
> If I understand the OP, he's deploying on debian and wants to be able to do:
>
> bin/buildout
>
> ...to set up a whole machine.

Right.

> I love that idea, and in that case, I'd want a recipe that looked like:
>
> [debian-packages]
> recipe = ...recipe.aptitude
> packages =
> ?logwatch
> ?postfix
> ?subversion
> ?...
>
> ...which would basically do:
>
> sudo aptitude install
>
> ...for each package in the list.

and
$ sudo aptitude remove

if a package is removed from the list.

I want buildout.cfg to be `executable documentation`, a place
to remember details in a helpful way.

>
> Have I missed something?
> If not, lemme know when the recipe is written ;-)

I appreciate the encouragement.

I've gotten several suggestions, I like the stdeb approach
but it adds another learning curve.

I think I'll copy iw.recipe.cmd to recipe.aptitude, change the
`cmds` option to `packages` and monkey with that.

Sound reasonable?

Thanks,
Kent

>
> Chris
>
> --
> Simplistix - Content Management, Zope & Python Consulting
> ? ? ? ? ? - http://www.simplistix.co.uk
>

From ziade.tarek at gmail.com  Sat Apr 11 15:33:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 11 Apr 2009 15:33:51 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
Message-ID: <94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>

On Fri, Apr 10, 2009 at 9:08 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> On Fri, Apr 10, 2009 at 9:40 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> > Why change the name? A different name isn't going to be better enough to
>> > be
>> > worth the hassle. Deprecation is waaaay overrated as a tool for reducing
>> > the
>> > pain of making people change their code or habits.
>>
>> I don't think it's a good idea to have a different name in PKG-INFO
>> and in the arguments to describe
>> the same element. we should have the same name everywhere for
>> consistency at the end.
>>
>> I don't see anything wrong about adding a simple deprecation warning
>> here, It won't happen again
>> for quite a while.
>
> People who install packages freak out over warnings.? If you could do a
> warning during a PyPI upload, then someone who can actually make a change
> might see it.? People installing a package should not see this warning.? I
> feel very strongly about this as a general rule - putting messages intended
> for packagers into the output presented during installation is distracting
> and disconcerting and useless.
>
> In the "check" command it would be entirely proper to issue a warning.? But
> no one is going to re-release a project just to fix the spelling of this
> argument in setup.py, and a lot of libraries just don't get updated often,
> or people deliberately use old versions to avoid regression.? So outside of
> the check command it should not cause any warning.

Right, sounds like

>
> --
> Ian Bicking ?| ?http://blog.ianbicking.org
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sat Apr 11 15:38:05 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 11 Apr 2009 15:38:05 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>
Message-ID: <94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>

On Sat, Apr 11, 2009 at 3:33 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

>>> I don't see anything wrong about adding a simple deprecation warning
>>> here, It won't happen again
>>> for quite a while.
>>
>> People who install packages freak out over warnings.? If you could do a
>> warning during a PyPI upload, then someone who can actually make a change
>> might see it.? People installing a package should not see this warning.? I
>> feel very strongly about this as a general rule - putting messages intended
>> for packagers into the output presented during installation is distracting
>> and disconcerting and useless.
>>
>> In the "check" command it would be entirely proper to issue a warning.? But
>> no one is going to re-release a project just to fix the spelling of this
>> argument in setup.py, and a lot of libraries just don't get updated often,
>> or people deliberately use old versions to avoid regression.? So outside of
>> the check command it should not cause any warning.
>
> Right, sounds like

oups, send it too early

Right, sounds like a good practice.

So what shall we do for install_requires ?

I'd be in favor of :

- keeping the new PKG-INFO Tres proposed
- maintaining both setup() arguments (like license and licence)
- documenting the new argument
- adding the warning in the 'check' command

Cheers
Tarek

From ziade.tarek at gmail.com  Sat Apr 11 16:09:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 11 Apr 2009 16:09:50 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>
	<94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>
Message-ID: <94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>

On Sat, Apr 11, 2009 at 3:38 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>
> Right, sounds like a good practice.
>
> So what shall we do for install_requires ?
>
> I'd be in favor of :
>
> - keeping the new PKG-INFO Tres proposed
> - maintaining both setup() arguments (like license and licence)
> - documenting the new argument
> - adding the warning in the 'check' command
>

(unless everyone is fine with having a install_requires argument and a
different PKG-INFO field)

From jim at zope.com  Sat Apr 11 16:13:43 2009
From: jim at zope.com (Jim Fulton)
Date: Sat, 11 Apr 2009 10:13:43 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>
	<94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>
	<94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>
Message-ID: <23D16F21-DE2A-4F2F-A19E-9C3A2D5EB5A9@zope.com>


On Apr 11, 2009, at 10:09 AM, Tarek Ziad? wrote:

> On Sat, Apr 11, 2009 at 3:38 PM, Tarek Ziad? <ziade.tarek at gmail.com>  
> wrote:
>>
>> Right, sounds like a good practice.
>>
>> So what shall we do for install_requires ?
>>
>> I'd be in favor of :
>>
>> - keeping the new PKG-INFO Tres proposed
>> - maintaining both setup() arguments (like license and licence)
>> - documenting the new argument
>> - adding the warning in the 'check' command
>>
>
> (unless everyone is fine with having a install_requires argument and a
> different PKG-INFO field)


I'd prefer that to having 2 arguments that mean the same thing.

fwiw, I'd also prefer calling the PKG-INFO field Install-Requires.

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Sat Apr 11 17:02:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 11 Apr 2009 17:02:09 +0200
Subject: [Distutils] Adding alias into Distutils ?
Message-ID: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>

Hello

setuptools 'alias' command is very handy
<http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-for-commonly-used-commands>

What about integrating it into Distutils ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From jim at zope.com  Sat Apr 11 19:02:53 2009
From: jim at zope.com (Jim Fulton)
Date: Sat, 11 Apr 2009 13:02:53 -0400
Subject: [Distutils] Adding alias into Distutils ?
In-Reply-To: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
References: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
Message-ID: <D256C33E-83FA-487B-97C0-01B0240BC91A@zope.com>


On Apr 11, 2009, at 11:02 AM, Tarek Ziad? wrote:

> Hello
>
> setuptools 'alias' command is very handy
> <http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-for-commonly-used-commands 
> >
>
> What about integrating it into Distutils ?


-1.  I love setuptools, but it suffers from feature bloat, making it  
intimidating and hard to learn.

Jim

--
Jim Fulton
Zope Corporation



From hannosch at hannosch.eu  Sat Apr 11 19:07:12 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Sat, 11 Apr 2009 19:07:12 +0200
Subject: [Distutils] Adding alias into Distutils ?
In-Reply-To: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
References: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
Message-ID: <grqinu$8ea$1@ger.gmane.org>

Tarek Ziad? wrote:
> setuptools 'alias' command is very handy
> <http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-for-commonly-used-commands>
> 
> What about integrating it into Distutils ?

-0 it's not an essential part in any way.

If the feature didn't exist, I would have just created some aliases via
Unix alias command or wrote me a simple shell script. There's some many
other ways to get aliases.

Hanno


From tseaver at palladion.com  Sat Apr 11 20:09:01 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 11 Apr 2009 14:09:01 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>	<49E08B09.9070307@simplistix.co.uk>
	<C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
Message-ID: <grqmbv$fjj$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> On Apr 11, 2009, at 8:20 AM, Chris Withers wrote:
> 
>> Tarek Ziad? wrote:
>>> I don't think it's a good idea to have a different name in PKG-INFO
>>> and in the arguments to describe
>>> the same element. we should have the same name everywhere for
>>> consistency at the end.
>>> I don't see anything wrong about adding a simple deprecation warning
>>> here, It won't happen again
>>> for quite a while.
>>> Distutils already had some argument name changes like that for
>>> backward compatiblity (licence<->license) but they were not
>>> displaying any warning.
>> What would be the problem with having both sets work and not  
>> deprecate either? (ie: make them synonyms of each other?)
> 
> 
> It violates toowtdi.

BBB concerns often trump TOOWTDI.

Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ4Ny9+gerLs4ltQ4RArIhAJ0YVl/MHK/MJENRd/aF2LcN/x5O3QCgppbL
rfhjfh+M5DRRgAFdRBoOa0M=
=X6S7
-----END PGP SIGNATURE-----


From workitharder at gmail.com  Sun Apr 12 09:36:42 2009
From: workitharder at gmail.com (Buck Golemon)
Date: Sun, 12 Apr 2009 00:36:42 -0700
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <20090411041320.3B27B3A4063@sparrow.telecommunity.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
Message-ID: <c70f1ac50904120036u5a953cfbjee0d4a9f8dfc4d26@mail.gmail.com>

On Fri, Apr 10, 2009 at 9:15 PM, P.J. Eby <pje at telecommunity.com> wrote:

> At 08:53 PM 4/10/2009 -0700, Buck wrote:
>
>> I see the kernel version and architecture, but this is insufficient;
>> RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in
>> provided libraries are sufficient to make many (most?) "impure"
>> libraries stop working ( numpy, python-ldap, and hashlib for
>> examples).
>>
>> easy_install apparently knows when packages are platform-dependant,
>> and the necessary directory is sys.exec_prefix (or maybe better:
>> distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems
>> like an easy change. Would a patch be accepted?
>>
>
> It would be a pretty major patch, since the distutils logic that's used for
> determining the installation directory is applied long before easy_install
> even knows what it's installing (and therefore, whether it's platform
> dependent).
>
> It would probably be a lot easier to improve the platform string generation
> and comparison logic, as has been done for OS X.
>
> If the distutils logic is used, then why does numpy install to the platlib
folder using distutils, but into the nonplatlib folder using easy_install?

Your tone seems to suggest that such an effort might be accepted upstream.

--Buck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090412/5fcc2d30/attachment-0001.htm>

From akitada at gmail.com  Sun Apr 12 09:42:47 2009
From: akitada at gmail.com (Akira Kitada)
Date: Sun, 12 Apr 2009 16:42:47 +0900
Subject: [Distutils] Adding alias into Distutils ?
In-Reply-To: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
References: <94bdd2610904110802t497cf858jbc1a7cc87979b7b7@mail.gmail.com>
Message-ID: <90bb445a0904120042r692ff17bt98a9bc732b6f0419@mail.gmail.com>

-1.
Please keep it simple and let 3rd party tools do the fancy things.
Simple is better than complex.

On Sun, Apr 12, 2009 at 12:02 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello
>
> setuptools 'alias' command is very handy
> <http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-for-commonly-used-commands>
>
> What about integrating it into Distutils ?
>
> Regards
> Tarek
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From jim at zope.com  Sun Apr 12 15:15:52 2009
From: jim at zope.com (Jim Fulton)
Date: Sun, 12 Apr 2009 09:15:52 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <grqmbv$fjj$2@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>	<49E08B09.9070307@simplistix.co.uk>
	<C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
	<grqmbv$fjj$2@ger.gmane.org>
Message-ID: <5998F8FB-1E0B-4415-82EA-A25A9727E39F@zope.com>


On Apr 11, 2009, at 2:09 PM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jim Fulton wrote:
>> On Apr 11, 2009, at 8:20 AM, Chris Withers wrote:
>>
>>> Tarek Ziad? wrote:
>>>> I don't think it's a good idea to have a different name in PKG-INFO
>>>> and in the arguments to describe
>>>> the same element. we should have the same name everywhere for
>>>> consistency at the end.
>>>> I don't see anything wrong about adding a simple deprecation  
>>>> warning
>>>> here, It won't happen again
>>>> for quite a while.
>>>> Distutils already had some argument name changes like that for
>>>> backward compatiblity (licence<->license) but they were not
>>>> displaying any warning.
>>> What would be the problem with having both sets work and not
>>> deprecate either? (ie: make them synonyms of each other?)
>>
>>
>> It violates toowtdi.
>
> BBB concerns often trump TOOWTDI.


Sure, but there's no BBB concern here -- unless we create one.

Jim

--
Jim Fulton
Zope Corporation



From zooko at zooko.com  Sun Apr 12 17:19:57 2009
From: zooko at zooko.com (zooko)
Date: Sun, 12 Apr 2009 09:19:57 -0600
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <20090411041320.3B27B3A4063@sparrow.telecommunity.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
Message-ID: <2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>

> It would probably be a lot easier to improve the platform string  
> generation and comparison logic, as has been done for OS X.

As PJE has mentioned, the intent is that the egg name should contain  
enough information to decide if that egg will work on your platform.   
For example, if it says "py2.5-win32" then you know it will work on  
your Python 2.5 on 32-bit Windows, and if it says "py2.5-macosx-10.3- 
ppc" then you know it will work on your PowerPC-based Mac with Python  
2.5.  This can be used for example by easy_install to get a directory  
listing of eggs and choose which one will work on the local platform  
based solely on its filename.

As PJE mentioned, it would be nice if this same technique worked on  
Linux.

However, it currently doesn't.  Eggs built on Linux are named  
something like py2.5-Linux-x86_64.  To know whether such an egg would  
actually work on your Linux system, you would also need to know  
whether the Python was compiled with UCS-2 or UCS-4 internal unicode  
representation, as well as what version of glibc you have.  Is there  
anything else that would need to be added into the egg name?

Regards,

Zooko

From zooko at zooko.com  Sun Apr 12 17:30:50 2009
From: zooko at zooko.com (zooko)
Date: Sun, 12 Apr 2009 09:30:50 -0600
Subject: [Distutils] Buildout recipe for managing .deb packages?
In-Reply-To: <ca816810904110611y4675ad64m5af771eb39550595@mail.gmail.com>
References: <ca816810904091237x362b87bfh1c9e617c07515a75@mail.gmail.com>
	<319e029f0904091324j6c7f3eb5g4fa3440dee96d561@mail.gmail.com>
	<ca816810904091326u7089ce52hb78ac60f2e870bd6@mail.gmail.com>
	<e5fff6640904091345r2e0e18f5pcdbb197c168760c1@mail.gmail.com>
	<49E05B93.9080602@simplistix.co.uk>
	<ca816810904110611y4675ad64m5af771eb39550595@mail.gmail.com>
Message-ID: <6CEBEA06-A18E-4F32-A4A2-254DDB500D2E@zooko.com>

> I've gotten several suggestions, I like the stdeb approach but it  
> adds another learning curve.

sudo apt-get install build-essential fakeroot debhelper
python ./setup.py sdist_dsc
cd deb_dist
cd $YOUR_PACKAGE_NAME
dpkg-buildpackage -rfakeroot -us -uc

And if it doesn't work, please let us know!

I'm not saying that it isn't a learning curve.

Regards,

Zooko

From zooko at zooko.com  Sun Apr 12 17:41:14 2009
From: zooko at zooko.com (zooko)
Date: Sun, 12 Apr 2009 09:41:14 -0600
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <23D16F21-DE2A-4F2F-A19E-9C3A2D5EB5A9@zope.com>
References: <49DE2889.8070807@palladion.com>
	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>
	<49DE7956.6080507@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>
	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>
	<94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>
	<94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>
	<23D16F21-DE2A-4F2F-A19E-9C3A2D5EB5A9@zope.com>
Message-ID: <2042D93D-DFF7-4C2B-A773-D35D6AB312EB@zooko.com>

On Apr 11, 2009, at 8:13 AM, Jim Fulton wrote:

> fwiw, I'd also prefer calling the PKG-INFO field Install-Requires.

+1 from me.

I was just browsing pypi, and there seem to be many tools out there  
which are designed to work with the current egg metadata.  The fewer  
unnecessary name-changes we impose, the more likely the authors of  
all those tools will update their tools to interoperate with the new  
Distutils.

Speaking of which, the proposed 'build_requires' seems to be somewhat  
similar to setuptools's 'setup_requires'.  The difference seems to be  
that 'build_requires' is intended for non-Python tools and  
'setup_requires' currently works only for Python packages  
("distributions").

Regards,

Zooko

From tseaver at palladion.com  Sun Apr 12 17:47:12 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 12 Apr 2009 11:47:12 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <5998F8FB-1E0B-4415-82EA-A25A9727E39F@zope.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>	<49E08B09.9070307@simplistix.co.uk>	<C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>	<grqmbv$fjj$2@ger.gmane.org>
	<5998F8FB-1E0B-4415-82EA-A25A9727E39F@zope.com>
Message-ID: <grt2ec$6om$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> On Apr 11, 2009, at 2:09 PM, Tres Seaver wrote:

>> BBB concerns often trump TOOWTDI.
> 
> 
> Sure, but there's no BBB concern here -- unless we create one.

Your desire to preserve the 'install_requires' argument feels like a
backward-compatibility requirement to me (and one which I'm fine with
keeping indefinitely). You have a separate desire, which is to use the
spelling 'Install-Requires' for the PKG-INFO field corresponding to
'install_requires'.  I would prefer to keep the names close to those
already proposed in PEP 345 ('Requries-External', 'Requires-Python').

My desire is to keep the PKG-INFO specification and field names clean
and internally consistent:  there is no BBB concern here, because the
existing 'Requires:' field won't be touched (although we will deprecate
it).  Keeping the fieldnames consistent within that file seems more
important to me than an (already violated) expectation that setup()
argument names map mechanically onto PKG-INFO field names.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ4g0A+gerLs4ltQ4RAs9gAKDGhLB69nyrBT0jQ5tkhHheP8hJLwCgsOO+
3LgxigdTWW+MYTaBEWziDlk=
=ymHa
-----END PGP SIGNATURE-----


From strawman at astraw.com  Sun Apr 12 17:54:08 2009
From: strawman at astraw.com (Andrew Straw)
Date: Sun, 12 Apr 2009 08:54:08 -0700
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>	<20090411031621.97A333A4063@sparrow.telecommunity.com>	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
Message-ID: <49E20EA0.8000601@astraw.com>

zooko wrote:
> However, it currently doesn't.  Eggs built on Linux are named something
> like py2.5-Linux-x86_64.  To know whether such an egg would actually
> work on your Linux system, you would also need to know whether the
> Python was compiled with UCS-2 or UCS-4 internal unicode representation,
> as well as what version of glibc you have.  Is there anything else that
> would need to be added into the egg name?

Yes, if you used symbols from any shared library in an extension module,
you'd need to know the version of that shared library. So it's not just
libc. This is the same on any OS, not just linux.

One example is anything that uses the numpy C API (e.g. matplotlib,
pyopengl 2.x, and so on). Fortunately, the numpy C API is very stable,
so there hasn't been any incompatibility introduced in the numpy 1.x
series. Yet.

-Andrew

From tseaver at palladion.com  Sun Apr 12 17:51:00 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 12 Apr 2009 11:51:00 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>	<20090411031621.97A333A4063@sparrow.telecommunity.com>	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
Message-ID: <grt2lg$6om$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

zooko wrote:
>> It would probably be a lot easier to improve the platform string  
>> generation and comparison logic, as has been done for OS X.
> 
> As PJE has mentioned, the intent is that the egg name should contain  
> enough information to decide if that egg will work on your platform.   
> For example, if it says "py2.5-win32" then you know it will work on  
> your Python 2.5 on 32-bit Windows, and if it says "py2.5-macosx-10.3- 
> ppc" then you know it will work on your PowerPC-based Mac with Python  
> 2.5.  This can be used for example by easy_install to get a directory  
> listing of eggs and choose which one will work on the local platform  
> based solely on its filename.
> 
> As PJE mentioned, it would be nice if this same technique worked on  
> Linux.
> 
> However, it currently doesn't.  Eggs built on Linux are named  
> something like py2.5-Linux-x86_64.  To know whether such an egg would  
> actually work on your Linux system, you would also need to know  
> whether the Python was compiled with UCS-2 or UCS-4 internal unicode  
> representation, as well as what version of glibc you have.  Is there  
> anything else that would need to be added into the egg name?

Better:  just don't distribute binary eggs for Linux at all:

 - Developers have the tools (or can install them) to build from
   'sdists'.

 - Systems without tools are "locked down", which means that they
   shouldn't be installing from public distributions anyway:
   the folks who run them should be able to build *private* eggs
   from 'sdists' which are known to work for their systems.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ4g3k+gerLs4ltQ4RAjepAKDK9/Y7Gq218CtUOw2KRtAkbLYcWACgtYF5
ogSEjrYyCWzliUo5/jHGm94=
=4smt
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Sun Apr 12 18:06:20 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 12 Apr 2009 12:06:20 -0400
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <2042D93D-DFF7-4C2B-A773-D35D6AB312EB@zooko.com>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>	<94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>	<94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>	<23D16F21-DE2A-4F2F-A19E-9C3A2D5EB5A9@zope.com>
	<2042D93D-DFF7-4C2B-A773-D35D6AB312EB@zooko.com>
Message-ID: <grt3i8$b1s$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

zooko wrote:
> On Apr 11, 2009, at 8:13 AM, Jim Fulton wrote:
> 
>> fwiw, I'd also prefer calling the PKG-INFO field Install-Requires.
> 
> +1 from me.
> 
> I was just browsing pypi, and there seem to be many tools out there  
> which are designed to work with the current egg metadata.  The fewer  
> unnecessary name-changes we impose, the more likely the authors of  
> all those tools will update their tools to interoperate with the new  
> Distutils.

None of them are working with any dependency-based PKG-INFO fields,
becuase there is no useful information in those fields.  Some of them
may be working with arguments passed to 'setup()' (e.g., by running
'egg-info' in a tempdir and parsing the setuptools-specific files), but
such tools will have to be changed anyway (e.g., to use the new PKG-INFO
fields instead of parsing requries.txt).

> Speaking of which, the proposed 'build_requires' seems to be somewhat  
> similar to setuptools's 'setup_requires'.  The difference seems to be  
> that 'build_requires' is intended for non-Python tools and  
> 'setup_requires' currently works only for Python packages  
> ("distributions").

Correct.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ4hF8+gerLs4ltQ4RApg/AKCQy7YFO0qj2e8dLIwbdP47NOzG7ACgkLIL
ImQeogWLYTUUAjqFvDUkKZQ=
=2Ukp
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Sun Apr 12 18:20:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 12 Apr 2009 18:20:40 +0200
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <grt2ec$6om$1@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>
	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>
	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>
	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>
	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>
	<49E08B09.9070307@simplistix.co.uk>
	<C2B3FF58-2D6A-4729-9341-26E257AA1C42@zope.com>
	<grqmbv$fjj$2@ger.gmane.org>
	<5998F8FB-1E0B-4415-82EA-A25A9727E39F@zope.com>
	<grt2ec$6om$1@ger.gmane.org>
Message-ID: <94bdd2610904120920v50c9d571q4f5970fa5cbc83e5@mail.gmail.com>

On Sun, Apr 12, 2009 at 5:47 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jim Fulton wrote:
>> On Apr 11, 2009, at 2:09 PM, Tres Seaver wrote:
>
>>> BBB concerns often trump TOOWTDI.
>>
>>
>> Sure, but there's no BBB concern here -- unless we create one.
>
> Your desire to preserve the 'install_requires' argument feels like a
> backward-compatibility requirement to me (and one which I'm fine with
> keeping indefinitely). You have a separate desire, which is to use the
> spelling 'Install-Requires' for the PKG-INFO field corresponding to
> 'install_requires'. ?I would prefer to keep the names close to those
> already proposed in PEP 345 ('Requries-External', 'Requires-Python').
>
> My desire is to keep the PKG-INFO specification and field names clean
> and internally consistent: ?there is no BBB concern here, because the
> existing 'Requires:' field won't be touched (although we will deprecate
> it). ?Keeping the fieldnames consistent within that file seems more
> important to me than an (already violated) expectation that setup()
> argument names map mechanically onto PKG-INFO field names.

While there's no bijection between the arguments and  the PKG-INFO fields,
some arguments *are*  PKG-INFO fields;

If we would start a package from scratch today, what we would have for thos ?
probably the same name for the argument and the field.

What I hear here is that since we don't start from scratch we have to
compromise on consistency so we don't bather with BBB. But the goal
is also to make Distutils a better citizen in this area *and* to use
the best name.

And some people in the future might not understand why they can't use
"requires_dist" in the arguments, like they use "classifiers" and "author",

and they will find it rather strange if we tell them "This is for
historical reason !"

So, while I perfectly understand Jim and Tres points, I do think we
need to have an argument that matches the PKG-INFO field name, even
if it's doubled with another one.

Cheers
Tarek

From workitharder at gmail.com  Sun Apr 12 19:43:12 2009
From: workitharder at gmail.com (Buck Golemon)
Date: Sun, 12 Apr 2009 10:43:12 -0700
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <49E20EA0.8000601@astraw.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<49E20EA0.8000601@astraw.com>
Message-ID: <c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>

On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw <strawman at astraw.com> wrote:

> zooko wrote:
> > However, it currently doesn't.  Eggs built on Linux are named something
> > like py2.5-Linux-x86_64.  To know whether such an egg would actually
> > work on your Linux system, you would also need to know whether the
> > Python was compiled with UCS-2 or UCS-4 internal unicode representation,
> > as well as what version of glibc you have.  Is there anything else that
> > would need to be added into the egg name?
>
> Yes, if you used symbols from any shared library in an extension module,
> you'd need to know the version of that shared library. So it's not just
> libc. This is the same on any OS, not just linux.
>
> -Andrew
>

Exactly. The platform name/version ( RedHat 3 / Ubuntu 7.07 / Gentoo X.X )
can serve as a much better proxy for the "version of all shared libraries"
than just the kernel version alone (2.4 / 2.6).

To add to Andrew's example, python-ldap created for Redhat 4 depends on
ldap_r.so.1.0.4 but for RedHat 5 depends on ldap_r.so.1.0.6. Both have
kernel 2.6, so are indistinguishable under current naming scheme.

I notice that for Debian/Ubuntu the lsb_release command is implemented in
Python, although it's mostly just reading out /etc/debian_version. RedHat's
lsb_release uses bash, but similarly just reads in a file. Gentoo also has a
lsb_release.

The "completely correct" way to do this would be to run ldd on all binaries,
stick the result in a metadata file somewhere, then load the version of the
egg where the most (hopefully all) libraries exist.

On the other hand, sticking the egg into the place that distutils uses when
not under easy_install would fix this much more simply, although from what I
hear this would be a big change.

-- 
Buck Golemon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090412/daa0266f/attachment.htm>

From zooko at zooko.com  Sun Apr 12 20:13:41 2009
From: zooko at zooko.com (zooko)
Date: Sun, 12 Apr 2009 12:13:41 -0600
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<49E20EA0.8000601@astraw.com>
	<c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>
Message-ID: <AD7EFAB4-2CA1-473A-BD27-42377E54DF55@zooko.com>

> Yes, if you used symbols from any shared library in an extension  
> module, you'd need to know the version of that shared library. So  
> it's not just libc. This is the same on any OS, not just linux.

Wait a minute, an extension module built into the Python Standard  
Library, you mean?  Because for separately packaged packages  
("distributions") such as the numpy that you mentioned, your package  
("distribution") would express its requirement on that other package  
("distribution") in its install_requires metadata, not in its name.   
There, in the install_requires metadata, it can also express which  
version it requires.  Right?

Regards,

Zooko


From strawman at astraw.com  Sun Apr 12 21:01:10 2009
From: strawman at astraw.com (Andrew Straw)
Date: Sun, 12 Apr 2009 12:01:10 -0700
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <AD7EFAB4-2CA1-473A-BD27-42377E54DF55@zooko.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<49E20EA0.8000601@astraw.com>
	<c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>
	<AD7EFAB4-2CA1-473A-BD27-42377E54DF55@zooko.com>
Message-ID: <49E23A76.8000708@astraw.com>

zooko wrote:
>> Yes, if you used symbols from any shared library in an extension
>> module, you'd need to know the version of that shared library. So it's
>> not just libc. This is the same on any OS, not just linux.
> 
> Wait a minute, an extension module built into the Python Standard
> Library, you mean?  

No, I'm writing about non stdlib extension modules.

Because for separately packaged packages
> ("distributions") such as the numpy that you mentioned, your package
> ("distribution") would express its requirement on that other package
> ("distribution") in its install_requires metadata, not in its name. 
> There, in the install_requires metadata, it can also express which
> version it requires.  Right?

No, because the act of compiling your .egg fixes the specific version
(e.g. numpy==1.3) to keep the ABI compatible with the version of numpy
installed at extension module compile time. Whereas the install_requires
is about API compatibility, and could thus be numpy>=1.2, for example.

(For pure Python modules, this isn't a problem because there is only an
API version to worry about. But with compiled extensions, there is also
the ABI version to worry about.)

From pje at telecommunity.com  Sun Apr 12 21:40:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 12 Apr 2009 15:40:51 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <c70f1ac50904120036u5a953cfbjee0d4a9f8dfc4d26@mail.gmail.co
 m>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<c70f1ac50904120036u5a953cfbjee0d4a9f8dfc4d26@mail.gmail.com>
Message-ID: <20090412193825.26E803A40AF@sparrow.telecommunity.com>

At 12:36 AM 4/12/2009 -0700, Buck Golemon wrote:
>On Fri, Apr 10, 2009 at 9:15 PM, P.J. Eby 
><<mailto:pje at telecommunity.com>pje at telecommunity.com> wrote:
>At 08:53 PM 4/10/2009 -0700, Buck wrote:
>I see the kernel version and architecture, but this is insufficient;
>RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in
>provided libraries are sufficient to make many (most?) "impure"
>libraries stop working ( numpy, python-ldap, and hashlib for
>examples).
>
>easy_install apparently knows when packages are platform-dependant,
>and the necessary directory is sys.exec_prefix (or maybe better:
>distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems
>like an easy change. Would a patch be accepted?
>
>
>It would be a pretty major patch, since the distutils logic that's 
>used for determining the installation directory is applied long 
>before easy_install even knows what it's installing (and therefore, 
>whether it's platform dependent).
>
>It would probably be a lot easier to improve the platform string 
>generation and comparison logic, as has been done for OS X.
>
>If the distutils logic is used, then why does numpy install to the 
>platlib folder using distutils, but into the nonplatlib folder using 
>easy_install?

Please read what I wrote after the word "distutils", i.e.:

"""the distutils logic that's used for determining the installation 
directory is applied long before easy_install even knows what it's 
installing"""

That is, the installation directory is determined *before* any 
packages are examined, so there is no way to know whether prefix or 
exec-prefix should be used.  So prefix is used to determine the 
default installation location.

Currently, there's only a way to specify one installation location.


>Your tone seems to suggest that such an effort might be accepted upstream.

I'm suggesting that fixing the platform strings would be more 
beneficial (i.e. benefit other use cases besides this one), and would 
probably be a LOT easier to implement as a patch, because the changes 
would be limited to the internals of a few well-defined, isolated functions.

Of course, if you happen to find some sane way to get easy_install to 
do what you want, I'll certainly take a look at it.  I'm just hard 
pressed to imagine how you'd go about doing it without major 
refactoring that would be very difficult to verify was done correctly.


From pje at telecommunity.com  Sun Apr 12 21:46:35 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 12 Apr 2009 15:46:35 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.co
 m>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<49E20EA0.8000601@astraw.com>
	<c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>
Message-ID: <20090412194406.8CB0E3A40AF@sparrow.telecommunity.com>

At 10:43 AM 4/12/2009 -0700, Buck Golemon wrote:


>On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw 
><<mailto:strawman at astraw.com>strawman at astraw.com> wrote:
>zooko wrote:
> > However, it currently doesn't.  Eggs built on Linux are named something
> > like py2.5-Linux-x86_64.  To know whether such an egg would actually
> > work on your Linux system, you would also need to know whether the
> > Python was compiled with UCS-2 or UCS-4 internal unicode representation,
> > as well as what version of glibc you have.  Is there anything else that
> > would need to be added into the egg name?
>
>Yes, if you used symbols from any shared library in an extension module,
>you'd need to know the version of that shared library. So it's not just
>libc. This is the same on any OS, not just linux.
>
>-Andrew
>
>
>Exactly. The platform name/version ( RedHat 3 / Ubuntu 7.07 / Gentoo 
>X.X ) can serve as a much better proxy for the "version of all 
>shared libraries" than just the kernel version alone (2.4 / 2.6).
>
>To add to Andrew's example, python-ldap created for Redhat 4 depends 
>on ldap_r.so.1.0.4 but for RedHat 5 depends on ldap_r.so.1.0.6. Both 
>have kernel 2.6, so are indistinguishable under current naming scheme.
>
>I notice that for Debian/Ubuntu the lsb_release command is 
>implemented in Python, although it's mostly just reading out 
>/etc/debian_version. RedHat's lsb_release uses bash, but similarly 
>just reads in a file. Gentoo also has a lsb_release.

On OS X, there's a similar thing done to get the platform 
name/version, and there's special version comparison code to 
determine compatibility; if this could be implemented for other 
platforms, that would be great.


>The "completely correct" way to do this would be to run ldd on all 
>binaries, stick the result in a metadata file somewhere, then load 
>the version of the egg where the most (hopefully all) libraries exist.
>
>On the other hand, sticking the egg into the place that distutils 
>uses when not under easy_install would fix this much more simply, 
>although from what I hear this would be a big change.

For easy_install, yes.  For pip, probably not so much.  In fact, my 
guess would be that pip already supports this, as long as you can 
install from source, rather than from eggs.


From yanegomi at gmail.com  Mon Apr 13 05:45:00 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Sun, 12 Apr 2009 20:45:00 -0700
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <grfq4a$2df$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
Message-ID: <364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>

On Tue, Apr 7, 2009 at 8:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>
> 1. easy_remove!
>
> 2. Various utilities to provide query package management.
> ? - easy_install --list (list files installed)

    Implementing is easier said than done, and I think you got that
idea from the lack of response from folks...
HTH,
-Garrett

From zooko at zooko.com  Mon Apr 13 06:27:00 2009
From: zooko at zooko.com (zooko)
Date: Sun, 12 Apr 2009 22:27:00 -0600
Subject: [Distutils] RFC: Updating PEP 345
In-Reply-To: <grt3i8$b1s$1@ger.gmane.org>
References: <49DE2889.8070807@palladion.com>	<74D64CF8-AD23-4089-9D4E-D1349A38CB06@zope.com>	<49DE7956.6080507@palladion.com>	<B6FA806D-31E6-4643-8098-1142818C8C93@zope.com>	<94bdd2610904100730w17463861l6ba0840228d0db45@mail.gmail.com>	<4F080ED9-DC18-47B1-9388-12721C75294E@zope.com>	<94bdd2610904100740q2c84a645r9ad169ff30961ae5@mail.gmail.com>	<b654cd2e0904101208h26b52adeqd83a5e64db782e21@mail.gmail.com>	<94bdd2610904110633n441ab78eg5324e6effc2b5bce@mail.gmail.com>	<94bdd2610904110638g80d008ei9a8cdc1e80ec067a@mail.gmail.com>	<94bdd2610904110709v7c69616ck1ef12d8638a64dd0@mail.gmail.com>	<23D16F21-DE2A-4F2F-A19E-9C3A2D5EB5A9@zope.com>
	<2042D93D-DFF7-4C2B-A773-D35D6AB312EB@zooko.com>
	<grt3i8$b1s$1@ger.gmane.org>
Message-ID: <7F130FF1-21D4-41AA-B914-432BC72346A4@zooko.com>

On Apr 12, 2009, at 10:06 AM, Tres Seaver wrote:

>> ... there seem to be many tools out there which are designed to  
>> work with the current egg metadata.  The fewer unnecessary name- 
>> changes we impose, the more likely the authors of all those tools  
>> will update their tools to interoperate with the new Distutils.
>
> None of them are working with any dependency-based PKG-INFO fields,
...
> Some of them may be working with arguments passed to 'setup 
> ()' (e.g., by running 'egg-info' in a tempdir and parsing the  
> setuptools-specific files), but such tools will have to be changed  
> anyway (e.g., to use the new PKG-INFO fields instead of parsing  
> requries.txt).


That makes sense. What I said wasn't that the tools would  
automatically interoperate, rather that using the same name would  
make it easier for those authors to start supporting the New Distutils.

However, I think I've expressed my opinion on the issue with  
sufficient thoroughness that I can now leave the decision up to you.

Regards,

Zooko

From distutils at celestial.com  Mon Apr 13 06:56:19 2009
From: distutils at celestial.com (Bill Campbell)
Date: Sun, 12 Apr 2009 21:56:19 -0700
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
Message-ID: <20090413045619.GA21812@ayn.mi.celestial.com>

On Sun, Apr 12, 2009, Garrett Cooper wrote:
>On Tue, Apr 7, 2009 at 8:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>
>> 1. easy_remove!
>>
>> 2. Various utilities to provide query package management.
>> ? - easy_install --list (list files installed)
>
>    Implementing is easier said than done, and I think you got that
>idea from the lack of response from folks...

I don't see why this should be all that difficult to implement.

I work with the OpenPKG portable package management system, an
RPM based system that exists independently of the underlying
systems' packaging/software management, and it is not difficult
to define procedures that cleanly management package removal and
the associated dependencies.

On the other hand, when building packages, I never want the build
to be affected by processes that go to the Internet to pick up
pieces that are thought to be out of date -- which is why I am
not comfortable with much of the easy_install, setuptools philosophy.

After almost 20 years of working primarily with perl, I found
one of the big advantages of python was the lack of magic.  While
magic can be used to advantage by people like Damian Conway,
author of Objective Perl, I prefer things that are reaily
understood, and not subject to unexpected behavior.

Bill
-- 
INTERNET:   bill at celestial.com  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
Voice:          (206) 236-1676  Mercer Island, WA 98040-0820
Fax:            (206) 232-9186  Skype: jwccsllc (206) 855-5792

The end move in politics is always to pick up a gun.  -- Buckminster Fuller

From david at ar.media.kyoto-u.ac.jp  Mon Apr 13 06:54:24 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 13 Apr 2009 13:54:24 +0900
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <20090413045619.GA21812@ayn.mi.celestial.com>
References: <grfism$7j5$1@ger.gmane.org>
	<grfq4a$2df$1@ger.gmane.org>	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
Message-ID: <49E2C580.5010704@ar.media.kyoto-u.ac.jp>

Bill Campbell wrote:
> On Sun, Apr 12, 2009, Garrett Cooper wrote:
>   
>> On Tue, Apr 7, 2009 at 8:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>     
>>> 1. easy_remove!
>>>
>>> 2. Various utilities to provide query package management.
>>>   - easy_install --list (list files installed)
>>>       
>>    Implementing is easier said than done, and I think you got that
>> idea from the lack of response from folks...
>>     
>
> I don't see why this should be all that difficult to implement.
>   

In part because different people want different things from the
packaging, with often conflicting goals. Those conflicts increase pretty
quickly when you take into account different platforms/type of
development (desktop apps vs "libraries" vs webapp vs etc...). The
"cross-platformness" alone, with the underlying philosophical
differences on how to do deployment  (bundling vs backward compatible
dependencies, etc...) is quite hard to manage.

David

From ianb at colorstudy.com  Mon Apr 13 19:09:26 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 13 Apr 2009 12:09:26 -0500
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <grfq4a$2df$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
Message-ID: <b654cd2e0904131009u1fbe58cxe63217c490ece36a@mail.gmail.com>

 For people interested in uninstall, you might want to check out this branch
of pip:
http://bitbucket.org/carljm/pip-uninstall/overview/

It only uninstalls things pip installed (because pip is keeping a record of
installed files, which is used for the uninstallation).

On Tue, Apr 7, 2009 at 10:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

>
> 1. easy_remove!
>
> 2. Various utilities to provide query package management.
>   - easy_install --list (list files installed)
>

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090413/4bd61eda/attachment.htm>

From workitharder at gmail.com  Mon Apr 13 20:11:42 2009
From: workitharder at gmail.com (Buck)
Date: Mon, 13 Apr 2009 11:11:42 -0700 (PDT)
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <20090412194406.8CB0E3A40AF@sparrow.telecommunity.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com> 
	<49E20EA0.8000601@astraw.com>
	<c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com> 
	<20090412194406.8CB0E3A40AF@sparrow.telecommunity.com>
Message-ID: <54ccedae-547e-445d-91b6-439dbac64a36@b16g2000yqb.googlegroups.com>



On Apr 12, 12:46?pm, "P.J. Eby" <p... at telecommunity.com> wrote:
> >On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw
> ><<mailto:straw... at astraw.com>straw... at astraw.com> wrote:
> >On the other hand, sticking the egg into the place that distutils
> >uses when not under easy_install would fix this much more simply,
> >although from what I hear this would be a big change.
>
> For easy_install, yes. ?For pip, probably not so much. ?In fact, my
> guess would be that pip already supports this, as long as you can
> install from source, rather than from eggs.

Are you saying I should dump easy_install in favor of pip? I hadn't
heard of it till now.

From workitharder at gmail.com  Mon Apr 13 20:16:17 2009
From: workitharder at gmail.com (Buck)
Date: Mon, 13 Apr 2009 11:16:17 -0700 (PDT)
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <grt2lg$6om$2@ger.gmane.org>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com> 
	<grt2lg$6om$2@ger.gmane.org>
Message-ID: <2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>

On Apr 12, 8:51?am, Tres Seaver <tsea... at palladion.com> wrote:
> zooko wrote:
> >> It would probably be a lot easier to improve the platform string ?
> >> generation and comparison logic, as has been done for OS X.
>
> > However, it (egg naming scheme on Linux) currently doesn't. ?
> > Eggs built on Linux are named something like py2.5-Linux-x86_64. ?
> > ...
>
> Better: ?just don't distribute binary eggs for Linux at all:
>
> ?- Developers have the tools (or can install them) to build from
> ? ?'sdists'.
>
> ?- Systems without tools are "locked down", which means that they
> ? ?shouldn't be installing from public distributions anyway:
> ? ?the folks who run them should be able to build *private* eggs
> ? ?from 'sdists' which are known to work for their systems.
>
> Tres.

I have no clue what you mean by 'sdists'. Is this a widely-known
thing?
A URL to an example would be sufficient.

Thanks,
--Buck

From pje at telecommunity.com  Mon Apr 13 20:37:11 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 13 Apr 2009 14:37:11 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <54ccedae-547e-445d-91b6-439dbac64a36@b16g2000yqb.googlegro
	ups.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<49E20EA0.8000601@astraw.com>
	<c70f1ac50904121043v643ea63ayea2656f4f3ae4f0c@mail.gmail.com>
	<20090412194406.8CB0E3A40AF@sparrow.telecommunity.com>
	<54ccedae-547e-445d-91b6-439dbac64a36@b16g2000yqb.googlegroups.com>
Message-ID: <20090413183441.3073B3A40AC@sparrow.telecommunity.com>

At 11:11 AM 4/13/2009 -0700, Buck wrote:


>On Apr 12, 12:46 pm, "P.J. Eby" <p... at telecommunity.com> wrote:
> > >On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw
> > ><<mailto:straw... at astraw.com>straw... at astraw.com> wrote:
> > >On the other hand, sticking the egg into the place that distutils
> > >uses when not under easy_install would fix this much more simply,
> > >although from what I hear this would be a big change.
> >
> > For easy_install, yes.  For pip, probably not so much.  In fact, my
> > guess would be that pip already supports this, as long as you can
> > install from source, rather than from eggs.
>
>Are you saying I should dump easy_install in favor of pip? I hadn't
>heard of it till now.

If it meets your other use cases, I would say yes.


From pje at telecommunity.com  Mon Apr 13 20:39:47 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 13 Apr 2009 14:39:47 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegro
	ups.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<grt2lg$6om$2@ger.gmane.org>
	<2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>
Message-ID: <20090413183717.B6BFE3A40AC@sparrow.telecommunity.com>

At 11:16 AM 4/13/2009 -0700, Buck wrote:
>On Apr 12, 8:51 am, Tres Seaver <tsea... at palladion.com> wrote:
> > zooko wrote:
> > >> It would probably be a lot easier to improve the platform string
> > >> generation and comparison logic, as has been done for OS X.
> >
> > > However, it (egg naming scheme on Linux) currently doesn't.
> > > Eggs built on Linux are named something like py2.5-Linux-x86_64.
> > > ...
> >
> > Better:  just don't distribute binary eggs for Linux at all:
> >
> >  - Developers have the tools (or can install them) to build from
> >    'sdists'.
> >
> >  - Systems without tools are "locked down", which means that they
> >    shouldn't be installing from public distributions anyway:
> >    the folks who run them should be able to build *private* eggs
> >    from 'sdists' which are known to work for their systems.
> >
> > Tres.
>
>I have no clue what you mean by 'sdists'. Is this a widely-known
>thing?
>A URL to an example would be sufficient.

An sdist is a source distribution, usually in .tgz or .zip 
form.  Sdists contain certain well-defined directory layouts and 
files.  In particular:

* the entire contents are in a subdirectory named for the project and version

* that subdirectory contains a setup.py and a PKG-INFO

easy_install and pip can both find and retrieve sdists from PyPI and 
install them, they just go about the actual installation process a 
bit differently.


From ziade.tarek at gmail.com  Mon Apr 13 23:23:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 13 Apr 2009 23:23:10 +0200
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
Message-ID: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>

Hello

I am working on PEP 376
<http://svn.python.org/projects/peps/trunk/pep-0376.txt>,  and there's
a part about adding an install and uninstall script into Distutils.

The question is, do we want to propose a reference implementation of
these scripts into Distutils ?

I'd be in favor of just describing in this PEP what a install script
and an uninstall script should do, and not include any reference
implementation in Distutils.
(and leave it to pip, setuptools, etc..)

It forces people to pick on third party tool, but as long as it
complies with the PEP, I guess it's better to leave it outside.

Any opinion ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Tue Apr 14 00:06:03 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 13 Apr 2009 18:06:03 -0400
Subject: [Distutils] PEP 376 - install/uninstall script in  Distutils ?
In-Reply-To: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com
 >
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
Message-ID: <20090413220335.328763A40AC@sparrow.telecommunity.com>

At 11:23 PM 4/13/2009 +0200, Tarek Ziad? wrote:
>Hello
>
>I am working on PEP 376
><http://svn.python.org/projects/peps/trunk/pep-0376.txt>,  and there's
>a part about adding an install and uninstall script into Distutils.

By the way, the PEP would benefit from some clarifications and 
separation of terminology between "project" names and "package" 
names.  AFAICT, it speaks of "packages" to mean both Python packages 
and a distribution of a Python project.

Also, does pip really create a .pth file per installed package (project?)?

(Btw, I'm also not clear on the purpose of including MANIFEST in .egg-info.)


From david at ar.media.kyoto-u.ac.jp  Tue Apr 14 03:45:56 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 14 Apr 2009 10:45:56 +0900
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>	<20090411031621.97A333A4063@sparrow.telecommunity.com>	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<grt2lg$6om$2@ger.gmane.org>
	<2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>
Message-ID: <49E3EAD4.6070805@ar.media.kyoto-u.ac.jp>

Buck wrote:
>
> I have no clue what you mean by 'sdists'. Is this a widely-known
> thing?
>   

sdist is the name of the distutils command - it just means distributing
a source tarball.

cheers,

David

From ben+python at benfinney.id.au  Tue Apr 14 04:16:35 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 14 Apr 2009 12:16:35 +1000
Subject: [Distutils] How to make easy_install handle platlibs?
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>
	<20090411031621.97A333A4063@sparrow.telecommunity.com>
	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>
	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>
	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>
	<grt2lg$6om$2@ger.gmane.org>
	<2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>
	<49E3EAD4.6070805@ar.media.kyoto-u.ac.jp>
Message-ID: <87eivw7zek.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Buck wrote:
> >
> > I have no clue what you mean by 'sdists'. Is this a widely-known
> > thing?
> 
> sdist is the name of the distutils command - it just means
> distributing a source tarball.

Again, please stop blurring the distinction. I can only assume at this
point that it's deliberate on your part, but I have no idea why.

A distutils ?sdist? is a combination of the source files *and* metadata
for distutils. A mere source tarball is *not* an sdist.

-- 
 \     ?I know you believe you understood what you think I said, but I |
  `\         am not sure you realize that what you heard is not what I |
_o__)                                     meant.? ?Robert J. McCloskey |
Ben Finney


From marius at pov.lt  Tue Apr 14 09:39:18 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 14 Apr 2009 10:39:18 +0300
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <20090413045619.GA21812@ayn.mi.celestial.com>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
Message-ID: <20090414073918.GA21024@fridge.pov.lt>

On Sun, Apr 12, 2009 at 09:56:19PM -0700, Bill Campbell wrote:
> On Sun, Apr 12, 2009, Garrett Cooper wrote:
> >On Tue, Apr 7, 2009 at 8:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> >>
> >> 1. easy_remove!
> >>
> >> 2. Various utilities to provide query package management.
> >> ? - easy_install --list (list files installed)
> >
> >    Implementing is easier said than done, and I think you got that
> >idea from the lack of response from folks...
> 
> I don't see why this should be all that difficult to implement.

Please do not hesitate to send patches.

;)

Marius Gedminas
-- 
For vi emulation of emacs, just type ":sh emacs" (without the quotes).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090414/32bb2931/attachment.pgp>

From ndbecker2 at gmail.com  Tue Apr 14 14:56:21 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 14 Apr 2009 08:56:21 -0400
Subject: [Distutils] What's missing from easy_install
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
Message-ID: <gs215l$q5m$1@ger.gmane.org>

David Cournapeau wrote:

> Bill Campbell wrote:
>> On Sun, Apr 12, 2009, Garrett Cooper wrote:
>>   
>>> On Tue, Apr 7, 2009 at 8:05 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>>     
>>>> 1. easy_remove!
>>>>
>>>> 2. Various utilities to provide query package management.
>>>>   - easy_install --list (list files installed)
>>>>       
>>>    Implementing is easier said than done, and I think you got that
>>> idea from the lack of response from folks...
>>>     
>>
>> I don't see why this should be all that difficult to implement.
>>   
> 
> In part because different people want different things from the
> packaging, with often conflicting goals. Those conflicts increase pretty
> quickly when you take into account different platforms/type of
> development (desktop apps vs "libraries" vs webapp vs etc...). The
> "cross-platformness" alone, with the underlying philosophical
> differences on how to do deployment  (bundling vs backward compatible
> dependencies, etc...) is quite hard to manage.
> 

The issue I need to address is to cooperate with other packaging systems.  
I'm using Fedora, which is rpm/yum based.  A new python module is announced, 
I'd like to easy_install it.  The official fedora package may be delayed by 
weeks.  So I easy_install.  But when the fedora update comes, they may 
conflict.  For example, scons from fedora will place things in 
/usr/lib/scons, not the same as easy_install.  easy_install will modify 
easy-install.pth.  Nothing will clean it.  So, there is a real need for 
easy_uninstall. 



From david at ar.media.kyoto-u.ac.jp  Tue Apr 14 14:46:55 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 14 Apr 2009 21:46:55 +0900
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <gs215l$q5m$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org>
	<grfq4a$2df$1@ger.gmane.org>	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>	<20090413045619.GA21812@ayn.mi.celestial.com>	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
Message-ID: <49E485BF.8080409@ar.media.kyoto-u.ac.jp>

Hi Neal,

Neal Becker wrote:
> I'm using Fedora, which is rpm/yum based.  A new python module is announced, 
> I'd like to easy_install it.  The official fedora package may be delayed by 
> weeks.  So I easy_install.  But when the fedora update comes, they may 
> conflict.  For example, scons from fedora will place things in 
> /usr/lib/scons, not the same as easy_install.

In that case, there is an easy way to fix this: do not install anything
in /usr. You should *never* install anything in /usr, because it is
considered owned by the system packager. Either you install it in
/usr/local (if you want to keep things installed system-wide), or
somewhere where you can write as a unprivileged user.

> Nothing will clean it.  So, there is a real need for 
> easy_uninstall. 
>   

I did not want to imply that it was not a useful feature - I think it is
a very useful feature (I dream of the day when I won't have to say
"please remove the installed directory before installing an updated
numpy checkout" :) ). It is just much harder to do reliably than what
some people think, in part because there are now so many ways to install
things in python (distutils, setuptools, pip, yolk, etc...), with
different side-effects on installation.

cheers,

David

From berthold.hoellmann at gl-group.com  Tue Apr 14 16:58:33 2009
From: berthold.hoellmann at gl-group.com (=?ISO-8859-15?Q?Berthold_=22H=F6llmann=22?=)
Date: Tue, 14 Apr 2009 16:58:33 +0200
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <grfq4a$2df$1@ger.gmane.org> (Neal Becker's message of "Tue\,
	07 Apr 2009 11\:05\:46 -0400")
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
Message-ID: <sed4bfnuxy.fsf@pc081634.dhcp.germanlloyd.org>

Neal Becker <ndbecker2 at gmail.com> writes:

> 1. easy_remove!
>
> 2. Various utilities to provide query package management.
>    - easy_install --list (list files installed)

Support for installations that differentiate between `--prefix` and
`--exec-prefix` on configure, i.e. as with all Python installations most
header files are in the directory

  >>> "%s/include/python%s" % (sys.prefix, sys.version[:3])

But `pyconfig.h` is found in

  >>> "%s/include/python%s" % (sys.exec_prefix, sys.version[:3])

And for installation, modules should go to 

  >>> "%s/lib/python%s/site-packages" % (sys.prefix, sys.version[:3])

But C extensions to 

  >>> "%s/lib/python%s/site-packages" %  (sys.exec_prefix, sys.version[:3])

Kind regards

Berthold H?llmann
-- 
Germanischer Lloyd AG
CAE Development
Vorsetzen 35
20459 Hamburg
Phone: +49(0)40 36149-7374
Fax: +49(0)40 36149-7320
e-mail: berthold.hoellmann at gl-group.com
Internet: http://www.gl-group.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090414/3e261fde/attachment.pgp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: disclaimer.txt
Type: application/octet-stream
Size: 2196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090414/3e261fde/attachment.obj>

From pje at telecommunity.com  Tue Apr 14 18:35:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 14 Apr 2009 12:35:26 -0400
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <gs215l$q5m$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
Message-ID: <20090414163258.79EE13A4100@sparrow.telecommunity.com>

At 08:56 AM 4/14/2009 -0400, Neal Becker wrote:
>easy_install will modify
>easy-install.pth.  Nothing will clean it.

"easy_install -mxN projectname" will.  (The options mean: install 
multi-version (i.e. remove from the .pth), exclude scripts, and don't 
install dependencies).


From zooko at zooko.com  Tue Apr 14 20:45:49 2009
From: zooko at zooko.com (zooko)
Date: Tue, 14 Apr 2009 12:45:49 -0600
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <gs215l$q5m$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
Message-ID: <011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>

On Apr 14, 2009, at 6:56 AM, Neal Becker wrote:

> The issue I need to address is to cooperate with other packaging  
> systems.  I'm using Fedora, which is rpm/yum based.  A new python  
> module is announced, I'd like to easy_install it.  The official  
> fedora package may be delayed by weeks.  So I easy_install.  But  
> when the fedora update comes, they may conflict.  For example,  
> scons from fedora will place things in /usr/lib/scons, not the same  
> as easy_install.  easy_install will modify easy-install.pth.   
> Nothing will clean it.  So, there is a real need for easy_uninstall.

GNU stow is great for this kind of thing.  If the New Distutils only  
writes new files and directories on installation (i.e. it does not  
need to *change* an existing file, the way the current easy_install  
has to change the contents of easy_install.pth), then it will be  
compatible with GNU stow, which will give me the best uninstall I  
could want.  (For one thing, because I can use the same tool -- GNU  
stow -- to install and uninstall any software package, regardless of  
what programming language it is written in).

Another option would be if you have a sufficiently automatic  
"bdist_rpm" feature with which you can easily produce .rpm files for  
your Python packages ("distributions").

Regards,

Zooko


From hannosch at hannosch.eu  Tue Apr 14 21:07:55 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Tue, 14 Apr 2009 21:07:55 +0200
Subject: [Distutils] Should metadata known about compatibility?
Message-ID: <gs2mub$8pj$1@ger.gmane.org>

Hi,

I'd like to suggest that distutils metadata should gain a way of
specifying compatibility with packages in addition to the requirement of
packages.

What are the use cases?

- A package wants to express that it works with both Python 2 and 3.
- A package wants to express that it works with TurboGears 2 or Plone 4

>From my point of view this information is most useful for framework-like
packages, when those do major backwards incompatible changes to their
code base.

In those circumstances typical question that people want to have answers
to are:

- For a press release of Python or a Framework, how many packages in
PyPi already support the new version?

- Running a particular application consisting of 200 packages, can I
upgrade to the next version of Framework X?

This type of information would always be optional and won't ever be used
for all packages. I think it could be nonetheless helpful in many
circumstances.

On a side note to express this information for Python itself, it would
be a good idea to expose the Python version in a distutils way, so I
could also write:

install_requires=['Python >= 3.0']

Comments?

Hanno

P.S. We want this kind of behavior for Plone, to give "plugins" to the
application / framework the ability to express this information.


From ziade.tarek at gmail.com  Tue Apr 14 21:46:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 14 Apr 2009 21:46:07 +0200
Subject: [Distutils] Should metadata known about compatibility?
In-Reply-To: <49E4E388.1010100@hannosch.eu>
References: <gs2mub$8pj$1@ger.gmane.org>
	<94bdd2610904141219j51b984b4w94a3fbd267789d5b@mail.gmail.com>
	<49E4E388.1010100@hannosch.eu>
Message-ID: <94bdd2610904141246r241e5055pc053cfaf6b0bbd03@mail.gmail.com>

Oops, I got off list by accident..

On Tue, Apr 14, 2009 at 9:27 PM, Hanno Schlichting <hannosch at hannosch.eu> wrote:
> Hi Tarek,
>
> Tarek Ziad? wrote:
>> On Tue, Apr 14, 2009 at 9:07 PM, Hanno Schlichting <hannosch at hannosch.eu> wrote:
>>> Hi,
>>>
>>> I'd like to suggest that distutils metadata should gain a way of
>>> specifying compatibility with packages in addition to the requirement of
>>> packages.
>>>
>>> What are the use cases?
>>>
>>> - A package wants to express that it works with both Python 2 and 3.
>>
>> You may use the classifiers for this use case (Martin has added soem trove
>> classifier for python 2/3)
>
> Right, which restricts this system to Python itself and won't make it
> possible to use it for other packages. Adding classifiers for each major
> version of each framework seems tedious.
>
>>> - A package wants to express that it works with TurboGears 2 or Plone 4
>>
>> How it would be different from install_requires (that can point to Plone==3.xx)
>
> Many plug-ins would be compatible with both Plone 3.x and Plone 4.x. So
> they only require Plone >= 3. The additional information is "works with
> Plone 3" and "Plone 4".

I think the classifiers works pretty well there. (If we look at your use cases,
and the classifiers browsing features at PyPI)

The only problem I can see is the fact that they have to be added "manually"
at PyPI.  If we put apart this problem, do you see any remaining problem ?


Cheers
Tarek

From hannosch at hannosch.eu  Tue Apr 14 21:59:44 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Tue, 14 Apr 2009 21:59:44 +0200
Subject: [Distutils] Should metadata known about compatibility?
In-Reply-To: <94bdd2610904141246r241e5055pc053cfaf6b0bbd03@mail.gmail.com>
References: <gs2mub$8pj$1@ger.gmane.org>	<94bdd2610904141219j51b984b4w94a3fbd267789d5b@mail.gmail.com>	<49E4E388.1010100@hannosch.eu>
	<94bdd2610904141246r241e5055pc053cfaf6b0bbd03@mail.gmail.com>
Message-ID: <gs2pvg$is0$1@ger.gmane.org>

Tarek Ziad? wrote:
> On Tue, Apr 14, 2009 at 9:27 PM, Hanno Schlichting <hannosch at hannosch.eu> wrote:
>> Tarek Ziad? wrote:
>>> On Tue, Apr 14, 2009 at 9:07 PM, Hanno Schlichting <hannosch at hannosch.eu> wrote:
>>>>
>>>> I'd like to suggest that distutils metadata should gain a way of
>>>> specifying compatibility with packages in addition to the requirement of
>>>> packages.
> 
> I think the classifiers works pretty well there. (If we look at your use cases,
> and the classifiers browsing features at PyPI)

Right, it seems to be a bit of an abuse to use classifiers which include
version numbers, but it might work well enough.

What I care most about is to have a defined way of stating this
information, so we can build tools on top of it.

I'd like to have Plone get the ability to get a button saying "Check for
upgrade" which can produce a version list of locally installed packages,
post them to a service at plone.org and get back a result saying:
"There's a new version of Plone out, but 3 of your 9 installed plug-ins
are not yet available for this version."

> The only problem I can see is the fact that they have to be added "manually"
> at PyPI.  If we put apart this problem, do you see any remaining problem ?

Yes, but this might just as well be solved by documenting the procedure
for requesting new classifiers clearly.

Hanno


From a.cavallo at acm.org  Wed Apr 15 00:25:23 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Tue, 14 Apr 2009 23:25:23 +0100
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp> <gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
Message-ID: <a71153f0904141525o58fbe01bja6f23be999fa7955@mail.gmail.com>

Stow (and encap) are great little nice tools but hardly they
fit the rpm-linux ecosystem.

A typical situation stow/encap cannot handle is when there're
two library versions installed (*.so) and the related headers:
in this case rpm can handle this situation where stow/encap
doesn't help very much.

Regards,
Antonio

On Tue, Apr 14, 2009 at 7:45 PM, zooko <zooko at zooko.com> wrote:
> On Apr 14, 2009, at 6:56 AM, Neal Becker wrote:
>
>> The issue I need to address is to cooperate with other packaging systems.
>> ?I'm using Fedora, which is rpm/yum based. ?A new python module is
>> announced, I'd like to easy_install it. ?The official fedora package may be
>> delayed by weeks. ?So I easy_install. ?But when the fedora update comes,
>> they may conflict. ?For example, scons from fedora will place things in
>> /usr/lib/scons, not the same as easy_install. ?easy_install will modify
>> easy-install.pth. ?Nothing will clean it. ?So, there is a real need for
>> easy_uninstall.
>
> GNU stow is great for this kind of thing. ?If the New Distutils only writes
> new files and directories on installation (i.e. it does not need to *change*
> an existing file, the way the current easy_install has to change the
> contents of easy_install.pth), then it will be compatible with GNU stow,
> which will give me the best uninstall I could want. ?(For one thing, because
> I can use the same tool -- GNU stow -- to install and uninstall any software
> package, regardless of what programming language it is written in).
>
> Another option would be if you have a sufficiently automatic "bdist_rpm"
> feature with which you can easily produce .rpm files for your Python
> packages ("distributions").
>
> Regards,
>
> Zooko
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From ndbecker2 at gmail.com  Wed Apr 15 01:20:36 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 14 Apr 2009 19:20:36 -0400
Subject: [Distutils] What's missing from easy_install
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
Message-ID: <gs35o4$m3a$1@ger.gmane.org>

zooko wrote:

> On Apr 14, 2009, at 6:56 AM, Neal Becker wrote:
> 
>> The issue I need to address is to cooperate with other packaging
>> systems.  I'm using Fedora, which is rpm/yum based.  A new python
>> module is announced, I'd like to easy_install it.  The official
>> fedora package may be delayed by weeks.  So I easy_install.  But
>> when the fedora update comes, they may conflict.  For example,
>> scons from fedora will place things in /usr/lib/scons, not the same
>> as easy_install.  easy_install will modify easy-install.pth.
>> Nothing will clean it.  So, there is a real need for easy_uninstall.
> 
> GNU stow is great for this kind of thing.  If the New Distutils only
> writes new files and directories on installation (i.e. it does not
> need to *change* an existing file, the way the current easy_install
> has to change the contents of easy_install.pth), then it will be
> compatible with GNU stow, which will give me the best uninstall I
> could want.  (For one thing, because I can use the same tool -- GNU
> stow -- to install and uninstall any software package, regardless of
> what programming language it is written in).
> 

That's a good point.  For many uses, unix/linux systems have moved from 
trying to have various packages modify a file, to having a directory where 
each package installs a file.  Much easier to maintain.

/usr/lib/python-xxx/site-packages/easy_install.d

would be a good choice.




From kevin at bud.ca  Wed Apr 15 05:40:10 2009
From: kevin at bud.ca (Kevin Teague)
Date: Tue, 14 Apr 2009 20:40:10 -0700
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
In-Reply-To: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
Message-ID: <A056BBAC-EFC6-45A2-AC05-499A3AB075CE@bud.ca>

-1 for install/uninstall scripts in Distutils

I'd argue that the scope of Distutils is already wide enough that it  
doesn't need to be extended to also be a "package manager" -- even if  
it's a really simple one.

If a install/uninstall tool does go into Python, I'd rather see it as  
something like 'simplepackageinstaller.install' instead of  
'Distutils.scripts.install'. This would also make it more clear that  
this tool is simply working with the standard packaging formats and  
tools, and doesn't muddy the format/implementation as much as if it's  
just another part of Distutils.

But then I don't think Python should have a built-in installer or  
package manager. There are excellent tools already available  
(Buildout, pip, dpkg, RPM), it would be better if we guided people to  
these tools and let them pick the right one for their installation use  
case.

If there was a installer, I'm assuming it'd be quite a simple one -  
e.g. installs single-version into site-packages. This caters to well  
to casual user -- they can just run a "standard" command out-of-the- 
box and take-off running with a distribution, but it also teaches them  
bad habits (e.g. that you want to be commonly installing into site- 
packages or that you want to develop your own code without properly  
expressing it's dependencies). When they want to use better  
development practices, they'll have to switch to a "non-standard"  
tools to do "non-standard" installations.


From jannis at leidel.info  Wed Apr 15 08:50:21 2009
From: jannis at leidel.info (Jannis Leidel)
Date: Wed, 15 Apr 2009 08:50:21 +0200
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <gs215l$q5m$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
Message-ID: <124237E0-F2BE-434A-9B74-6B187BF76EBC@leidel.info>

> The issue I need to address is to cooperate with other packaging  
> systems.
> I'm using Fedora, which is rpm/yum based.  A new python module is  
> announced,
> I'd like to easy_install it.  The official fedora package may be  
> delayed by
> weeks.  So I easy_install.  But when the fedora update comes, they may
> conflict.

Just as a FYI, yolk (http://tools.assembla.com/yolk/) has a plugin  
system to search for Python packages installed by other tools than  
distutils/setuptools. For example it bundles "yolk-portage", that  
shows if a package was installed by Gentoo's Portage.

http://tools.assembla.com/yolk/browser/trunk/examples/plugins

Cheers,
Jannis


From david.lyon at preisshare.net  Wed Apr 15 08:45:42 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 15 Apr 2009 02:45:42 -0400
Subject: [Distutils]
 =?utf-8?q?What=27s_missing_from_easy=5Finstall_-_just?=
 =?utf-8?q?_a_nice_gui=2E=2E?=
In-Reply-To: <gs35o4$m3a$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
	<gs35o4$m3a$1@ger.gmane.org>
Message-ID: <5bfd624e3938c09fa9eeb3c8487e9da1@preisshare.net>


Just a GUI installer to make it easy....

Hopefully I can finish off my Python Package Manager program
sometime soon and offer that for inclusion.

I think I am about four weeks away at this rate... for an
initial release..

David


From doko at ubuntu.com  Wed Apr 15 09:13:08 2009
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 15 Apr 2009 09:13:08 +0200
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
In-Reply-To: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
Message-ID: <49E58904.6060205@ubuntu.com>

Tarek Ziad? schrieb:
> Hello
> 
> I am working on PEP 376
> <http://svn.python.org/projects/peps/trunk/pep-0376.txt>,  and there's
> a part about adding an install and uninstall script into Distutils.

according to the PEP files mentioned in RECORD are relative paths, which follows
the assumption that everything is contained in the .egg directory. A new PEP
should not assume this, but support the installation into a FHS layout as well
(e.g. as done by (Linux) distributions).

  Matthias


From carlos.tejo at fundacionctic.org  Wed Apr 15 13:03:19 2009
From: carlos.tejo at fundacionctic.org (Carlos Tejo Alonso)
Date: Wed, 15 Apr 2009 13:03:19 +0200
Subject: [Distutils] Metadata-Version in PKG-INFO
Message-ID: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>

Hello,

This is my first email in distutils-sig mailing list. So I may introduce
myself. I am Carlos Tejo, a member of the Semantic Web activity line at
CTIC Foundation, a non-profit organization, where I work with a team of
research fellows. We are developing an small library [1] in python
language, and now we want to release it properly :-)

I wonder how I should "write" setup.py in order that setuptools use the
parameters of setup() to create a proper PKG-INFO file.

Right now, I have a couple of troubles:
- How can be configured the version of the metadata (AFAIK, rigth it is
always "Metadata-Version: 1.0")? I would like to use version 1.1 (PEP
314. status: final) [2] or version 1.2 (PEP 345. status: draft) [3]
- How should be written the "platform" parameter?

Regards,

-------------------------------------
Carlos Tejo Alonso
Research & Development Department
CTIC Foundation [Asturias, Spain]
www.fundacionctic.org
-------------------------------------

[1] http://rest-in-py.sourceforge.net/
[2] http://www.python.org/dev/peps/pep-0314/
[3] http://www.python.org/dev/peps/pep-0345/

From pje at telecommunity.com  Wed Apr 15 16:21:34 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 15 Apr 2009 10:21:34 -0400
Subject: [Distutils] PEP 376 - install/uninstall script in  Distutils ?
In-Reply-To: <49E58904.6060205@ubuntu.com>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
	<49E58904.6060205@ubuntu.com>
Message-ID: <20090415141906.01B023A4100@sparrow.telecommunity.com>

At 09:13 AM 4/15/2009 +0200, Matthias Klose wrote:
>Tarek Ziad? schrieb:
> > Hello
> >
> > I am working on PEP 376
> > <http://svn.python.org/projects/peps/trunk/pep-0376.txt>,  and there's
> > a part about adding an install and uninstall script into Distutils.
>
>according to the PEP files mentioned in RECORD are relative paths, 
>which follows
>the assumption that everything is contained in the .egg directory. A new PEP
>should not assume this, but support the installation into a FHS layout as well
>(e.g. as done by (Linux) distributions).

Actually, the assumption is that everything is relative to 
site-packages in a flat install.  Scripts and data should be able to 
use absolute paths, however.

The main purpose of the recommendation is that anything installed 
relative to the target directory (typically site-packages) should use 
a relative path, so that simple installations are relocatable and 
have cross-platform-compatible path information, for use cases where 
a single installation of code is shared between e.g. Windows, OS X, and Linux.


From tseaver at palladion.com  Wed Apr 15 19:48:32 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 15 Apr 2009 13:48:32 -0400
Subject: [Distutils] How to make easy_install handle platlibs?
In-Reply-To: <49E3EAD4.6070805@ar.media.kyoto-u.ac.jp>
References: <4e24bc2f-6da6-43f0-9a80-073d10fcbb44@v19g2000yqn.googlegroups.com>	<20090411031621.97A333A4063@sparrow.telecommunity.com>	<a52efb18-f79a-45a2-87bf-14e1b345d664@s28g2000vbp.googlegroups.com>	<20090411041320.3B27B3A4063@sparrow.telecommunity.com>	<2FBBA94F-58B4-4406-B935-D632A6712654@zooko.com>	<grt2lg$6om$2@ger.gmane.org>	<2fe95cb3-4635-4b9b-8678-da5d4206e4f1@w40g2000yqd.googlegroups.com>
	<49E3EAD4.6070805@ar.media.kyoto-u.ac.jp>
Message-ID: <gs56lh$54s$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> Buck wrote:
>> I have no clue what you mean by 'sdists'. Is this a widely-known
>> thing?
>>   
> 
> sdist is the name of the distutils command - it just means distributing
> a source tarball.

Nope, by "sdist" I mean the specific tarball generated by the 'sdist'
command (PJE spelled out what is special about them).

Manually-created source tarballs don't have the layout / metadata which
makes automating their installation feasible.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ5h3w+gerLs4ltQ4RAjMUAJ0bG4aCjLVeC7J2czCbrcMQshxg1QCfexDG
u0/T9gwas+IJeqN+4GsFcsM=
=l29V
-----END PGP SIGNATURE-----


From floris.bruynooghe at gmail.com  Thu Apr 16 12:42:28 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 16 Apr 2009 11:42:28 +0100
Subject: [Distutils] setup.py build_ext --rpath behaviour
Message-ID: <20090416104228.GA416@laurie.devork>

Hello

Distutils allows you to use a handy --rpath option to the build_ext
command to add an RPATH value into the linked object.  However some of
the semantics are not great IMHO.


For the first issue some background first.  On SysV-like systems
(systems using the ELF binary format) the RPATH field was created in
the .dynamic section to allow the shared object to specify extra
locations to look for their required shared objects.  However this did
overwrite the use of the LD_LIBRARY_PATH and that was no good for
administrators, so another new field was added: RUNPATH.  This was
only used *after* LD_LIBRARY_PATH instead of before it solving the
problem.  When both fields are present the runtime linker ignores
RPATH (it is impossible to create a shared object with a RUNPATH but
no RPATH).

The Solaris linker decided to always encode RUNPATH into the shared
object whenever an RPATH is specified (using the -R option).  This is
a very sensible option most likely the correct behaviour.  The GNU
linker however decided that backwards compatibility was more important
and to make it add the RUNPATH field you have to pass
--enable-new-dtags to the linker, if you use -R or -rpath on it's own
you will only get an RPATH, no RUNPATH.

Now when using the --rpath option to the build_ext command
distutils.unixcompiler.UnixCCompiler.runtime_library_dir_option() will
use some heuristics to figure out which option to pass to compiler to
get the runpath in (i.e. "-R" or "-Wl,-R" etc).  I'm going to argue
that this needs to be extened pass in -Wl,--enable-new-dtags,-R if the
GNU linker is used so that the newer and better RUNPATH gets put into
the shared objects all the time.  (I don't yet know how to detect the
GNU linker but would like a consensus on the desired behaviour before
looking into this).

Note that this is not completely a disaster, thanks to
distutils.sysconfig.customize_compiler() adding the contents of the
LDFLAGS environment variable to the command line of the linker
invocation you can work around this currently.


The second issue with build_ext --rpath is on AIX.  Again some
background on AIX shared objects, AIX is not SysV-like and uses the
XCOFF binary format instead of ELF.  Therefore they don't have a RPATH
or RUNPATH, but they do have a think called LIBPATH which does
something similar.  The difference between XCOFF's LIBPATH and ELF's
RUNPATH is that AIX's runtime linker does not have a default search
path, hence the full search path needs to be encoded into the LIBPATH
of the shared object.

Now I would propose for build_ext --rpath to encode the LIBPATH when
used on AIX since that is the correct thing to do IMHO.

(Implementation note: since this is done by passing
-blibpath:/opt/example.com/lib:/usr/lib:/lib to the linker note that
distutils.unixccompiler.UnixCCompiler.link() would have to be changed
not to strip out /usr/lib and /lib from the runtime_library_dirs on
AIX.  And
distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option()
would have to use -blibpath:... or -Wl,-blibpath as appropriate)

Again, this can currently be circumvented by using the LDFLAGS
environment variable.


Do these improvements sound sensible?  And if so should I create one
patch for each (and two bug reports) or combine them into one patch?


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Thu Apr 16 12:47:33 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 16 Apr 2009 12:47:33 +0200
Subject: [Distutils] Metadata-Version in PKG-INFO
In-Reply-To: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
Message-ID: <94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>

On Wed, Apr 15, 2009 at 1:03 PM, Carlos Tejo Alonso
<carlos.tejo at fundacionctic.org> wrote:
> Hello,

Hello Carlos,

> - How can be configured the version of the metadata (AFAIK, rigth it is
> always "Metadata-Version: 1.0")? I would like to use version 1.1 (PEP
> 314. status: final) [2] or version 1.2 (PEP 345. status: draft) [3]

If you use the fields mentioned in 1.1, Distutils will automatically
set the metadata version
in the PKG-INFO for you. That said, we are currently re-thinking this
part (see the archive
in this mailing list about PEP 345)


> - How should be written the "platform" parameter?

What do you need to define ? Most of the time, people use the trove classifiers.

Take a look at the Trove classifiers, to see if your platform is not
listed there. (under Operating System :: )

http://pypi.python.org/pypi?%3Aaction=list_classifiers


regards
Tarek


>
> Regards,
>
> -------------------------------------
> Carlos Tejo Alonso
> Research & Development Department
> CTIC Foundation [Asturias, Spain]
> www.fundacionctic.org
> -------------------------------------
>
> [1] http://rest-in-py.sourceforge.net/
> [2] http://www.python.org/dev/peps/pep-0314/
> [3] http://www.python.org/dev/peps/pep-0345/
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Apr 16 12:54:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 16 Apr 2009 12:54:02 +0200
Subject: [Distutils] setup.py build_ext --rpath behaviour
In-Reply-To: <20090416104228.GA416@laurie.devork>
References: <20090416104228.GA416@laurie.devork>
Message-ID: <94bdd2610904160354k6e9c5aa7yff2c1508a1cef512@mail.gmail.com>

On Thu, Apr 16, 2009 at 12:42 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> Do these improvements sound sensible? ?And if so should I create one
> patch for each (and two bug reports) or combine them into one patch?

They sound reasonable,  please fill two differents bugs with your patches
and if you can, add some unit tests.

As long as you provide some support to test the final vesion on the various
platforms (eg being a beta tester of the distutils trunk),
I am able to work on your patches, and on the tests.

I have not been very active on this command for the moment because of
my need of beta-testers
for the various platforms you mentioned.

I do have a vmware running for Windows, Linux and Mac but that 's it so far

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From carlos.tejo at fundacionctic.org  Thu Apr 16 16:13:18 2009
From: carlos.tejo at fundacionctic.org (Carlos Tejo Alonso)
Date: Thu, 16 Apr 2009 16:13:18 +0200
Subject: [Distutils] Metadata-Version in PKG-INFO
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
Message-ID: <09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>

Hello,

> > - How can be configured the version of the metadata (AFAIK, 
> rigth it is
> > always "Metadata-Version: 1.0")? I would like to use 
> version 1.1 (PEP
> > 314. status: final) [2] or version 1.2 (PEP 345. status: draft) [3]
> 
> If you use the fields mentioned in 1.1, Distutils will automatically
> set the metadata version
> in the PKG-INFO for you. That said, we are currently re-thinking this
> part (see the archive
> in this mailing list about PEP 345)

It's true. It was my fault because I had problems with setup parameters.

> > - How should be written the "platform" parameter?
> 
> What do you need to define ? Most of the time, people use the 
> trove classifiers.
> 
> Take a look at the Trove classifiers, to see if your platform is not
> listed there. (under Operating System :: )
> 
> http://pypi.python.org/pypi?%3Aaction=list_classifiers

It was also problem of the parameters of the setup ("platform" instead of "platforms")

Anyway, thanks for the link. About license classifiers, how can be define that the version of tht LGPL of the package is the version 3?

Another issue: Is possible to define more than one author of the package?

-------------------------------------
Carlos Tejo Alonso
Research & Development Department
CTIC Foundation [Asturias, Spain]
www.fundacionctic.org
------------------------------------- 

From ziade.tarek at gmail.com  Thu Apr 16 16:34:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 16 Apr 2009 16:34:29 +0200
Subject: [Distutils] Metadata-Version in PKG-INFO
In-Reply-To: <09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
Message-ID: <94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>

On Thu, Apr 16, 2009 at 4:13 PM, Carlos Tejo Alonso
<carlos.tejo at fundacionctic.org> wrote:
>
> Anyway, thanks for the link. About license classifiers, how can be define that the version of tht LGPL of the package is the version 3?
>

I don't see it in the list (just LGPL without any version detail), I
am ccing this request to catalog-sig so
they can add it if they think it's wise

> Another issue: Is possible to define more than one author of the package?

Well, you can put several names in the author field but just one email
in author_email. And this email has to
be the one in your PyPI account if you push your package there.

Notice that there's also the maintainer field.

>
> -------------------------------------
> Carlos Tejo Alonso
> Research & Development Department
> CTIC Foundation [Asturias, Spain]
> www.fundacionctic.org
> -------------------------------------
>



-- 
Tarek Ziad? | http://ziade.org

From Fadhley.Salim at uk.calyon.com  Thu Apr 16 18:36:46 2009
From: Fadhley.Salim at uk.calyon.com (Fadhley Salim)
Date: Thu, 16 Apr 2009 17:36:46 +0100
Subject: [Distutils] Building eggs of Numpy for Python 2.4 on Win32
Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204484080@MXCU10MX1.MSX.CIB>

Has anybody managed to build an egg from the Numpy official release
binaries? The Numpy developers recon that it's possible to do so. My
first attempt was not succsessfull:
 
D:\workspace\numpy>easy_install -zm
numpy-1.2.1-win32-superpack-python2.4.exe
Processing numpy-1.2.1-win32-superpack-python2.4.exe
error: numpy-1.2.1-win32-superpack-python2.4.exe is not a valid
distutils Windows .exe

Any ideas how I might be able to get this working?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090416/53bd0fc8/attachment-0001.htm>
-------------- next part --------------
This email does not create a legal relationship between any member of the 
Cr=E9dit Agricole group and the recipient or constitute investment advice. 
The content of this email should not be copied or disclosed (in whole or 
part) to any other person. It may contain information which is 
confidential, privileged or otherwise protected from disclosure. If you 
are not the intended recipient, you should notify us and delete it from 
your system. Emails may be monitored, are not secure and may be amended, 
destroyed or contain viruses and in communicating with us such conditions 
are accepted. Any content which does not relate to business matters is not 
endorsed by us.

Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des
Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire
in France and subject to limited regulation by the Financial Services Authority.
Details about the extent of our regulation by the Financial Services Authority
are available from us on request. Calyon is incorporated in France with limited
liability and registered in England & Wales. Registration number: FC008194.
Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA.
This message and/or any  attachments is intended for the sole use of its addressee.
If you are not the addressee, please immediately notify the sender and then destroy the message.
As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Cr?dit Agricole CIB.
All rights reserved.

Ce message et ses pi?ces jointes est destin? ? l'usage exclusif de son destinataire.
Si vous recevez ce message par erreur, merci d'en aviser imm?diatement l'exp?diteur et de le d?truire ensuite.
Le pr?sent message pouvant ?tre alt?r? ? notre insu, CALYON Cr?dit Agricole CIB ne peut pas ?tre engag? par son contenu.
Tous droits r?serv?s.

From wright at esrf.fr  Thu Apr 16 19:04:29 2009
From: wright at esrf.fr (Jon Wright)
Date: Thu, 16 Apr 2009 19:04:29 +0200
Subject: [Distutils] Building eggs of Numpy for Python 2.4 on Win32
In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204484080@MXCU10MX1.MSX.CIB>
References: <7F347D91614EBC48AA7540A2A76D3BB204484080@MXCU10MX1.MSX.CIB>
Message-ID: <49E7651D.6030600@esrf.fr>

Fadhley Salim wrote:
> Has anybody managed to build an egg from the Numpy official release 
> binaries? 


Unzip the superpack exe file (using something like 7-zip file manager). 
Choose one of the 3 exe installers inside there, extract it and use 
that. The nosse version probably works slowly on most machines.

Jon









From douglas at mayle.org  Thu Apr 16 19:03:28 2009
From: douglas at mayle.org (Douglas Mayle)
Date: Thu, 16 Apr 2009 13:03:28 -0400
Subject: [Distutils] Distutils and easy_install...
Message-ID: <7640101B-283D-47F9-83CB-12A5BA5CE881@mayle.org>

Hey everyone,
	I'm having an annoying problem and I was directed here to see if you  
knew what could be done.

	I'm using distutils for my package instead of setuptools because it's  
a command line app, and the half second that setup tools adds to each  
launch for pkg_resource scanning is unacceptable.  I use the scripts  
parameter, and it happily installs the script I expect and things are  
running along.  If I try to use easy_install to install the package,  
however, (and more importantly, if a user of mine does) it seems that  
setuptools is monkeypatching the distutils module and replacing  
setup.  This means that instead of just copying my script to bin,  
setuptools is creating it's own script that does a pkg_resource scan  
and then loads my script from the original location.  Is there any way  
to ensure that I'm using the distutils.core.setup that I expect, and  
not the one that setuptools monkeypatches into place?

Thanks,
Douglas Mayle

From pje at telecommunity.com  Thu Apr 16 20:32:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 16 Apr 2009 14:32:26 -0400
Subject: [Distutils] Distutils and easy_install...
In-Reply-To: <7640101B-283D-47F9-83CB-12A5BA5CE881@mayle.org>
References: <7640101B-283D-47F9-83CB-12A5BA5CE881@mayle.org>
Message-ID: <20090416182956.7C8603A4100@sparrow.telecommunity.com>

At 01:03 PM 4/16/2009 -0400, Douglas Mayle wrote:
>Hey everyone,
>         I'm having an annoying problem and I was directed here to 
> see if you
>knew what could be done.
>
>         I'm using distutils for my package instead of setuptools 
> because it's
>a command line app, and the half second that setup tools adds to each
>launch for pkg_resource scanning is unacceptable.  I use the scripts
>parameter, and it happily installs the script I expect and things are
>running along.  If I try to use easy_install to install the package,
>however, (and more importantly, if a user of mine does) it seems that
>setuptools is monkeypatching the distutils module and replacing
>setup.  This means that instead of just copying my script to bin,
>setuptools is creating it's own script that does a pkg_resource scan
>and then loads my script from the original location.  Is there any way
>to ensure that I'm using the distutils.core.setup that I expect, and
>not the one that setuptools monkeypatches into place?

Use "easy_install -eb. MyPackage" to download the source (which will 
be placed in a 'mypackage/' subdirectory, then change to that 
directory and run "setup.py install" to install using distutils 
instead of setuptools.


From douglas at mayle.org  Thu Apr 16 21:40:07 2009
From: douglas at mayle.org (Douglas Mayle)
Date: Thu, 16 Apr 2009 15:40:07 -0400
Subject: [Distutils] Distutils and easy_install...
In-Reply-To: <20090416182956.7C8603A4100@sparrow.telecommunity.com>
References: <7640101B-283D-47F9-83CB-12A5BA5CE881@mayle.org>
	<20090416182956.7C8603A4100@sparrow.telecommunity.com>
Message-ID: <5E8847D3-7B09-4647-8FEF-D6E1889DBA20@mayle.org>

Sorry, I meant to send this onlist.  Reposting:

In the end, after some great suggestions and debugging help from Ian  
Bicking, I managed to monkeypatch the monkeypatch to restore my  
original script.  I was using a hybrid approach (sometimes importing  
distutils, sometimes setuptools) to avoid the script handling, but  
with this workaround, I should be able just to switch back to  
setuptools so that I don't have to think about it.

In my case, I have an install_script stub:

def install_script(self, dist, script_name, script_text, dev_path=None):
     self.write_script(script_name, script_text, 'b')

And I then monkeypatch setuptools if it's loaded:

if sys.platform != 'win32' and 'setuptools' in sys.modules:
     # Someone used easy_install to run this.  I really want the correct
     # script installed.
     import setuptools.command.easy_install
     setuptools.command.easy_install.easy_install.install_script =  
install_script

You'll notice that I leave the patching alone on win32, because I  
understand the desirability of having an exe file there...

Thanks,
Douglas Mayle

On Apr 16, 2009, at 2:32 PM, P.J. Eby wrote:

> At 01:03 PM 4/16/2009 -0400, Douglas Mayle wrote:
>> Hey everyone,
>>        I'm having an annoying problem and I was directed here to  
>> see if you
>> knew what could be done.
>>
>>        I'm using distutils for my package instead of setuptools  
>> because it's
>> a command line app, and the half second that setup tools adds to each
>> launch for pkg_resource scanning is unacceptable.  I use the scripts
>> parameter, and it happily installs the script I expect and things are
>> running along.  If I try to use easy_install to install the package,
>> however, (and more importantly, if a user of mine does) it seems that
>> setuptools is monkeypatching the distutils module and replacing
>> setup.  This means that instead of just copying my script to bin,
>> setuptools is creating it's own script that does a pkg_resource scan
>> and then loads my script from the original location.  Is there any  
>> way
>> to ensure that I'm using the distutils.core.setup that I expect, and
>> not the one that setuptools monkeypatches into place?
>
> Use "easy_install -eb. MyPackage" to download the source (which will  
> be placed in a 'mypackage/' subdirectory, then change to that  
> directory and run "setup.py install" to install using distutils  
> instead of setuptools.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090416/83738339/attachment.htm>

From martin at v.loewis.de  Thu Apr 16 23:08:00 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 16 Apr 2009 23:08:00 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
Message-ID: <49E79E30.4000904@v.loewis.de>

> I don't see it in the list (just LGPL without any version detail), I
> am ccing this request to catalog-sig so
> they can add it if they think it's wise

How should this be done? As a separate classifier with the suffix v3,
or as a subclassifier of LGPL?

Regards,
Martin

From ziade.tarek at gmail.com  Thu Apr 16 23:41:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 16 Apr 2009 23:41:08 +0200
Subject: [Distutils] Fixing the mess in sdist/egg_info
Message-ID: <94bdd2610904161441k57df0918v62b5f3ddc53532bf@mail.gmail.com>

Hi,

I am back on that problem with the code that builds the file list. The
current Distutils trunk isn't working anymore with setuptools because
of a recursive loop:

distutils.sdist.run() -> setuptools.build_py.data_files ->
setuptools.egg_info.run() -> distutils.sdist.add_defaults() ->
setuptools.build_py.data_files -> etc

The mess is introduced by the fact that build_py is patched by
setuptools in order to inject the files that are provided by the
(D)VCS.
But it also uses some APIs provided by sdist to complete the file list.

In order to solve this, we need to define clearly what is the role of
each command, and make sure it's a distinct role.

which is not the case right now :

1/ distutils.sdist  = this command is used to build a source distribution
2/ setuptools.egg_info = this command is used to build an .egg-info
directory but *also* injects the files founded with (D)VCS in the
MANIFEST
3/ distutils.build_py = this command is used to build pure python
module but *also* to find all the .py files to include in a
distribution (used by sdist).   In fact, it plays a central role in
sdist to get the files to include.

Here's a first thaught to discuss:

what about introducing a new simple command called "manifest" that
could be used by sdist or any other command, and that would be
responsible of collecting the files that are part of the distribution
(and nothing else). This command would generate the MANIFEST file and
also provide the APIs to get the files included into MANIFEST.

This command would introduce and use a simple plugin system similar to
the "setuptools.file_finders" entry points setuptools has for (D)VCS
files collecting. But with the files list being built as an argument
to the plugin. (partly described here
http://wiki.python.org/moin/Distutils/ManifestPluginSystem) so
Distutils, setuptools or any third party tool can register some code
that add or remove file in this file list.

The manifest command would provide default plugins, and setuptools
could refactor part of its code to use the manifest command rather
than calling sdist APIs. The goal would be to have the same result at
the end, but make it simpler to extend, and avoid command
interdependencies like what we have today (and that makes it hard to
maitain and make evolve).

For instance the MANIFEST.in templating system would be one of the
default plugin provided by Distutils.

The initial work would consist of refactoring the current code that
gets the files, using different strategies, into plugins for the new
manifest command, then make sdist uses this command as a subcommand.

The second phase would consist of working at setuptools level to use
the same technique. Setuptools would be able to make existing
"setuptools.file_finders" entry points work with Distutils manifest
command registery by providing a bridge.

Now for the plugin system, I see two options so far:

1/ create a simple ad-hoc plugin system in Distutils, and declare the
plugins in a new section in distutils.cfg / pydistutils.cfg for
loading them (setuptools existing entry points would be collected
through a unique plugin declared that would act like a bridge)

2/ use entry points (so add them into Distutils) and define a entry
point name based on the command name, maybe
"distutils:metadata.file_finders"  so the plugin system could be used
elsewhere in distutils.

Opinions ?

Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Apr 16 23:49:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 16 Apr 2009 23:49:11 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <49E79E30.4000904@v.loewis.de>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
	<49E79E30.4000904@v.loewis.de>
Message-ID: <94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>

2009/4/16 "Martin v. L?wis" <martin at v.loewis.de>:
>> I don't see it in the list (just LGPL without any version detail), I
>> am ccing this request to catalog-sig so
>> they can add it if they think it's wise
>
> How should this be done? As a separate classifier with the suffix v3,
> or as a subclassifier of LGPL?
>

Looking at others (Mozilla Public License) I would go for a separate
one with the suffix.

But what about LGPL 2 and 2.1 (It seems that 2.1 is introduces a lot
of changes) ?

Maybe "LGPL2+" would be better for the 2.x series (and maybe 2+ includes v3 ?)

Maybe we could ask someone at the FSF so we have the best versions in
our Trove classifier.

Regards
Tarek

> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From a.badger at gmail.com  Fri Apr 17 00:02:22 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 16 Apr 2009 15:02:22 -0700
Subject: [Distutils] Fixing the mess in sdist/egg_info
In-Reply-To: <94bdd2610904161441k57df0918v62b5f3ddc53532bf@mail.gmail.com>
References: <94bdd2610904161441k57df0918v62b5f3ddc53532bf@mail.gmail.com>
Message-ID: <49E7AAEE.1000600@gmail.com>

Tarek Ziad? wrote:
> Hi,
> 
> I am back on that problem with the code that builds the file list. The
> current Distutils trunk isn't working anymore with setuptools because
> of a recursive loop:
> 
> distutils.sdist.run() -> setuptools.build_py.data_files ->
> setuptools.egg_info.run() -> distutils.sdist.add_defaults() ->
> setuptools.build_py.data_files -> etc
> 
> The mess is introduced by the fact that build_py is patched by
> setuptools in order to inject the files that are provided by the
> (D)VCS.
> But it also uses some APIs provided by sdist to complete the file list.
> 
> In order to solve this, we need to define clearly what is the role of
> each command, and make sure it's a distinct role.
> 
> which is not the case right now :
> 
> 1/ distutils.sdist  = this command is used to build a source distribution
> 2/ setuptools.egg_info = this command is used to build an .egg-info
> directory but *also* injects the files founded with (D)VCS in the
> MANIFEST
> 3/ distutils.build_py = this command is used to build pure python
> module but *also* to find all the .py files to include in a
> distribution (used by sdist).   In fact, it plays a central role in
> sdist to get the files to include.
> 
> Here's a first thaught to discuss:
> 
> what about introducing a new simple command called "manifest" that
> could be used by sdist or any other command, and that would be
> responsible of collecting the files that are part of the distribution
> (and nothing else). This command would generate the MANIFEST file and
> also provide the APIs to get the files included into MANIFEST.
> 
> This command would introduce and use a simple plugin system similar to
> the "setuptools.file_finders" entry points setuptools has for (D)VCS
> files collecting. But with the files list being built as an argument
> to the plugin. (partly described here
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem) so
> Distutils, setuptools or any third party tool can register some code
> that add or remove file in this file list.
> 
> The manifest command would provide default plugins, and setuptools
> could refactor part of its code to use the manifest command rather
> than calling sdist APIs. The goal would be to have the same result at
> the end, but make it simpler to extend, and avoid command
> interdependencies like what we have today (and that makes it hard to
> maitain and make evolve).
> 
> For instance the MANIFEST.in templating system would be one of the
> default plugin provided by Distutils.
> 
> The initial work would consist of refactoring the current code that
> gets the files, using different strategies, into plugins for the new
> manifest command, then make sdist uses this command as a subcommand.
> 
I think this is good stuff.  You might find that the pieces that collect
files for the MANIFEST are useful in more than creating the MANIFEST,
though.  Unless I've missed a message saying that we're going to
collapse sdist and package-installed-files together.

> The second phase would consist of working at setuptools level to use
> the same technique. Setuptools would be able to make existing
> "setuptools.file_finders" entry points work with Distutils manifest
> command registery by providing a bridge.
> 
> Now for the plugin system, I see two options so far:
> 
> 1/ create a simple ad-hoc plugin system in Distutils, and declare the
> plugins in a new section in distutils.cfg / pydistutils.cfg for
> loading them (setuptools existing entry points would be collected
> through a unique plugin declared that would act like a bridge)
> 
> 2/ use entry points (so add them into Distutils) and define a entry
> point name based on the command name, maybe
> "distutils:metadata.file_finders"  so the plugin system could be used
> elsewhere in distutils.
> 
Having a library that makes creating plugins easy is a good general
purpose thing.  Whatever plugin system is adopted/created, it would be
good for it to not be internal to distutils.  I've always hated that
distutils option handling is different from the option handling that a
coder can use from the standard library.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090416/a4f57a3a/attachment.pgp>

From ziade.tarek at gmail.com  Fri Apr 17 00:08:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 17 Apr 2009 00:08:22 +0200
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
In-Reply-To: <A056BBAC-EFC6-45A2-AC05-499A3AB075CE@bud.ca>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
	<A056BBAC-EFC6-45A2-AC05-499A3AB075CE@bud.ca>
Message-ID: <94bdd2610904161508y3df9526fib79330d9a9f28a89@mail.gmail.com>

On Wed, Apr 15, 2009 at 5:40 AM, Kevin Teague <kevin at bud.ca> wrote:
> -1 for install/uninstall scripts in Distutils
>
> I'd argue that the scope of Distutils is already wide enough that it doesn't
> need to be extended to also be a "package manager" -- even if it's a really
> simple one.
>
> If a install/uninstall tool does go into Python, I'd rather see it as
> something like 'simplepackageinstaller.install' instead of
> 'Distutils.scripts.install'. This would also make it more clear that this
> tool is simply working with the standard packaging formats and tools, and
> doesn't muddy the format/implementation as much as if it's just another part
> of Distutils.
>
> But then I don't think Python should have a built-in installer or package
> manager. There are excellent tools already available (Buildout, pip, dpkg,
> RPM), it would be better if we guided people to these tools and let them
> pick the right one for their installation use case.

I wouldn't put zc.buildout in the same level than pip or easy_install.
and I guess what we would have in DIstutils would be quite similar to
what easy_install
or pip offers.

>
> If there was a installer, I'm assuming it'd be quite a simple one - e.g.
> installs single-version into site-packages. This caters to well to casual
> user -- they can just run a "standard" command out-of-the-box and take-off
> running with a distribution, but it also teaches them bad habits (e.g. that
> you want to be commonly installing into site-packages or that you want to
> develop your own code without properly expressing it's dependencies). When
> they want to use better development practices, they'll have to switch to a
> "non-standard" tools to do "non-standard" installations.
>

For the "site-package" part, this is true, but so wrong. Many people
in this mailing list (and in real life) agrees that it's
wrong to install a package in site-packages.

So basically you are saying that a fresh Python installation teaches
people bad habits
because they end up installing packages in site-packages. And that
they should install
third party packages to have better practices. (eg like virtualenv or
zc.buildout)

It doesn't sound right. Imho Python shoulld provide multiple versions
for the same package.
and change its importer.

-- 
Tarek Ziad? | http://ziade.org

From a.badger at gmail.com  Fri Apr 17 00:10:21 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 16 Apr 2009 15:10:21 -0700
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
Message-ID: <49E7ACCD.8000608@gmail.com>

Tarek Ziad? wrote:
> Hello,
> 
> This is a side discussion but quiet important ihmo.
> 
> == Problem ==
> 
> Some people complained about the fact that is was hard to extend
> Distutils commands.
> You end up rewriting the whole command most of the time.
> 
> So what's a command ? It's a class that is used by the distribution
> instance when you run Distutils.
> 
> roughly:
> 
> cmd = Command(distribution)
> cmd.initialize_options()
> cmd.finalize_options()        <---  allows to check the options if
> subcommands where run
> cmd.run()                            <---  runs the code
> 
> each command can define sub commands, but most of the time it's a
> harcoded list, so you need to inherit the command
> if you want to add a new behavior.
> 
> == work in progress, ==
> 
> What we want to do here is being able to define subsets in run(),
> sharing the same options environment.
> 
> so basically, a rough, generic run() method could be:
> 
> def run():
>    for func in some_funcs:
>       func(self, options)
> 
> If "some_funcs" could be defined by a registery with simple names,
> anyone could provide new functions
> and configure the registery to run a sequence of function.
> 
> Given a command name, Distutils can get this list of function, through
> a registery.
> Each function could register itself into Distutils, like in what I
> have started to work here for the manifest file:
> see http://wiki.python.org/moin/Distutils/ManifestPluginSystem
> 
> The ordering would be configurable through the setup.cfg file.
> 
> Any opinion, idea for this part ?
> 
Have you looked at paver?  It's syntax makes extension easy.

@task
def run():
    function1()
    function2()
    function3()

or
@task
@needs([function1, function2, function3])
def run():
    pass

So if I want to do everything that setuptools.command.install does and
also install locale files using my own function I do:

@task
def install_locales():
    # My code to install locales goes here
    pass

@task
def install():
    # A new install task that overrides the system install task
    # Note that I control ordering just by changing the order
    # subtasks get called
    call_task('setuptools.command.install')
    call_task('install_locales')

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090416/820371e8/attachment.pgp>

From ianb at colorstudy.com  Fri Apr 17 01:16:09 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 16 Apr 2009 18:16:09 -0500
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
In-Reply-To: <94bdd2610904161508y3df9526fib79330d9a9f28a89@mail.gmail.com>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com> 
	<A056BBAC-EFC6-45A2-AC05-499A3AB075CE@bud.ca>
	<94bdd2610904161508y3df9526fib79330d9a9f28a89@mail.gmail.com>
Message-ID: <b654cd2e0904161616t6a15130eta6260c69d5ae9e0c@mail.gmail.com>

OK, just thinking through a little what it would mean to have an
installation tool in Python core...

On Thu, Apr 16, 2009 at 5:08 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> > But then I don't think Python should have a built-in installer or package
> > manager. There are excellent tools already available (Buildout, pip,
> dpkg,
> > RPM), it would be better if we guided people to these tools and let them
> > pick the right one for their installation use case.
>
> I wouldn't put zc.buildout in the same level than pip or easy_install.
> and I guess what we would have in DIstutils would be quite similar to
> what easy_install or pip offers.
>

I think there are questions about scope.  zc.buildout does more than pip,
and pip does more than easy_install.  I think easy_install has some
important usability problems, otherwise I wouldn't have written pip.  pip on
the other hand has features that extend its scope in ways that might make it
hard to include in the standard library.  For instance the version control
support, requirement files, bundles, and some miscellaneous functionality
like zipping.  Some of that could be separated out, though the version
control support is more difficult.

Also it really mostly makes sense in the context of virtualenv.  I'm
strongly considering having an interactive warning if you try to install
something with pip into the global site packages directory.  PYTHONUSERBASE
is an okay solution (not great, but okay), so I don't think this is
contingent on something like virtualenv being standard... but it would
help.  I'm not sure how I'd pursue the virtualenv concept in Python core, as
it's more a question of the concept of virtualenv than the implementation
itself, but if there was general interest I could try.

Anyway, it seems odd to me to include a tool that hasn't been written yet,
when there are tools that are being used and developed.  OTOH, the only tool
that is stable enough (not just bug wise, but stable as in isn't-changing)
is easy_install.  But while pip uses setuptools, it doesn't use easy_install
at all, so including easy_install would really only make pip development
harder.

That said, there might be parts of pip or easy_install which would be
useful.  I'm not sure what those would be though.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090416/aa5dd064/attachment-0001.htm>

From renesd at gmail.com  Fri Apr 17 04:15:53 2009
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 17 Apr 2009 12:15:53 +1000
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
	<49E79E30.4000904@v.loewis.de>
	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>
Message-ID: <64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com>

hellos,

Is it just me, or are these classifiers not as good as tags?

Should people really have to discuss and get approved what tags they put on
their software?

Seems like a big waste of everyones time, and doesn't result in as good a
database.

cheers,


2009/4/17 Tarek Ziad? <ziade.tarek at gmail.com>

> 2009/4/16 "Martin v. L?wis" <martin at v.loewis.de>:
> >> I don't see it in the list (just LGPL without any version detail), I
> >> am ccing this request to catalog-sig so
> >> they can add it if they think it's wise
> >
> > How should this be done? As a separate classifier with the suffix v3,
> > or as a subclassifier of LGPL?
> >
>
> Looking at others (Mozilla Public License) I would go for a separate
> one with the suffix.
>
> But what about LGPL 2 and 2.1 (It seems that 2.1 is introduces a lot
> of changes) ?
>
> Maybe "LGPL2+" would be better for the 2.x series (and maybe 2+ includes v3
> ?)
>
> Maybe we could ask someone at the FSF so we have the best versions in
> our Trove classifier.
>
> Regards
> Tarek
>
> > Regards,
> > Martin
> >
>
>
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090417/d7d3bac1/attachment.htm>

From david at ar.media.kyoto-u.ac.jp  Fri Apr 17 04:40:20 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 17 Apr 2009 11:40:20 +0900
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <49E7ACCD.8000608@gmail.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<49E7ACCD.8000608@gmail.com>
Message-ID: <49E7EC14.90208@ar.media.kyoto-u.ac.jp>

Toshio Kuratomi wrote:
> Tarek Ziad? wrote:
>   
>> Hello,
>>
>> This is a side discussion but quiet important ihmo.
>>
>> == Problem ==
>>
>> Some people complained about the fact that is was hard to extend
>> Distutils commands.
>> You end up rewriting the whole command most of the time.
>>
>> So what's a command ? It's a class that is used by the distribution
>> instance when you run Distutils.
>>
>> roughly:
>>
>> cmd = Command(distribution)
>> cmd.initialize_options()
>> cmd.finalize_options()        <---  allows to check the options if
>> subcommands where run
>> cmd.run()                            <---  runs the code
>>
>> each command can define sub commands, but most of the time it's a
>> harcoded list, so you need to inherit the command
>> if you want to add a new behavior.
>>
>> == work in progress, ==
>>
>> What we want to do here is being able to define subsets in run(),
>> sharing the same options environment.
>>
>> so basically, a rough, generic run() method could be:
>>
>> def run():
>>    for func in some_funcs:
>>       func(self, options)
>>
>> If "some_funcs" could be defined by a registery with simple names,
>> anyone could provide new functions
>> and configure the registery to run a sequence of function.
>>
>> Given a command name, Distutils can get this list of function, through
>> a registery.
>> Each function could register itself into Distutils, like in what I
>> have started to work here for the manifest file:
>> see http://wiki.python.org/moin/Distutils/ManifestPluginSystem
>>
>> The ordering would be configurable through the setup.cfg file.
>>
>> Any opinion, idea for this part ?
>>
>>     
> Have you looked at paver?  It's syntax makes extension easy.
>   

Yes, paver is nice, but I don't think it solves the problem that Tarek
aimed at solving here. Let me introduce some usercases. For example, if
you want to extend say the distutils sdist command, you could do:

@task
@needs(['distutils.command.sdist'])
def mysdist():
    # Add some more stuff here to the tarball
    ...

But what if you want to alter the original sdist behavior, e.g. place
some files elsewhere ? Paver works nice if you want to extend the
existing behavior in a simple way (after doing this, do that), but
unfortunately, some things do not fit well if at all in this scheme.

I would cite two examples, related to sdist and paver:
    -
http://groups.google.com/group/paver/browse_thread/thread/581534ca19cc541e
    -
http://groups.google.com/group/paver/browse_thread/thread/e153ba3da8cc3334

There are also many things which are annoying once you start using paver
for cross-platform deployment, because of distutils commands. For
example, you may need to move some distributions files (tarball,
binaries installers, doc) somewhere else. Even though the code used to
generate the files are in python, there is currently no way to use that
code outside distutils - things as simple as platform specific names of
the tarballs, etc... are not factored out, and you have to recreate it
in your own scripts.

So the current problem with distutils commands is not just how to easily
extend them - sometimes, you need to alter them as well, hence Tarek's
suggestion for plugins/registry.

cheers,

David

From yanegomi at gmail.com  Fri Apr 17 09:28:09 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Fri, 17 Apr 2009 00:28:09 -0700
Subject: [Distutils] multiversioning __import__ and modules?
Message-ID: <364299f40904170028s43f780cbl2c75f70210247103@mail.gmail.com>

Hi guys,
    I realize this may not look like a distutils question, but it's
closely tied to distutils and setuptools.
    I did some exploring today and it appears that there isn't a
simple way to profile different versioned packages and bootstrap one
version as the master version without going through the trouble of
implementing a lot of what virtualenv does.
    What I thought might be a good idea (and I'm just tossing it out
to see what others might think), is to apply logic to __import__ that
is similar to setuptools.pkg_resources.require() when importing
packages. Here's the new API I was thinking of:

    __import__(name[, globals[, locals[, fromlist[,
level][,version=None][,requires=expr]]]]])?

    expr can be anything of the form described under
<http://docs.python.org/distutils/setupscript.html#relationships-between-distributions-and-packages>:

    Here are some explicit examples using the new API:

    1. PackageA v2.0 requires PackageB v1.5:

    __import__('PackageA', version="2.0", requires="PackageB==1.5")

    2. PackageC v2.1 requires any PackageD version between v1.6 and
v1.8, minus v1.7 (buggy version):

    __import__('PackageC', version="2.1", requires="PackageD >=1.6,
!=1.7, <=1.8")

    3. PackageE v0.1.0 requires any PackageF version less than v0.1.10:

    __import__('PackageE', version="0.1.0", requires="PackageF<0.1.10")

    4. PackageG requires any version of PackageH:

    __import__('PackageG', requires="PackageH")

    Thoughts?
Thanks,
-Garrett

From yanegomi at gmail.com  Fri Apr 17 09:39:51 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Fri, 17 Apr 2009 00:39:51 -0700
Subject: [Distutils] multiversioning __import__ and modules?
In-Reply-To: <364299f40904170028s43f780cbl2c75f70210247103@mail.gmail.com>
References: <364299f40904170028s43f780cbl2c75f70210247103@mail.gmail.com>
Message-ID: <364299f40904170039y2362218cx19592e92f885a93c@mail.gmail.com>

On Fri, Apr 17, 2009 at 12:28 AM, Garrett Cooper <yanegomi at gmail.com> wrote:
> Hi guys,
> ? ?I realize this may not look like a distutils question, but it's
> closely tied to distutils and setuptools.
> ? ?I did some exploring today and it appears that there isn't a
> simple way to profile different versioned packages and bootstrap one
> version as the master version without going through the trouble of
> implementing a lot of what virtualenv does.
> ? ?What I thought might be a good idea (and I'm just tossing it out
> to see what others might think), is to apply logic to __import__ that
> is similar to setuptools.pkg_resources.require() when importing
> packages. Here's the new API I was thinking of:
>
> ? ?__import__(name[, globals[, locals[, fromlist[,
> level][,version=None][,requires=expr]]]]])?
>
> ? ?expr can be anything of the form described under
> <http://docs.python.org/distutils/setupscript.html#relationships-between-distributions-and-packages>:
>
> ? ?Here are some explicit examples using the new API:
>
> ? ?1. PackageA v2.0 requires PackageB v1.5:
>
> ? ?__import__('PackageA', version="2.0", requires="PackageB==1.5")
>
> ? ?2. PackageC v2.1 requires any PackageD version between v1.6 and
> v1.8, minus v1.7 (buggy version):
>
> ? ?__import__('PackageC', version="2.1", requires="PackageD >=1.6,
> !=1.7, <=1.8")
>
> ? ?3. PackageE v0.1.0 requires any PackageF version less than v0.1.10:
>
> ? ?__import__('PackageE', version="0.1.0", requires="PackageF<0.1.10")
>
> ? ?4. PackageG requires any version of PackageH:
>
> ? ?__import__('PackageG', requires="PackageH")
>
> ? ?Thoughts?
> Thanks,
> -Garrett
>

Sorry... I lost track of my original bright idea until now. What I
meant was something more along the lines of:

__import__(name[, globals[, locals[, fromlist[,level][,version=None]]]]])?

The modified examples:

1. Importer requires PackageA v1.5:

__import__('PackageA', version="==1.5")

2. Importer requires any version of PackageB between v1.6 and v1.8,
minus v1.7 (buggy version):

__import__('PackageB', version=">=1.6, !=1.7, <=1.8")

3. Importer requires any PackageC version less than v0.1.10:

__import__('PackageC', version="<0.1.10")

4. Importer requires any version of PackageD:

__import__('PackageD')

Thanks,
-Garrett

From a.badger at gmail.com  Fri Apr 17 11:19:02 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Fri, 17 Apr 2009 02:19:02 -0700
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <49E7EC14.90208@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<49E7ACCD.8000608@gmail.com>
	<49E7EC14.90208@ar.media.kyoto-u.ac.jp>
Message-ID: <49E84986.7090207@gmail.com>

David Cournapeau wrote:
> Toshio Kuratomi wrote:
>> Tarek Ziad? wrote:
>>   
>>> Hello,
>>>
>>> This is a side discussion but quiet important ihmo.
>>>
>>> == Problem ==
>>>
>>> Some people complained about the fact that is was hard to extend
>>> Distutils commands.
>>> You end up rewriting the whole command most of the time.
>>>
>>> So what's a command ? It's a class that is used by the distribution
>>> instance when you run Distutils.
>>>
>>> roughly:
>>>
>>> cmd = Command(distribution)
>>> cmd.initialize_options()
>>> cmd.finalize_options()        <---  allows to check the options if
>>> subcommands where run
>>> cmd.run()                            <---  runs the code
>>>
>>> each command can define sub commands, but most of the time it's a
>>> harcoded list, so you need to inherit the command
>>> if you want to add a new behavior.
>>>
>>> == work in progress, ==
>>>
>>> What we want to do here is being able to define subsets in run(),
>>> sharing the same options environment.
>>>
>>> so basically, a rough, generic run() method could be:
>>>
>>> def run():
>>>    for func in some_funcs:
>>>       func(self, options)
>>>
>>> If "some_funcs" could be defined by a registery with simple names,
>>> anyone could provide new functions
>>> and configure the registery to run a sequence of function.
>>>
>>> Given a command name, Distutils can get this list of function, through
>>> a registery.
>>> Each function could register itself into Distutils, like in what I
>>> have started to work here for the manifest file:
>>> see http://wiki.python.org/moin/Distutils/ManifestPluginSystem
>>>
>>> The ordering would be configurable through the setup.cfg file.
>>>
>>> Any opinion, idea for this part ?
>>>
>>>     
>> Have you looked at paver?  It's syntax makes extension easy.
>>   
> 
> Yes, paver is nice, but I don't think it solves the problem that Tarek
> aimed at solving here. Let me introduce some usercases. For example, if
> you want to extend say the distutils sdist command, you could do:
> 
> @task
> @needs(['distutils.command.sdist'])
> def mysdist():
>     # Add some more stuff here to the tarball
>     ...
> 
> But what if you want to alter the original sdist behavior, e.g. place
> some files elsewhere ? Paver works nice if you want to extend the
> existing behavior in a simple way (after doing this, do that), but
> unfortunately, some things do not fit well if at all in this scheme.
> 
@task
@needs(['distutils.command.sdist'])
def sdist():
	# Place some files in an alternate location


> I would cite two examples, related to sdist and paver:
>     -
> http://groups.google.com/group/paver/browse_thread/thread/581534ca19cc541e
>     -
> http://groups.google.com/group/paver/browse_thread/thread/e153ba3da8cc3334
> 
those are both bugs in paver.  (Whether they're resolvable or not within
paver, I don't know.)  They don't have bearing on talking about
redesigning how to design a new architecture that's easy to extend.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090417/67479819/attachment-0001.pgp>

From david at ar.media.kyoto-u.ac.jp  Fri Apr 17 11:21:26 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 17 Apr 2009 18:21:26 +0900
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <49E84986.7090207@gmail.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<49E7ACCD.8000608@gmail.com>
	<49E7EC14.90208@ar.media.kyoto-u.ac.jp>
	<49E84986.7090207@gmail.com>
Message-ID: <49E84A16.7010902@ar.media.kyoto-u.ac.jp>

Toshio Kuratomi wrote:
> They don't have bearing on talking about
> redesigning how to design a new architecture that's easy to extend.
>   

Those examples show why extending distutils commands with subclassing +
post processing is not always enough. I don't understand why they would
not be relevant for the design to improve distutils extensibility. They
are quite typical of the usual problems I have when I need to extend
distutils myself.

David

From david at ar.media.kyoto-u.ac.jp  Fri Apr 17 13:54:23 2009
From: david at ar.media.kyoto-u.ac.jp (cdavid)
Date: Fri, 17 Apr 2009 04:54:23 -0700 (PDT)
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
Message-ID: <23096059.post@talk.nabble.com>


Hi Tarek,


Tarek Ziad? wrote:
> 
> 
> == work in progress, ==
> 
> What we want to do here is being able to define subsets in run(),
> sharing the same options environment.
> 
> so basically, a rough, generic run() method could be:
> 
> def run():
>    for func in some_funcs:
>       func(self, options)
> 
> 

What exactly is options here ? The class instance member user_options or
something else (a dictionary of options common to each function, a bit like
environment variables in scons or waf).

A related problem, but maybe outside the scope of your proposal is dealing
with communication between commands. I think the only way to do it at the
moment is to attach those data to the Distribution instance - that's very
fragile (if only because every new distutils-related tool has its own
distribution class, so you have to special case for every one of them). For
me, that's one of the main issue in distutils: extending distutils without
breaking other tools (paver/setuptools).

I think we should also working with precise examples/usecases right away -
in my own experience at least, the difficulties with distutils are mostly
implementation details. I will work on a few examples which I found quite
painful to implement while working on the numpy build system and add them to
the wiki, to help the discussion,

cheers,

David
-- 
View this message in context: http://www.nabble.com/Making-commands-extensible-by-default-tp22978698p23096059.html
Sent from the Python - distutils-sig mailing list archive at Nabble.com.


From ziade.tarek at gmail.com  Fri Apr 17 19:38:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 17 Apr 2009 19:38:38 +0200
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <23096059.post@talk.nabble.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<23096059.post@talk.nabble.com>
Message-ID: <94bdd2610904171038q2ef3c5b9vca08539c9c2489af@mail.gmail.com>

On Fri, Apr 17, 2009 at 1:54 PM, cdavid <david at ar.media.kyoto-u.ac.jp> wrote:
>
> A related problem, but maybe outside the scope of your proposal is dealing
> with communication between commands. I think the only way to do it at the
> moment is to attach those data to the Distribution instance - that's very
> fragile (if only because every new distutils-related tool has its own
> distribution class, so you have to special case for every one of them). For
> me, that's one of the main issue in distutils: extending distutils without
> breaking other tools (paver/setuptools).
>
> I think we should also working with precise examples/usecases right away -
> in my own experience at least, the difficulties with distutils are mostly
> implementation details. I will work on a few examples which I found quite
> painful to implement while working on the numpy build system and add them to
> the wiki, to help the discussion,
>

ok great, thx. I have myself several use cases and I think I might
have a working solution on the paper
for the extension of existing commands.

a new kind of option you can use in any command, but wich is not a
simple type like a boolean
or a string.  It's a class that would works with plugins.

class MyCmd(Command):
    foo = ExtensibleOption('foo')

from there, the command would be able to ask in its code the option
some values, by
calling some plugins with a generic signature :

for plugin in self.foo._get_plugins():
    values = plugin(self.distribution, self)

It's up to the command to define its extension points, and document them.

I am doing this code in a code base for another project, when I have something
running I'll get back here with it as a support for my example.

++
Tarek

-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Sat Apr 18 01:18:37 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 18 Apr 2009 01:18:37 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <gsa2fl$gs1$2@ger.gmane.org>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>	<49E79E30.4000904@v.loewis.de>	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>	<64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com>
	<gsa2fl$gs1$2@ger.gmane.org>
Message-ID: <49E90E4D.4050109@v.loewis.de>

> For licensing, I would say we could update the PEP to say that the
> 'licence' argument could be used to clarify the classifier (e.g., to
> spell the version).

That works for me.

Regards,
Martin

From regebro at gmail.com  Sat Apr 18 11:09:11 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 18 Apr 2009 11:09:11 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
Message-ID: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>

Setuptools non-support for Python 3 is currently a serious hindrance
towards Python 3 aceptance. I'm trying to figure out what to do as a
next step in the Python 3 support for setuptools. And I have
encountered some obstacles. The first one is that setuptools requires
itself for installing and running tests. That makes it hard to install
it under Python 3. There are various solutions to this, but the next
obstacle I encounter in choosing the right solution is that the code
is hard to understand, and it makes me want to just rip it out and
start over, or in even more frustrated moments, avoid the problems by
not using setuptools at all. But the third obstacle for that is that I
don't actually know what features of setuptools people use.

I personally use setuptools for these reasons:

1. When I create projects with paster, it uses setuptools.
2. Setuptools makes it possible to specify requirements, which is then
used by buildout.
3. Namespace packages require pkg_resources?
4. The test command.

What are the other major reasons people use setuptools?

Is there any good reason to not extract the namespace package support
into a separate package?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Sat Apr 18 11:23:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 18 Apr 2009 11:23:34 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
Message-ID: <94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>

On Sat, Apr 18, 2009 at 11:09 AM, Lennart Regebro <regebro at gmail.com> wrote:
> Setuptools non-support for Python 3 is currently a serious hindrance
> towards Python 3 aceptance. I'm trying to figure out what to do as a
> next step in the Python 3 support for setuptools. And I have
> encountered some obstacles. The first one is that setuptools requires
> itself for installing and running tests. That makes it hard to install
> it under Python 3. There are various solutions to this, but the next
> obstacle I encounter in choosing the right solution is that the code
> is hard to understand, and it makes me want to just rip it out and
> start over, or in even more frustrated moments, avoid the problems by
> not using setuptools at all. But the third obstacle for that is that I
> don't actually know what features of setuptools people use.
>
> I personally use setuptools for these reasons:
>
> 1. When I create projects with paster, it uses setuptools.
> 2. Setuptools makes it possible to specify requirements, which is then
> used by buildout.
> 3. Namespace packages require pkg_resources?
> 4. The test command.
>
> What are the other major reasons people use setuptools?
>

entry points (in the "how to extend commands" topic I might propose to
include it/use it but it would need to be separated
from pkg_resources)


> Is there any good reason to not extract the namespace package support
> into a separate package?

It's on its way http://svn.python.org/projects/peps/trunk/pep-0382.txt

and PEP 345 is being reworked to include requires
http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt

Anyways, you can probably help if you worked on pkg_resources, by
splitting all its components in pep-8-ied, readable elements
in your branch

Cheers
Tarek

>
> --
> Lennart Regebro: Python, Zope, Plone, Grok
> http://regebro.wordpress.com/
> +33 661 58 14 64
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From regebro at gmail.com  Sat Apr 18 11:41:49 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 18 Apr 2009 11:41:49 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
Message-ID: <319e029f0904180241t100a30e8lbc9616b836eff15d@mail.gmail.com>

On Sat, Apr 18, 2009 at 11:23, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> It's on its way http://svn.python.org/projects/peps/trunk/pep-0382.txt

Ah, OK, very good. I missed that one.

> and PEP 345 is being reworked to include requires
> http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt

Sure.

Do you have any feeling of when these could get released? 1 month? 6
months? 12 months? Five years? :-)

> Anyways, you can probably help if you worked on pkg_resources, by
> splitting all its components in pep-8-ied, readable elements
> in your branch

Heh. That would require me to actually *understand* the code. :-)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From tseaver at palladion.com  Sat Apr 18 13:19:31 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 18 Apr 2009 07:19:31 -0400
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
Message-ID: <gscd0j$b8n$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lennart Regebro wrote:
> Setuptools non-support for Python 3 is currently a serious hindrance
> towards Python 3 aceptance. I'm trying to figure out what to do as a
> next step in the Python 3 support for setuptools. And I have
> encountered some obstacles. The first one is that setuptools requires
> itself for installing and running tests. That makes it hard to install
> it under Python 3. There are various solutions to this, but the next
> obstacle I encounter in choosing the right solution is that the code
> is hard to understand, and it makes me want to just rip it out and
> start over, or in even more frustrated moments, avoid the problems by
> not using setuptools at all. But the third obstacle for that is that I
> don't actually know what features of setuptools people use.
> 
> I personally use setuptools for these reasons:
> 
> 1. When I create projects with paster, it uses setuptools.
> 2. Setuptools makes it possible to specify requirements, which is then
> used by buildout.
> 3. Namespace packages require pkg_resources?
> 4. The test command.
> 
> What are the other major reasons people use setuptools?

- - setuptools.find_pacakges built-in SVN support makes a whole class of
  packaging errors go away for me.

- - virtualenv makes isolation between different applications sane;
  it installs setuptools, and then makes 'easy_install' a pleasure
  to use (I still can't believe people use easy_install in their
  system Python!)

- - entry points serve as crude equivalents of "named utilities"
  in the Zope component architecture;  they allow an application to
  define a class of plug points, which are then filled by other
  libraries.  These plug points can then be configured together
  declaratively (e.g. in an INI file) using their names.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ6bdD+gerLs4ltQ4RApw2AKDO9zRTVw5BGYEd0NKCFWdwc0sbFACghRqW
8BqG80vsK6kS8OEcG+UFelw=
=fTfI
-----END PGP SIGNATURE-----


From floris.bruynooghe at gmail.com  Sat Apr 18 17:36:36 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 18 Apr 2009 16:36:36 +0100
Subject: [Distutils] setup.py build_ext --rpath behaviour
In-Reply-To: <94bdd2610904160354k6e9c5aa7yff2c1508a1cef512@mail.gmail.com>
References: <20090416104228.GA416@laurie.devork>
	<94bdd2610904160354k6e9c5aa7yff2c1508a1cef512@mail.gmail.com>
Message-ID: <20090418153636.GA31815@laurie.devork>

On Thu, Apr 16, 2009 at 12:54:02PM +0200, Tarek Ziad? wrote:
> On Thu, Apr 16, 2009 at 12:42 PM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com> wrote:
> > Do these improvements sound sensible?  And if so should I create one
> > patch for each (and two bug reports) or combine them into one patch?
>
> They sound reasonable,  please fill two differents bugs with your patches
> and if you can, add some unit tests.

Attached are new tests for the distutils.unixccompiler module, testing
the current behaviour of what I'm about to change (not testing the
entire module!).  Any comments on these tests would be appreciated,
just to make sure I'm not doing it all wrong.


> As long as you provide some support to test the final vesion on the various
> platforms (eg being a beta tester of the distutils trunk),
> I am able to work on your patches, and on the tests.

I should be able to test on some platforms (GNU/Linux-gcc,
Solaris-gcc, AIX-gcc).  Is the ditutils trunk still python 2.x trunk?


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_unixccompiler.py
Type: text/x-python
Size: 1816 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090418/12f89cbf/attachment.py>

From carlos.tejo at fundacionctic.org  Sat Apr 18 20:04:33 2009
From: carlos.tejo at fundacionctic.org (Carlos Tejo Alonso)
Date: Sat, 18 Apr 2009 20:04:33 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>	<49E79E30.4000904@v.loewis.de>	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>	<64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com>
	<gsa2fl$gs1$2@ger.gmane.org> <49E90E4D.4050109@v.loewis.de>
Message-ID: <09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>

>> For licensing, I would say we could update the PEP to say that the
>> 'licence' argument could be used to clarify the classifier (e.g., to
>> spell the version).
>
>That works for me.

>From my point of view, in order to achive that humans and machines could read by themselves the license of a package, why not create a proper classifier?

BTW, I asked today a friend who is involved in license issue and she explained me that: if the version of a license is not declared in a software product, that means that the license applied is the last one.

Regards,

Carlos Tejo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090418/b0ffae65/attachment.htm>

From regebro at gmail.com  Sat Apr 18 22:55:34 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 18 Apr 2009 22:55:34 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180241t100a30e8lbc9616b836eff15d@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
	<319e029f0904180241t100a30e8lbc9616b836eff15d@mail.gmail.com>
Message-ID: <319e029f0904181355w7dcaf237rfedf28747add1881@mail.gmail.com>

On Sat, Apr 18, 2009 at 11:41, Lennart Regebro <regebro at gmail.com> wrote:
> On Sat, Apr 18, 2009 at 11:23, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> It's on its way http://svn.python.org/projects/peps/trunk/pep-0382.txt
>
> Ah, OK, very good. I missed that one.

But.... it aims for Python 3.x and possible 2.7. That doesn't help me,
as I will need a method that can support at least 2.5 and 2.6 as well.
But then again, there is one already, and it seems the pkgutil version
of namespaces would work fine, except that it doens't support
Zip-files, which is OK, because they are a pain in the ass anyway.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From fdrake at gmail.com  Sun Apr 19 02:16:26 2009
From: fdrake at gmail.com (Fred Drake)
Date: Sat, 18 Apr 2009 20:16:26 -0400
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
	<49E79E30.4000904@v.loewis.de>
	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>
	<64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com>
	<gsa2fl$gs1$2@ger.gmane.org> <49E90E4D.4050109@v.loewis.de>
	<09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>
Message-ID: <9cee7ab80904181716t52949d77m46f67a9c747afd4e@mail.gmail.com>

2009/4/18 Carlos Tejo Alonso <carlos.tejo at fundacionctic.org>:
> BTW, I asked today a friend who is involved in license issue and she
> explained me that: if the version of a license is not declared in a software
> product, that means that the license applied is the last one.

The last one at the time of licensing or the last one at the time
someone comes back later and asks?

Either way, the answer will change depending on who you ask; there's
not necessarily exactly one answer.

The licensor is responsible for specifying the license; there's no
value in an unspecified version.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From carlos.tejo at fundacionctic.org  Sun Apr 19 11:11:23 2009
From: carlos.tejo at fundacionctic.org (Carlos Tejo Alonso)
Date: Sun, 19 Apr 2009 11:11:23 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org><94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com><09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org><94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com><49E79E30.4000904@v.loewis.de><94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com><64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com><gsa2fl$gs1$2@ger.gmane.org>
	<49E90E4D.4050109@v.loewis.de><09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>
	<9cee7ab80904181716t52949d77m46f67a9c747afd4e@mail.gmail.com>
Message-ID: <09700B613C4DD84FA9F2FEA52188281902BB1461@ayalga.fundacionctic.org>

>> BTW, I asked today a friend who is involved in license issue and she
>> explained me that: if the version of a license is not declared in a software
>> product, that means that the license applied is the last one.

>The last one at the time of licensing or the last one at the time
>someone comes back later and asks?

As my friend told me, this is an example:

2018 - LGPL 3.0 is released
2019 - Package X is licensed by LPGL (no version)
2020 - LPGL 4.0 is released
2021 - What's the license of the package X? LGPL 4.0

Regards,

Carlos Tejo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090419/c9d43f45/attachment-0001.htm>

From martin at v.loewis.de  Sun Apr 19 12:14:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Apr 2009 12:14:11 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <09700B613C4DD84FA9F2FEA52188281902BB1461@ayalga.fundacionctic.org>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org><94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com><09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org><94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com><49E79E30.4000904@v.loewis.de><94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com><64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com><gsa2fl$gs1$2@ger.gmane.org>
	<49E90E4D.4050109@v.loewis.de><09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>
	<9cee7ab80904181716t52949d77m46f67a9c747afd4e@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281902BB1461@ayalga.fundacionctic.org>
Message-ID: <49EAF973.5050104@v.loewis.de>

> 2018 - LGPL 3.0 is released
> 2019 - Package X is licensed by LPGL (no version)
> 2020 - LPGL 4.0 is released
> 2021 - What's the license of the package X? LGPL 4.0

IANAL, but I don't believe this example; in addition, I
consider it fairly artificial. The LGPL recommends that
you include a verbatim copy of it in your source
distribution; if you do so, it seems fairly clear that
the license that you specified is the very version that
you include with your code, even if you don't mention
a version number explicitly.

OTOH, if you then also include the following text in
the source files (which the LGPL suggests that you do),
then clearly, you explicitly make it the user's choice
to pick a version:

    This library is free software; you can redistribute it and/or
    modify it under the terms of the GNU Lesser General Public
    License as published by the Free Software Foundation; either
    version 2.1 of the License, or (at your option) any later version.

However, that wording is specific to the LGPL (and the GPL),
and does not apply to any other license.

Regards,
Martin

From ziade.tarek at gmail.com  Sun Apr 19 13:46:57 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 19 Apr 2009 13:46:57 +0200
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <94bdd2610904171038q2ef3c5b9vca08539c9c2489af@mail.gmail.com>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<23096059.post@talk.nabble.com>
	<94bdd2610904171038q2ef3c5b9vca08539c9c2489af@mail.gmail.com>
Message-ID: <94bdd2610904190446w1b9a9c23wac0e41e8d64dbe5a@mail.gmail.com>

Here's a draft for an extensible command based on plugin;

http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft

Let me know how it fits your use cases.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From marius at pov.lt  Sun Apr 19 14:59:10 2009
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 19 Apr 2009 15:59:10 +0300
Subject: [Distutils] PEP 376 - install/uninstall script in Distutils ?
In-Reply-To: <94bdd2610904161508y3df9526fib79330d9a9f28a89@mail.gmail.com>
References: <94bdd2610904131423v4d743413p94a8e4fa1828071@mail.gmail.com>
	<A056BBAC-EFC6-45A2-AC05-499A3AB075CE@bud.ca>
	<94bdd2610904161508y3df9526fib79330d9a9f28a89@mail.gmail.com>
Message-ID: <20090419125910.GA19409@fridge.pov.lt>

On Fri, Apr 17, 2009 at 12:08:22AM +0200, Tarek Ziad? wrote:
> On Wed, Apr 15, 2009 at 5:40 AM, Kevin Teague <kevin at bud.ca> wrote:
> > If there was a installer, I'm assuming it'd be quite a simple one - e.g.
> > installs single-version into site-packages. This caters to well to casual
> > user -- they can just run a "standard" command out-of-the-box and take-off
> > running with a distribution, but it also teaches them bad habits (e.g. that
> > you want to be commonly installing into site-packages or that you want to
> > develop your own code without properly expressing it's dependencies). When
> > they want to use better development practices, they'll have to switch to a
> > "non-standard" tools to do "non-standard" installations.
> 
> For the "site-package" part, this is true, but so wrong. Many people
> in this mailing list (and in real life) agrees that it's
> wrong to install a package in site-packages.

Half of those people say that it's wrong to install into
/usr/lib/pythonX.Y/site-packages (because /usr belongs to apt/rpm) but
fine to install into /usr/local/lib/pythonX.Y/site-packages.

The other half say that it's always wrong to install globally, since you
may want to use different sets of packages for different purposes.

Marius Gedminas
-- 
To err is human, but to really foul things up requires a computer.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090419/f5813fee/attachment.pgp>

From marius at pov.lt  Sun Apr 19 15:07:42 2009
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 19 Apr 2009 16:07:42 +0300
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <gscd0j$b8n$1@ger.gmane.org>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<gscd0j$b8n$1@ger.gmane.org>
Message-ID: <20090419130742.GB19409@fridge.pov.lt>

On Sat, Apr 18, 2009 at 07:19:31AM -0400, Tres Seaver wrote:
> > What are the other major reasons people use setuptools?
> 
> - setuptools.find_pacakges built-in SVN support makes a whole class of
>   packaging errors go away for me.
> 
> - virtualenv makes isolation between different applications sane;
>   it installs setuptools, and then makes 'easy_install' a pleasure
>   to use (I still can't believe people use easy_install in their
>   system Python!)

Why not?  They're told to.  Go to a random Python package's web page and
chances are you'll see "to install it, simply run easy_install
MyAwesomePackage".

> - entry points serve as crude equivalents of "named utilities"
>   in the Zope component architecture;  they allow an application to
>   define a class of plug points, which are then filled by other
>   libraries.  These plug points can then be configured together
>   declaratively (e.g. in an INI file) using their names.

Marius Gedminas
-- 
Microsoft -- because God hates us.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090419/e2b34ba3/attachment.pgp>

From marius at pov.lt  Sun Apr 19 15:10:32 2009
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 19 Apr 2009 16:10:32 +0300
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
Message-ID: <20090419131032.GC19409@fridge.pov.lt>

On Sat, Apr 18, 2009 at 11:09:11AM +0200, Lennart Regebro wrote:
> I don't actually know what features of setuptools people use.

1. 'python setup.py sdist register upload' (you could say this is a
   distutils feature, but my packages tend to unconditionally import
   setuptools in their setup.py)

2. virtualenv + easy_installing packages inside + getting all the
   dependencies specified with install_requires in step 1.

2b. same but with zc.buildout

3. entry points (usually for specifying console_scripts)

Marius Gedminas
-- 
If you sat a monkey down in front of a keyboard, the first thing typed would be
a unix command.
                -- Bill Lye
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090419/ed45c355/attachment.pgp>

From fdrake at gmail.com  Sun Apr 19 17:40:14 2009
From: fdrake at gmail.com (Fred Drake)
Date: Sun, 19 Apr 2009 11:40:14 -0400
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
In-Reply-To: <49EAF973.5050104@v.loewis.de>
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<49E79E30.4000904@v.loewis.de>
	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>
	<64ddb72c0904161915p5df5c7acwdb9c1f69c70bb46@mail.gmail.com>
	<gsa2fl$gs1$2@ger.gmane.org> <49E90E4D.4050109@v.loewis.de>
	<09700B613C4DD84FA9F2FEA52188281902BB1460@ayalga.fundacionctic.org>
	<9cee7ab80904181716t52949d77m46f67a9c747afd4e@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281902BB1461@ayalga.fundacionctic.org>
	<49EAF973.5050104@v.loewis.de>
Message-ID: <9cee7ab80904190840v25bb4bf1n6207286f386cbedf@mail.gmail.com>

On Sun, Apr 19, 2009 at 6:14 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> However, that wording is specific to the LGPL (and the GPL),
> and does not apply to any other license.

More importantly, it only applies if you specifically include it.

The problem I see is with non-specification; it should be more
difficult to specify imprecisely (by including text as described) than
to specify precisely.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From ianb at colorstudy.com  Mon Apr 20 09:17:01 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 20 Apr 2009 02:17:01 -0500
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
Message-ID: <b654cd2e0904200017uadbfda5xf975b9240ccaea4@mail.gmail.com>

On Sat, Apr 18, 2009 at 4:09 AM, Lennart Regebro <regebro at gmail.com> wrote:

> Setuptools non-support for Python 3 is currently a serious hindrance
> towards Python 3 aceptance. I'm trying to figure out what to do as a
> next step in the Python 3 support for setuptools. And I have
> encountered some obstacles. The first one is that setuptools requires
> itself for installing and running tests. That makes it hard to install
> it under Python 3. There are various solutions to this, but the next
> obstacle I encounter in choosing the right solution is that the code
> is hard to understand, and it makes me want to just rip it out and
> start over, or in even more frustrated moments, avoid the problems by
> not using setuptools at all. But the third obstacle for that is that I
> don't actually know what features of setuptools people use.
>
> I personally use setuptools for these reasons:
>
> 1. When I create projects with paster, it uses setuptools.


Of course if you create a project template that uses distutils, then that's
what you'll get... just happens no one does that with paster templates.


>
> 2. Setuptools makes it possible to specify requirements, which is then
> used by buildout.


In pip at least it does this with *.egg-info/install_requires.txt, but in
easy_install and I'm guessing buildout it's using the
pkg_resources.Distribution object.


>
> 3. Namespace packages require pkg_resources?


There's a way of doing it with pkgutils, but in some way that I don't
understand, pkg_resources does it better.


>
> 4. The test command.
>
> What are the other major reasons people use setuptools?
>

Entry points are the big one for me.  There is some other metadata that I
use from time to time, but I'm sure I could work around it.  Of course
there's the specific things pip uses as noted in a previous email.


-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090420/cad66844/attachment-0001.htm>

From regebro at gmail.com  Mon Apr 20 09:30:37 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 09:30:37 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <b654cd2e0904200017uadbfda5xf975b9240ccaea4@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<b654cd2e0904200017uadbfda5xf975b9240ccaea4@mail.gmail.com>
Message-ID: <319e029f0904200030u27f6b9adpb7c9d3c6d8c2350c@mail.gmail.com>

On Mon, Apr 20, 2009 at 09:17, Ian Bicking <ianb at colorstudy.com> wrote:
> Of course if you create a project template that uses distutils, then that's
> what you'll get... just happens no one does that with paster templates.

Oh, sure, it's just lazyness from my side, nothing else.

> Entry points are the big one for me.? There is some other metadata that I
> use from time to time, but I'm sure I could work around it.? Of course
> there's the specific things pip uses as noted in a previous email.

Yeah, it's clear to me that there is enough different features of
setuptools in such widespread usage that replacing it with something
else that actually is easily portable to Python 3 isn't feasible. To
get more modules running under Python 3 we really have to make
setuptools run and install smoothly under Python 3 too. I'll make a
post on this.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr 20 10:19:13 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 10:19:13 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
Message-ID: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>

Executive summary:
I have catch 22 problems in finishing the setuptools port to Python 3,
and unless somebody can come up with alternatives, I may rip out
setuptools dependency on itself completely, and only depend on
distutils.


I have as you know ported setuptools to Python 3. I'm in the process
of trying to fix and clean up that porting so that it can be merged
into trunk, but there is one obstacle left, and that's the question of
installing.

Currently there are several ways to install setuptools:

1) Run ez_install.py
2) Download / check-out source and run python setup.py install.
3) Download the correct egg and install that (which always confuses
me, I usually fail, and end up doing 2) instead).

Setuptools also helps with running tests by

4) python setup.py test

Currently for the Python 3 version, I've made a script that copies the
source tree and runs 2to3 on it. Then you can use that new tree to run
tests, install, make eggs and releases etc. The problems with this is:

a) The script is made for my computer only. It wouldn't work on
Windows, and it requires you to have lib2to3 checked out in a
particular place (although I think that's only if you use 3.0.0, and
not 3.0.1, but I haven't tested).

b) It's slow, as it copies all files, even those who hasn't changed
when you fixed a bug. Making a script that only copies the changed
files and runs 2to3 on them is possible, but I'd like not too, it
takes time and is likely to break. It seems like a waste.

Obviously, this is exactly the things distutils are meant to solve.
And indeed, distutils in Python 3 has a command called build_2to3
which solves exactly these problems. And indeed, this is the technique
I've used for the Python 3 branch of zope.interfaces. There I just run
setup.py install, and if I'm doing that under Python 3, it will first
run 2to3 on the code before installing. Same thing with running
setup.py test. It will sync changes to build/, run 2to3 on changed
files and then run the tests in build. It makes porting to Python 3
much easier, and it makes installing from source painless.

So why don't I use that for setuptools? Well, because:

c) The setup of setuptools requires setuptools. So to be able to do
the 2to3 conversion in the setup, I need to first convert the source
with 2to3. Yes, catch 22.

I've tried to get around this problem, but failed. Solutions I tried was:

I) Fall back to distutils if setuptools can't be imported. This means
that I can with some effort make a setup that will work under Python 3
and install setuptools under Python 3. Problem solved? No. Because
running setup.py again will just mean that it tries to import
setuptools from the local Python2 location, and it will fail. So in
this case I can't run tests of build eggs for setuptool.

II) Moving the source into a src directory. This means that when
setup.py runs, it will use the version of setuptools installed under
I), and building eggs etc is possible (I think, I haven't tested that
the eggs are correct). Problem solved? No, because you still can't
test it. Which leads us to problem d:

d) When running the tests, the setuptools module will already be
imported. But it will have imported the one in site-packages, *not*
the one in src, and it will therefore test the one in site-packages.
And that one doesn't have the api_tests.txt copied in, so that will
fail, and

e) even if api_tests.txt was copied, this would mean you had to
install setuptools before you test it, which is daft.


I don't have a good solution to this, unless we can drop setuptools
dependency on setuptools completely, and just use plain distutils for
installing and testing setuptools. This will mean no more setuptools
source eggs, but then I don't see the purpose of those, really.
Arguments for not doing this and alternative solutions are highly
welcome.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Mon Apr 20 10:26:29 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 20 Apr 2009 09:26:29 +0100
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
Message-ID: <79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>

2009/4/20 Lennart Regebro <regebro at gmail.com>:
> I don't have a good solution to this, unless we can drop setuptools
> dependency on setuptools completely, and just use plain distutils for
> installing and testing setuptools. This will mean no more setuptools
> source eggs, but then I don't see the purpose of those, really.
> Arguments for not doing this and alternative solutions are highly
> welcome.

Personally, this type of circular dependency seems totally wrong. I'd
argue that making setuptools not depend on itself is the right thing
to do, unless someone can provide a compelling reason why it is
necessary (and such a reason would hopefully then provide more
concrete avenues for looking for other possible alternative
approaches).

Paul.

From ben+python at benfinney.id.au  Mon Apr 20 11:44:21 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 20 Apr 2009 19:44:21 +1000
Subject: [Distutils] The problem with Setuptools on Python 3.
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
Message-ID: <87myabmzgq.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> 2009/4/20 Lennart Regebro <regebro at gmail.com>:
> > I don't have a good solution to this, unless we can drop setuptools
> > dependency on setuptools completely, and just use plain distutils
> > for installing and testing setuptools.
> 
> Personally, this type of circular dependency seems totally wrong. I'd
> argue that making setuptools not depend on itself is the right thing
> to do, unless someone can provide a compelling reason why it is
> necessary (and such a reason would hopefully then provide more
> concrete avenues for looking for other possible alternative
> approaches).

Yes, that seems crazy-making to me and I wasn't aware it was the current
situation.

+1 for building setuptools on a base of distutils only, especially if
it's already been achieved.

-- 
 \          ?A man's only as old as the woman he feels.? ?Groucho Marx |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From regebro at gmail.com  Mon Apr 20 12:35:14 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 12:35:14 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <87myabmzgq.fsf@benfinney.id.au>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
Message-ID: <319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>

On Mon, Apr 20, 2009 at 11:44, Ben Finney <ben+python at benfinney.id.au> wrote:
> +1 for building setuptools on a base of distutils only, especially if
> it's already been achieved.

No, we are going to have to make special custom extensions, at least
for running tests. It's not that much work, but it is code
duplication. I don't have the full overview of what features we lose
either. But it does seem to me that most of these features, like the
source eggs, cause more trouble than they solve.

We'll probably need a separate Python 3 version of easy_install and
the virtual-python scripts as well, but I already did those. But I
think getting easy_install to download the source tgz and run setup.py
install on it are going to simplify the code and remove the need for
the eggs.

Setuptools seems to have a lot of clever code to fix non-problems.
This may very well be because I don't know what the real problems are,
and these problems are not documented. I guess we need Phillip's input
on that.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Mon Apr 20 14:21:34 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 20 Apr 2009 13:21:34 +0100
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
Message-ID: <79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>

2009/4/20 Lennart Regebro <regebro at gmail.com>:
> On Mon, Apr 20, 2009 at 11:44, Ben Finney <ben+python at benfinney.id.au> wrote:
>> +1 for building setuptools on a base of distutils only, especially if
>> it's already been achieved.
>
> No, we are going to have to make special custom extensions, at least
> for running tests. It's not that much work, but it is code
> duplication. I don't have the full overview of what features we lose
> either. But it does seem to me that most of these features, like the
> source eggs, cause more trouble than they solve.

Are you saying that you need to use setuptools (or at least the
feaures of setuptools) to develop setuptools? That's crazy. To run the
setuptools tests, just run the test.py (or whatever) script. The
setuptools ability to type python setup.py test, while convenient,
simply isn't available while you're developing setuptools. The same
logic applies to *any* setuptools feature that is used in the
development of setuptools itself. Trying to make it available adds
lots of complexity for the benefit of very few people (ie, people
writing the setuptools code).

Bootstrapping like this should be reserved for people writing C
compilers in C, and other equally major-league projects.

Just my view, and I'm not a setuptools fan, so you can probably
disregard it. (But this type of complexity for its own sake is the
reason I dislike the whole setuptools philosophy, so maybe it's worth
making explicit...)

Paul.

From regebro at gmail.com  Mon Apr 20 14:37:36 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 14:37:36 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
Message-ID: <319e029f0904200537i5ba160cascfa026535a23b4b0@mail.gmail.com>

On Mon, Apr 20, 2009 at 14:21, Paul Moore <p.f.moore at gmail.com> wrote:
> Are you saying that you need to use setuptools (or at least the
> feaures of setuptools) to develop setuptools?

Currently, yes. The testsuite is run with the testrunner of
setuptools. You can run them separately too, I'm sure, but that's a
pain. There is no test.py to run. Maybe you can use nose or something
to run the tests, but then again you would need to port *nose* to
Python 3... :-/

> That's crazy. To run the
> setuptools tests, just run the test.py (or whatever) script.

There isn't any. And to do that you need to convert it to Python 3
first, and we are back to the "run a porting script, then run a test
command", which is what we want to avoid in the first place. distutils
have code exactly to help this kind of situation, so I think we should
depend on that.

> setuptools ability to type python setup.py test, while convenient,
> simply isn't available while you're developing setuptools.

Sure it is. Maybe it *shouldn't*. But it *is*. Except under Python 3,
of course, which is the problem.

> logic applies to *any* setuptools feature that is used in the
> development of setuptools itself. Trying to make it available adds
> lots of complexity for the benefit of very few people (ie, people
> writing the setuptools code).

It's not so much "trying to". Since the structure of the package is
such that setuptools end up directly on the path, it *is* available.
So the question is if we should make sure it *isn't* available, by
moving it all down into a src directory, and then modify the
setup-script so that it no longer depends on setuptools.

> Bootstrapping like this should be reserved for people writing C
> compilers in C, and other equally major-league projects.

Are you saying setuptools isn't major-league? ;-)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From david at ar.media.kyoto-u.ac.jp  Mon Apr 20 15:23:25 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 20 Apr 2009 22:23:25 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>	<87myabmzgq.fsf@benfinney.id.au>	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
Message-ID: <49EC774D.9030808@ar.media.kyoto-u.ac.jp>

Paul Moore wrote:
>
> Are you saying that you need to use setuptools (or at least the
> feaures of setuptools) to develop setuptools? That's crazy.To run the
> setuptools tests, just run the test.py (or whatever) script. The
> setuptools ability to type python setup.py test, while convenient,
> simply isn't available while you're developing setuptools. The same
> logic applies to *any* setuptools feature that is used in the
> development of setuptools itself. Trying to make it available adds
> lots of complexity for the benefit of very few people (ie, people
> writing the setuptools code).
>
> Bootstrapping like this should be reserved for people writing C
> compilers in C, and other equally major-league projects.
>   

I don't know the details of setuptools, but it is generally quite
tempting to develop a new build/distribution tool using the new build
tool, with some bootstrap process. That's how most build tools I know
work, actually. FWIW, we use this as well in numpy for numpy.distutils
extensions.

And as much as you can count me in the "not a setuptools fan camp", I
think it is easy to say  setuptools code is bad - that's the natural
reaction, really, and I would be surprised if P.J.E would not agree. But
I also think anyone who had to deal with distutils extensions will tell
you the same story - that's inherent to how distutils was conceived (10
years ago) and implemented. The distutils codebase is pretty horrible -
I find m4 macro and 100000 lines of shell code in autoconf easier to
deal with, really. You can deal with it, but it will certainly never be
pretty.

David

From ziade.tarek at gmail.com  Mon Apr 20 15:50:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 20 Apr 2009 15:50:12 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EC774D.9030808@ar.media.kyoto-u.ac.jp>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>

On Mon, Apr 20, 2009 at 3:23 PM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Paul Moore wrote:
>>
>> Are you saying that you need to use setuptools (or at least the
>> feaures of setuptools) to develop setuptools? That's crazy.To run the
>> setuptools tests, just run the test.py (or whatever) script. The
>> setuptools ability to type python setup.py test, while convenient,
>> simply isn't available while you're developing setuptools. The same
>> logic applies to *any* setuptools feature that is used in the
>> development of setuptools itself. Trying to make it available adds
>> lots of complexity for the benefit of very few people (ie, people
>> writing the setuptools code).
>>
>> Bootstrapping like this should be reserved for people writing C
>> compilers in C, and other equally major-league projects.
>>
>
> I don't know the details of setuptools, but it is generally quite
> tempting to develop a new build/distribution tool using the new build
> tool, with some bootstrap process. That's how most build tools I know
> work, actually. FWIW, we use this as well in numpy for numpy.distutils
> extensions.
>
> And as much as you can count me in the "not a setuptools fan camp", I
> think it is easy to say ?setuptools code is bad - that's the natural
> reaction, really, and I would be surprised if P.J.E would not agree. But
> I also think anyone who had to deal with distutils extensions will tell
> you the same story - that's inherent to how distutils was conceived (10
> years ago) and implemented. The distutils codebase is pretty horrible -
> I find m4 macro and 100000 lines of shell code in autoconf easier to
> deal with, really. You can deal with it, but it will certainly never be
> pretty.
>

this can change. I am working on it. I need feedback though.

let me know how my extension code proposal fits your needs.

http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft


> David
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From david at ar.media.kyoto-u.ac.jp  Mon Apr 20 16:00:39 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 20 Apr 2009 23:00:39 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>	
	<87myabmzgq.fsf@benfinney.id.au>	
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>	
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
Message-ID: <49EC8007.6090103@ar.media.kyoto-u.ac.jp>

Hi Tarek,

Tarek Ziad? wrote:
>
> this can change. I am working on it. I need feedback though.
>
> let me know how my extension code proposal fits your needs.
>
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft
>   

I will fill in the examples (this pages does not concern manifest only,
then ?) from my own experience.

IMHO, adding plugins systems will not change the fundamental
deficiencies of distutils, though. You may be able to mitigate some
problems, but I don't think you will be able to solve the fundamental
issues, because they are fundamental design issues. The only way to fix
them would induce serious breakage, and in that case, given the quality
of distutils code, starting from scratch would be easier - I have yet
encountered a single major distutils abstraction which was not poorly
conceived and implemented (my own experience is related to build issues,
for which numpy has requirements which go much beyond the usual python
package, though). Commands classes, compiler classes, distributions, all
this is very wrong for someone who wants more than compiling a couple of
C files and tight control.

David


From ziade.tarek at gmail.com  Mon Apr 20 16:26:53 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 20 Apr 2009 16:26:53 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EC8007.6090103@ar.media.kyoto-u.ac.jp>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>

On Mon, Apr 20, 2009 at 4:00 PM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Hi Tarek,
>
> Tarek Ziad? wrote:
>>
>> this can change. I am working on it. I need feedback though.
>>
>> let me know how my extension code proposal fits your needs.
>>
>> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft
>>
>
> I will fill in the examples (this pages does not concern manifest only,
> then ?) from my own experience.

no it doesn't.  it's a generic tool to extend a command using plugins .

>
> IMHO, adding plugins systems will not change the fundamental
> deficiencies of distutils, though. You may be able to mitigate some
> problems, but I don't think you will be able to solve the fundamental
> issues, because they are fundamental design issues. The only way to fix
> them would induce serious breakage, and in that case, given the quality
> of distutils code, starting from scratch would be easier -

Sorry, it's too easy to say "it sucks, scratch it and re-do it" ;)

1/ I am on Guido side here. I'd rather refactor the existing code.
2/  It's there, it works for a lot of people. let's improve it slowly
step by step
3/ the quality is increasing, the test coverage too. compare the trunk
with Python 2.5 one..

my proposal:

Step 1 - check the extension mechanism and see if it fits your brain
Step 2 - see where you would like to refactor distutils in order to
make it configurable/extensible

--

Tarek

From david at ar.media.kyoto-u.ac.jp  Mon Apr 20 16:36:51 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 20 Apr 2009 23:36:51 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>	
	<87myabmzgq.fsf@benfinney.id.au>	
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>	
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
Message-ID: <49EC8883.10106@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
>
> no it doesn't.  it's a generic tool to extend a command using plugins.
>   

OK.

>
> Sorry, it's too easy to say "it sucks, scratch it and re-do it" ;)
>   

Yes, it is easy to say it sucks, but I am not saying that out of thin
air. Refactoring works if the general API is ok - but that's not the
case with distutils. As a data point, several people have tried to add
features to distutils in numpy extensions, and I went much further than
anyone else by bypassing distutils entirely, replacing the build_*
commands by scons. The core codebase is ten times smaller than the
original code it replaces (numpy.distutils is around 10 000 LOC, like
distutils itself).

I agree that in the short term, distutils should be improved, but in the
long term, I don't see anything which would be both a significant
improvement while staying backward compatible with distutils, if only
because so many setup.py files depend on implementation details of
distutils.

cheers,

Davud

From pje at telecommunity.com  Mon Apr 20 17:03:36 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 11:03:36 -0400
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <b654cd2e0904200017uadbfda5xf975b9240ccaea4@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<b654cd2e0904200017uadbfda5xf975b9240ccaea4@mail.gmail.com>
Message-ID: <20090420150108.778063A4070@sparrow.telecommunity.com>

At 02:17 AM 4/20/2009 -0500, Ian Bicking wrote:
>3. Namespace packages require pkg_resources?
>
>
>There's a way of doing it with pkgutils, but in some way that I 
>don't understand, pkg_resources does it better.

pkgutil doesn't support zip files (or any other non-filename 
importers/path strings), later additions to sys.path, nested 
namespaces...  and that may not be a complete list. 


From ziade.tarek at gmail.com  Mon Apr 20 17:02:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 20 Apr 2009 17:02:26 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EC8883.10106@ar.media.kyoto-u.ac.jp>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610904200802q53e3c30ds7ebe9d3d5af940ef@mail.gmail.com>

On Mon, Apr 20, 2009 at 4:36 PM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Tarek Ziad? wrote:
>>
>> no it doesn't. ?it's a generic tool to extend a command using plugins.
>>
>
> OK.
>
>>
>> Sorry, it's too easy to say "it sucks, scratch it and re-do it" ;)
>>
>
> Yes, it is easy to say it sucks, but I am not saying that out of thin
> air. Refactoring works if the general API is ok - but that's not the
> case with distutils. As a data point, several people have tried to add
> features to distutils in numpy extensions, and I went much further than
> anyone else by bypassing distutils entirely, replacing the build_*
> commands by scons. The core codebase is ten times smaller than the
> original code it replaces (numpy.distutils is around 10 000 LOC, like
> distutils itself).
>
> I agree that in the short term, distutils should be improved, but in the
> long term, I don't see anything which would be both a significant
> improvement while staying backward compatible with distutils, if only
> because so many setup.py files depend on implementation details of
> distutils.

In the short term, Distutils needs to be a better citizen when people
wants to extend it, so they can do it without re-writing everything.

You have this experience of "pain", share it through real use cases.

In the long term Distutils needs to be *smaller* and to provide
a playground for other tools. in particular for all the tools that need to
build a binary package on a given platform.

(We can't keep those platform specific tools in the scope of distutils)

And the work done on various PEPs (all links at the top of
http://wiki.python.org/moin/Distutils)
are aiming to this goal.

What about organizing a sprint soon btw, so we don't loose Pycon efforts ?

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Mon Apr 20 17:07:39 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 11:07:39 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
Message-ID: <20090420150508.5A56D3A4070@sparrow.telecommunity.com>

At 10:19 AM 4/20/2009 +0200, Lennart Regebro wrote:
>I don't have a good solution to this, unless we can drop setuptools
>dependency on setuptools completely, and just use plain distutils for
>installing and testing setuptools.

I still don't understand why you can't have a setup3.py that uses 
only distutils, in order to run the build2to3 on.  Am I missing something?

i.e., ISTM that you should be able to do something like:

   python3 setup3.py build_2to3
   cd wherevertheoutputgoes
   python3 setup.py test  (or whatever)


From regebro at gmail.com  Mon Apr 20 18:02:06 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 18:02:06 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420150508.5A56D3A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>

On Mon, Apr 20, 2009 at 17:07, P.J. Eby <pje at telecommunity.com> wrote:
> I still don't understand why you can't have a setup3.py that uses only
> distutils, in order to run the build2to3 on. ?Am I missing something?

Well, the question then becomes why not just depends on distutils all
the way? With a setup3.py that does this, most of the work for
removing this self-dependency is done.

> ?python3 setup3.py build_2to3
> ?cd wherevertheoutputgoes
> ?python3 setup.py test ?(or whatever)

Well, first of all the distutils build command doesn't include the
files needed, such as setup.py, and also it doesn't copy the txt files
that is the doctests. So it would need an extended version of
build_py_2to3 that does copy these files. And instead of doing "cd
wherever, python2 setup.py test", we can just as well just do a custom
command that runs the custom build_py_2to3 and runs the tests from
there too.

So in fact, what you are suggesting here is almost exactly what I'm
suggesting, except that I don't see the point in confusing and
complicating things by keeping the old setup.py.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Mon Apr 20 18:17:08 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 12:17:08 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
Message-ID: <20090420161437.9FD583A4070@sparrow.telecommunity.com>

At 06:02 PM 4/20/2009 +0200, Lennart Regebro wrote:
>On Mon, Apr 20, 2009 at 17:07, P.J. Eby <pje at telecommunity.com> wrote:
> > I still don't understand why you can't have a setup3.py that uses only
> > distutils, in order to run the build2to3 on.  Am I missing something?
>
>Well, the question then becomes why not just depends on distutils all
>the way? With a setup3.py that does this, most of the work for
>removing this self-dependency is done.
>
> >  python3 setup3.py build_2to3
> >  cd wherevertheoutputgoes
> >  python3 setup.py test  (or whatever)
>
>Well, first of all the distutils build command doesn't include the
>files needed, such as setup.py, and also it doesn't copy the txt files
>that is the doctests. So it would need an extended version of
>build_py_2to3 that does copy these files. And instead of doing "cd
>wherever, python2 setup.py test", we can just as well just do a custom
>command that runs the custom build_py_2to3 and runs the tests from
>there too.
>
>So in fact, what you are suggesting here is almost exactly what I'm
>suggesting, except that I don't see the point in confusing and
>complicating things by keeping the old setup.py.

Because that's the one that generates the metadata setuptools needs 
to run, test itself, etc.


From a.badger at gmail.com  Mon Apr 20 18:10:57 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Mon, 20 Apr 2009 09:10:57 -0700
Subject: [Distutils] Making commands extensible by default
In-Reply-To: <49E84A16.7010902@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904091330h443914cct96be4f55b8861b23@mail.gmail.com>
	<49E7ACCD.8000608@gmail.com>
	<49E7EC14.90208@ar.media.kyoto-u.ac.jp>
	<49E84986.7090207@gmail.com>
	<49E84A16.7010902@ar.media.kyoto-u.ac.jp>
Message-ID: <49EC9E91.4050703@gmail.com>

David Cournapeau wrote:
> Toshio Kuratomi wrote:
>> They don't have bearing on talking about
>> redesigning how to design a new architecture that's easy to extend.
>>   
> 
> Those examples show why extending distutils commands with subclassing +
> post processing is not always enough. I don't understand why they would
> not be relevant for the design to improve distutils extensibility. They
> are quite typical of the usual problems I have when I need to extend
> distutils myself.
> 
+1 to this argument :-)

subclassing is a bad way to implement extensibility for essentially
imperative tasks.

I was just saying that the fact that distutils commands being used from
paver having bugs does not invalidate paver's design of having functions
 be the task unit to build upon.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090420/29da3aa3/attachment.pgp>

From regebro at gmail.com  Mon Apr 20 18:30:13 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 18:30:13 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420161437.9FD583A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>

On Mon, Apr 20, 2009 at 18:17, P.J. Eby <pje at telecommunity.com> wrote:
> Because that's the one that generates the metadata setuptools needs to run,
> test itself, etc.

You really need to stop assuming everybody else here understands the
issues as well as you do, because we don't. Or well, I don't, anyway.
I don't understand a word in your answer, it makes no sense to me
whatsoever, Try answering like I'm a moron, because I probably am.

> Because that's the one that generates the metadata setuptools needs to run,
> test itself, etc.

No, setup3.py does that. Why do I need setup.py?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr 20 18:31:35 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 18:31:35 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
Message-ID: <319e029f0904200931k2749a9aj6f35c1d313dd8f89@mail.gmail.com>

Let me reformulate that:

> Because that's the one that generates the metadata setuptools needs to run,
> test itself, etc.

Why do I need setuptools to do that? Why is not distutils enough?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From jpellerin at gmail.com  Mon Apr 20 18:39:37 2009
From: jpellerin at gmail.com (jason pellerin)
Date: Mon, 20 Apr 2009 12:39:37 -0400
Subject: [Distutils] nose on python 3 [was setuptools on python 3]
Message-ID: <3bb02d620904200939u72a0777bj5f272b9a425cbb89@mail.gmail.com>

Lennart Regebro wrote:

> Currently, yes. The testsuite is run with the testrunner of
> setuptools. You can run them separately too, I'm sure, but that's a
> pain. There is no test.py to run. Maybe you can use nose or something
> to run the tests, but then again you would need to port *nose* to
> Python 3... :-/

There is a branch of nose for python 3:

svn checkout http://python-nose.googlecode.com/svn/branches/py3k

No stable release yet, so user beware. But I've gotten good reports
from the few brave first adopters.

JP

From pje at telecommunity.com  Mon Apr 20 19:02:36 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 13:02:36 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
Message-ID: <20090420170005.C3CB63A4070@sparrow.telecommunity.com>

At 06:30 PM 4/20/2009 +0200, Lennart Regebro wrote:
> > Because that's the one that generates the metadata setuptools needs to run,
> > test itself, etc.
>
>No, setup3.py does that.

I thought you couldn't import setuptools in setup3.py, for all the 
reasons you just described?

>Why do I need setup.py?

To define the core entry points, basically.  These are really 
bootstrapped from the existing entry_points.txt (which is why it's in 
SVN), but they need to be regenerated at egg_info/bdist_egg/install 
time to match the current Python version.

Apart from that, I believe setuptools only uses itself for finding 
its own packages (including data) and manifest building (i.e. finding 
files that are under SVN control).

So I suppose that if you dropped the use of find_packages(), you 
could make a setup.py that would let you run distutils commands on 
the code base, I just don't see how you'd get any setuptools-level 
commands to work properly without setuptools being imported first.

i.e., you could probably manage having one setup.py, or two 
setup.py's whose only difference is whether they import setuptools 
and use find_packages; like a setup3.py that does distutils, and a 
regular setup.py that imports setuptools and then execfiles setup3.py.

None of this makes setuptools any less reliant on itself for testing 
and building, though, since the core entry points have to exist in 
order for commands to be found, options to be verified, etc.


From pje at telecommunity.com  Mon Apr 20 19:04:28 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 13:04:28 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200931k2749a9aj6f35c1d313dd8f89@mail.gmail.com
 >
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<319e029f0904200931k2749a9aj6f35c1d313dd8f89@mail.gmail.com>
Message-ID: <20090420170156.695633A4070@sparrow.telecommunity.com>

At 06:31 PM 4/20/2009 +0200, Lennart Regebro wrote:
>Let me reformulate that:
>
> > Because that's the one that generates the metadata setuptools needs to run,
> > test itself, etc.
>
>Why do I need setuptools to do that? Why is not distutils enough?

Because distutils doesn't have an egg_info command, or SVN-based 
manifest building (for making sdists).


From regebro at gmail.com  Mon Apr 20 19:13:41 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 19:13:41 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420170005.C3CB63A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>

On Mon, Apr 20, 2009 at 19:02, P.J. Eby <pje at telecommunity.com> wrote:
> I thought you couldn't import setuptools in setup3.py, for all the reasons
> you just described?

Yes? And then you draw some sort of conclusion of this that totally escapes me.

>> Why do I need setup.py?
>
> To define the core entry points, basically. ?These are really bootstrapped
> from the existing entry_points.txt (which is why it's in SVN), but they need
> to be regenerated at egg_info/bdist_egg/install time to match the current
> Python version.

OK. And why are they needed to install and test setuptools?

> Apart from that, I believe setuptools only uses itself for finding its own
> packages (including data) and manifest building (i.e. finding files that are
> under SVN control).

Sure, but that's easily avoided.

> So I suppose that if you dropped the use of find_packages(), you could make
> a setup.py that would let you run distutils commands on the code base, I
> just don't see how you'd get any setuptools-level commands to work properly
> without setuptools being imported first.

They won't work. The question is why they need to work.

> None of this makes setuptools any less reliant on itself for testing and
> building, though, since the core entry points have to exist in order for
> commands to be found, options to be verified, etc.

Which still doesn't really answer the question: Why setuptools need to
rely on setuptools.

> Because distutils doesn't have an egg_info command

I've already mentioned that getting rid of the setuptools dependency
on itself means you can't build setuptools eggs. But I've also pointed
out that I don't understand why they would be needed. For all other
packages it's recommended that only source distributions are done
unless you have c-extensions. Why is setuptools an exception?

> or SVN-based manifest building (for making sdists).

Well, we can survive without that, as far as I can see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From chrish at apstat.com  Mon Apr 20 19:28:51 2009
From: chrish at apstat.com (Christian Hudon)
Date: Mon, 20 Apr 2009 13:28:51 -0400
Subject: [Distutils] Help making setuptools install more like plain
	distutils one
Message-ID: <49ECB0D3.6050905@apstat.com>

Hello,

We have a setup where a central /usr/local is copied to all our 
machines. The packages installed in said central /usr/local are managed 
via stow. (Basically, each package is installed in a separate directory 
under /usr/local/stow, and invoking the stow command creates symlinks to 
make the package appear under /usr/local.)

This works very well for a wide range of packages: autoconf packages, 
CRAN packages for R, etc. This includes plain distutils python packages 
(install via "python setup.py install --prefix 
/usr/local/stow/some-package-name", then run stow). The only exception 
is setuptools-based python packages.

Is there a way to ask setuptools to do an install that looks more like a 
standard distutils install? I don't care if I lose some the advanced 
setuptools features. Basically, I need an install that's done via just 
copying new files. Problems like setuptoosl checking that the 
destination directory is in the PYTHONPATH I can work around (although 
if there's a switch to disable that check, I'd be happy to learn about 
it). The main problem is the file that's edited on each install to add a 
new line for each package install via setuptools. Is there a way with 
setuptools of getting just a directory tree (or a tarball, etc.) that 
either setuptools or myself can just copy somewhere to have an installed 
python module?

Thanks in advance for any help,

  Christian


From regebro at gmail.com  Mon Apr 20 19:47:41 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 19:47:41 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EC8883.10106@ar.media.kyoto-u.ac.jp>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>

On Mon, Apr 20, 2009 at 16:36, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> I agree that in the short term, distutils should be improved, but in the
> long term, I don't see anything which would be both a significant
> improvement while staying backward compatible with distutils, if only
> because so many setup.py files depend on implementation details of
> distutils.

I think the answer to this can be only one: Since this is open source,
you can't expect somebody to do it. It will only be done if you do it
yourself, "you" being the people who experience the pain, in this
case, well, you. :-D

I sure as heck do not have the knowledge to do it. So the answer I
think, is the cold and brutal "Yes! Great Idea! When are you done?"
:-)


What I'm worried about with distutils is both what you mention:
Missing flexibility internally, and also that the plans are to make it
*smaller*, which inevitably will break backwards compatibility, which
will cause pains. And I still haven't gotten any answer on how to make
a backport of a new distutils and have that automatically installed
when you install a package that uses the new features. (Although maybe
just failing with an error asking you to install distutils vX.Y might
be acceptable).

It's quite possible that it's better to make something that at least
has another name, which totally avoids all the above problems. That
doesn't mean the code has to be all new and shiny. So I sure wouldn't
mind something new, that avoids the distutil problems. But wanting it
is something else than doing it. I can't. Can you? And it would be
needed rather quickly too. :-D

At least somebody could maybe make some sort of rough plan of what
this golden distutils should contain and how it should work, and we
can get a feeling if it's even remotely feasible.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ianb at colorstudy.com  Mon Apr 20 19:57:13 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 20 Apr 2009 12:57:13 -0500
Subject: [Distutils] Help making setuptools install more like plain
	distutils one
In-Reply-To: <49ECB0D3.6050905@apstat.com>
References: <49ECB0D3.6050905@apstat.com>
Message-ID: <b654cd2e0904201057s2a32f923oc84e9a03cbfbd13a@mail.gmail.com>

Use --single-version-externally-managed (also pip using this flag by
default, so if you install via pip you'll get this behavior).

On Mon, Apr 20, 2009 at 12:28 PM, Christian Hudon <chrish at apstat.com> wrote:
> Hello,
>
> We have a setup where a central /usr/local is copied to all our machines.
> The packages installed in said central /usr/local are managed via stow.
> (Basically, each package is installed in a separate directory under
> /usr/local/stow, and invoking the stow command creates symlinks to make the
> package appear under /usr/local.)
>
> This works very well for a wide range of packages: autoconf packages, CRAN
> packages for R, etc. This includes plain distutils python packages (install
> via "python setup.py install --prefix /usr/local/stow/some-package-name",
> then run stow). The only exception is setuptools-based python packages.
>
> Is there a way to ask setuptools to do an install that looks more like a
> standard distutils install? I don't care if I lose some the advanced
> setuptools features. Basically, I need an install that's done via just
> copying new files. Problems like setuptoosl checking that the destination
> directory is in the PYTHONPATH I can work around (although if there's a
> switch to disable that check, I'd be happy to learn about it). The main
> problem is the file that's edited on each install to add a new line for each
> package install via setuptools. Is there a way with setuptools of getting
> just a directory tree (or a tarball, etc.) that either setuptools or myself
> can just copy somewhere to have an installed python module?
>
> Thanks in advance for any help,
>
> ?Christian
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Ian Bicking  |  http://blog.ianbicking.org

From pje at telecommunity.com  Mon Apr 20 21:01:09 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 15:01:09 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
Message-ID: <20090420185839.67ACB3A4070@sparrow.telecommunity.com>

At 07:13 PM 4/20/2009 +0200, Lennart Regebro wrote:
>Which still doesn't really answer the question: Why setuptools need to
>rely on setuptools.

Because there's less duplication and chances of error that 
way.  (Earlier versions of setuptools relied on manually-created text 
files, instead of using egg_info to generate them, and 
creating/maintaining the files was a pain.)


> > Because distutils doesn't have an egg_info command
>
>I've already mentioned that getting rid of the setuptools dependency
>on itself means you can't build setuptools eggs. But I've also pointed
>out that I don't understand why they would be needed.

Because the egg-info files (irrespective of whether you build an 
sdist, an .egg, an .exe, or even install a "flat" package) contain 
information like entry points and requirements.

In the case of setuptools itself, the entry points are critical 
because setuptools finds its own commands, options, etc., almost 
exclusively by using entry points.  This is so that it's on a level 
playing field with other packages wanting to extend those features, 
and serves as an example of how to extend itself.

Look, if you want to fork setuptools to not depend on itself, knock 
yourself out.  I sure can't stop you.

It's simply that, from my POV, I'd rather not complicate the main 
branches with duplication to support Python 3, when ISTM that the 
Python-3-support bits could be isolated instead.  Also, I think that 
setuptools self-bootstrapping is both a non-trivial integration test 
that shouldn't be dropped, AND a non-trivial example of how to extend 
setuptools (since so much of it is implemented using independent 
entry points rather than hardwired behavior).

All that having been said, I'm not the one doing the port, so do what 
you think best.


>  For all other
>packages it's recommended that only source distributions are done
>unless you have c-extensions.

Recommended by whom, for what purpose?


>Why is setuptools an exception?

For end-user convenience.  A large number of people installing 
setuptools are not installing it because they are personally 
interested in it, but because something else they want uses it.


From pje at telecommunity.com  Mon Apr 20 21:03:57 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 15:03:57 -0400
Subject: [Distutils] Help making setuptools install more like plain
 distutils one
In-Reply-To: <49ECB0D3.6050905@apstat.com>
References: <49ECB0D3.6050905@apstat.com>
Message-ID: <20090420190124.0638B3A4070@sparrow.telecommunity.com>

At 01:28 PM 4/20/2009 -0400, Christian Hudon wrote:
>Is there a way to ask setuptools to do an install that looks more 
>like a standard distutils install?

Yes, use "setup.py install --single-version-externally-managed 
--record=/some/file".  This will install the distutils way, and 
record all the installed files in /some/file (which you can then 
discard if you like).


>I don't care if I lose some the advanced setuptools features. 
>Basically, I need an install that's done via just copying new files. 
>Problems like setuptoosl checking that the destination directory is 
>in the PYTHONPATH I can work around (although if there's a switch to 
>disable that check, I'd be happy to learn about it). The main 
>problem is the file that's edited on each install to add a new line 
>for each package install via setuptools. Is there a way with 
>setuptools of getting just a directory tree (or a tarball, etc.) 
>that either setuptools or myself can just copy somewhere to have an 
>installed python module?

"setup.py bdist_dumb" is probably what you want for that.  It will 
work pretty much identically with both distutils and setuptools.


From regebro at gmail.com  Mon Apr 20 21:11:30 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 21:11:30 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420185839.67ACB3A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>

On Mon, Apr 20, 2009 at 21:01, P.J. Eby <pje at telecommunity.com> wrote:
> Look, if you want to fork setuptools to not depend on itself, knock yourself
> out. ?I sure can't stop you.

Well, unless somebody can come up with an alternative... But I have to
say I'm extremely reluctant to do so. And I think that my action
instead will be to abandon all further efforts of porting things to
Python 3. This is too much pain, takes to much time and doesn't lead
anywhere. I have since July wanted to port the zope.component
architecture to Python 3. I've put down weeks of work on it. Almost
*all* that work has been related to setuptools.

> It's simply that, from my POV, I'd rather not complicate the main branches
> with duplication to support Python 3, when ISTM that the Python-3-support
> bits could be isolated instead.

"Isolated"? What do you mean? If you have suggestions on how to solve
the problems I'm describing, can you please come forward with them,
I've asked several times already. All you do is say "no, no, no, no,
no", and "becasue". It isn't helpful.

> All that having been said, I'm not the one doing the port, so do what you
> think best.

No, because although you aren't the one doing the port, you still are
the main setuptools maintainer, and this will need to merged. Soon. As
I mentioned above, the fact that setuptools doesn't have good Python 3
support is a major hindrance for Python 3 acceptance.

>> ?For all other
>> packages it's recommended that only source distributions are done
>> unless you have c-extensions.
>
> Recommended by whom, for what purpose?

Everybody, for all purposes.

> For end-user convenience. ?A large number of people installing setuptools
> are not installing it because they are personally interested in it, but
> because something else they want uses it.

Yes? And again you don't explain how this leads to you conclusion.
Eggs are not easier to install, on the contrary, I have tried and
failed a couple of times, and ended up using the source install
instead. It's easy to install. You download, unpack, run python
setup.py install, like any other package.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Mon Apr 20 21:52:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 20 Apr 2009 15:52:18 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
Message-ID: <20090420194947.377203A4070@sparrow.telecommunity.com>

At 09:11 PM 4/20/2009 +0200, Lennart Regebro wrote:
>"Isolated"? What do you mean?

Making a separate setup script for Python 3, at least for setuptools 
itself, if not having a general convention for that, since other 
packages may want to ship 2+3 stuff in the same package.

Or, in the alternative, using version testing in setup.py to run an 
alternate script for Python 3.


>If you have suggestions on how to solve
>the problems I'm describing, can you please come forward with them,
>I've asked several times already. All you do is say "no, no, no, no,
>no", and "becasue". It isn't helpful.

I'm sorry you feel that way, as I've been *trying* to help.  I just 
still don't get what the problem is.  If I were porting setuptools to 
Python 3, I would *want* it to be circular, even if I had to hack on 
it a little at first.  So I have a hard time understanding why you don't.

But I'm not you -- I understand the code base and I'm not trying to 
port it.  Maybe if I were trying to port it, I would get what problem 
you're having, or maybe I would just keep right on going and not 
notice.  I don't know.  I've been trying to find out what exactly is 
stopping you and just can't seem to wrap my brain around it, any more 
than you've been able to about the reverse.


> > For end-user convenience.  A large number of people installing setuptools
> > are not installing it because they are personally interested in it, but
> > because something else they want uses it.
>
>Yes? And again you don't explain how this leads to you conclusion.

Mostly because your questions aren't pinpointing what you want to 
know.  You ask general questions, so I give general answers.  We seem 
to both be suffering from a surplus of assumptions of what should be 
"obvious" to the other person.  For example, this:

>Eggs are not easier to install, on the contrary, I have tried and
>failed a couple of times, and ended up using the source install
>instead.

...seems to indicate that your question was actually about eggs, not 
other-than-source distributions in general.  I was mostly talking 
about the wininst installers and source RPM.  The eggs are there, on 
the other hand, for ez_setup.py to download. (Not to mention 
buildout's bootstrap script, and other tools that depend on 
setuptools and want to have an automated overall install process.)


From regebro at gmail.com  Mon Apr 20 22:12:17 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 22:12:17 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420194947.377203A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>

On Mon, Apr 20, 2009 at 21:52, P.J. Eby <pje at telecommunity.com> wrote:
> Making a separate setup script for Python 3, at least for setuptools itself,
> if not having a general convention for that, since other packages may want
> to ship 2+3 stuff in the same package.

The typical setup script will look exactly the same under python 2 and
python 3. There is no need for separate scripts in the general
usecase. If you want to run 2to3 automatically, all you need to do is
set up build_py_2to3 instead of build_2to3, that's the only
difference, and that's easily fixed with a importexception.

This goes pretty much for setuptools also. The setup3.py script will
more or less work under python2 as well.

> Or, in the alternative, using version testing in setup.py to run an
> alternate script for Python 3.

You don't need alternative scripts.

setuptools is an exception, because it depends on itself, providing a
catch22 situation.

> I'm sorry you feel that way, as I've been *trying* to help. ?I just still
> don't get what the problem is. ?If I were porting setuptools to Python 3, I
> would *want* it to be circular, even if I had to hack on it a little at
> first. ?So I have a hard time understanding why you don't.

But it CAN NOT be circular under Python 3.

> ?Maybe if I were trying to port it, I would get what problem you're having,
> or maybe I would just keep right on going and not notice. ?I don't know.

You will have it and I explained in the mail I sent as a start of this
discussion. If I was unclear, please tell me what you didn't
understand.

> ?I've been trying to find out what exactly is stopping you and just can't
> seem to wrap my brain around it, any more than you've been able to about the
> reverse.

> Mostly because your questions aren't pinpointing what you want to know.

It is pinpointing them exactly. I want to know why setuptools need to
depends on setuptools. Your answer, as I can understand it is for
convenience, and so that it serves as a test and example of it's own
features.

The fact that is serves as a test of it's own features is another
pain. That was a big reason for the difficulty of porting, as even
when testcases all passed, not all features worked. So I have to say
that although it sounds reasonable, I think it's misguided.

>> Eggs are not easier to install, on the contrary, I have tried and
>> failed a couple of times, and ended up using the source install
>> instead.
>
> ...seems to indicate that your question was actually about eggs, not
> other-than-source distributions in general.

Which probably is why I said eggs...

> ?The eggs are there, on the other hand,
> for ez_setup.py to download. (Not to mention buildout's bootstrap script,
> and other tools that depend on setuptools and want to have an automated
> overall install process.)

OK, I wonder if there is a way around that. If not, then as far as I
can see, there is no way to install or develop with setuptools
smoothly in Python 3.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Mon Apr 20 22:16:08 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 20 Apr 2009 22:16:08 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
Message-ID: <319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>

2009/4/20 Lennart Regebro <regebro at gmail.com>:
> At least somebody could maybe make some sort of rough plan of what
> this golden distutils should contain and how it should work, and we
> can get a feeling if it's even remotely feasible.

And, as I frequently run into walls that make me thing setuptools
should be completely ignored, and then after fiddling about quite a
bit, find a way around it, and then run into the next wall, etc, etc,
etc. And these walls are getting more and more frequent... I'm
beginning to think that indeed rewrites are the only way.

We need a plan. Somebody please make a plan, maybe I can help. But
probably not. But I can try.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From chrish at apstat.com  Mon Apr 20 22:49:08 2009
From: chrish at apstat.com (Christian Hudon)
Date: Mon, 20 Apr 2009 16:49:08 -0400
Subject: [Distutils] Help making setuptools install more like plain
 distutils one
In-Reply-To: <20090420190124.0638B3A4070@sparrow.telecommunity.com>
References: <49ECB0D3.6050905@apstat.com>
	<20090420190124.0638B3A4070@sparrow.telecommunity.com>
Message-ID: <49ECDFC4.6050407@apstat.com>

P.J. Eby wrote:
> At 01:28 PM 4/20/2009 -0400, Christian Hudon wrote:
>> Is there a way to ask setuptools to do an install that looks more 
>> like a standard distutils install?
>
> Yes, use "setup.py install --single-version-externally-managed 
> --record=/some/file".  This will install the distutils way, and record 
> all the installed files in /some/file (which you can then discard if 
> you like).
Thanks for the answers. I looked at both options, and the 
--single-version-externally-managed one would be easier for me: compared 
to the bdist_dumb, it saves me the steps of finding and untarring the 
tarball. Also, the bdist_dumb commands creates tarballs that can be 
untarred at "/" (they contain /usr, etc.) while I would need something 
more like the equivalent of what --prefix does (tarball that start with 
lib, etc. that can be untarred in /usr/local or /usr...)

The only thing I'd need would be way to reliably determine if I'm 
dealing with setuptools-based setup.py or not. I can always call "python 
setup.py installl --help" and look if 
--single-version-externally-managed is present in the output. Is there a 
cleaner version of doing this?

Thanks for the help,

  Christian


From robert.kern at gmail.com  Mon Apr 20 23:24:08 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 Apr 2009 16:24:08 -0500
Subject: [Distutils] Help making setuptools install more like plain
	distutils one
In-Reply-To: <49ECDFC4.6050407@apstat.com>
References: <49ECB0D3.6050905@apstat.com>	<20090420190124.0638B3A4070@sparrow.telecommunity.com>
	<49ECDFC4.6050407@apstat.com>
Message-ID: <gsip5o$aj8$1@ger.gmane.org>

On 2009-04-20 15:49, Christian Hudon wrote:

> The only thing I'd need would be way to reliably determine if I'm
> dealing with setuptools-based setup.py or not. I can always call "python
> setup.py installl --help" and look if
> --single-version-externally-managed is present in the output. Is there a
> cleaner version of doing this?

You could ensure that you always have a setuptools-based setup.py:

python -c "import setuptools; execfile('setup.py')" install --single-version-.....

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From chrish at apstat.com  Mon Apr 20 23:52:56 2009
From: chrish at apstat.com (Christian Hudon)
Date: Mon, 20 Apr 2009 17:52:56 -0400
Subject: [Distutils] Help making setuptools install more like
 plain	distutils one
In-Reply-To: <gsip5o$aj8$1@ger.gmane.org>
References: <49ECB0D3.6050905@apstat.com>	<20090420190124.0638B3A4070@sparrow.telecommunity.com>	<49ECDFC4.6050407@apstat.com>
	<gsip5o$aj8$1@ger.gmane.org>
Message-ID: <49ECEEB8.1080308@apstat.com>

Robert Kern wrote:
> On 2009-04-20 15:49, Christian Hudon wrote:
>
>> The only thing I'd need would be way to reliably determine if I'm
>> dealing with setuptools-based setup.py or not. I can always call "python
>> setup.py installl --help" and look if
>> --single-version-externally-managed is present in the output. Is there a
>> cleaner version of doing this?
>
> You could ensure that you always have a setuptools-based setup.py:
>
> python -c "import setuptools; execfile('setup.py')" install 
> --single-version-.....
>
Tried it. It works great! Wouldn't have thought of that... Thanks!

  Christian


From a.cavallo at acm.org  Tue Apr 21 05:05:48 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Tue, 21 Apr 2009 04:05:48 +0100
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090420170156.695633A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<319e029f0904200931k2749a9aj6f35c1d313dd8f89@mail.gmail.com>
	<20090420170156.695633A4070@sparrow.telecommunity.com>
Message-ID: <a71153f0904202005t33047e3ep204c4fa1a9a4fe98@mail.gmail.com>

On Mon, Apr 20, 2009 at 6:04 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:31 PM 4/20/2009 +0200, Lennart Regebro wrote:
>>
>> Let me reformulate that:
>>
>> > Because that's the one that generates the metadata setuptools needs to
>> > run,
>> > test itself, etc.
>>
>> Why do I need setuptools to do that? Why is not distutils enough?
>
> Because distutils doesn't have an egg_info command, or SVN-based manifest
> building (for making sdists).

Why the SVN handling should be part of setup.py anyway?
Regards,
Antonio

From a.cavallo at acm.org  Tue Apr 21 05:31:17 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Tue, 21 Apr 2009 04:31:17 +0100
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
Message-ID: <a71153f0904202031n24924440x1f822faf9e9c6837@mail.gmail.com>

> And, as I frequently run into walls that make me thing setuptools
> should be completely ignored, and then after fiddling about quite a
> bit, find a way around it, and then run into the next wall, etc, etc,
> etc. And these walls are getting more and more frequent... I'm
> beginning to think that indeed rewrites are the only way.
>
> We need a plan. Somebody please make a plan, maybe I can help. But
> probably not. But I can try.

I second that especially because I think setuptools looks like an over
engineering excercise: I have many doubts about about setuptools,
especially from a system administrative point of view.

Why a build system (and setup.py is) should have a scm support in it?
Why the plugin entry points should be part of such a system?
Why the dependecied should be part of it?

The scm support is part of a "development" stage, possibly useful to
the developers only: that'd be better covered by a python script not
tighted to the "building system".

The plugin entry points (belonging to a "deployment" stage) are a bit
of a problem because the paths might not be in their "final" form
(think of DESTDIR temporary staging area while building a rpm): this
mean the onlyreliable way to define entry points would be relative to
modulename.__file__ variable.

The dependencies part (belonging to a "deployment" stage) is a bit of
a system wide problem.
Rpm defines the same fields and then this introduce a (not so) hidden
duplication: will the package respect the python dependencies or the
rpm ones?

I would recomend having a look at what rpm (and especially the spec
files) does because it had embedded a lot of knoweledge about building
packages across the years: and it still has its own problems....

Moving forward, itwould be beneficial a flow chart of what distutils
should do (and shouldn't) and a possibly a usage cases study.
Regards,
Antonio

From david at ar.media.kyoto-u.ac.jp  Tue Apr 21 05:34:33 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 21 Apr 2009 12:34:33 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904200802q53e3c30ds7ebe9d3d5af940ef@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>	
	<87myabmzgq.fsf@benfinney.id.au>	
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>	
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200802q53e3c30ds7ebe9d3d5af940ef@mail.gmail.com>
Message-ID: <49ED3EC9.6090505@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
>
> In the short term, Distutils needs to be a better citizen when people
> wants to extend it, so they can do it without re-writing everything.
>
> You have this experience of "pain", share it through real use cases.
>   

I have added 5 usecases in the manifest plugin page, in order of
difficulty/pain. The first ones should be solvable in distutils with
your proposal, I am not sure about the last ones.

David

From greg.ewing at canterbury.ac.nz  Tue Apr 21 07:19:50 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 21 Apr 2009 17:19:50 +1200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
Message-ID: <49ED5776.3020304@canterbury.ac.nz>

Lennart Regebro wrote:

> We need a plan. Somebody please make a plan

Whatever plan you come up with, I'd like to see the
basic architecture built around a dependency graph,
at least for the parts concerned with building
extension modules from sources.

My use case for handling Pyrex sources. I'd like
to be able to plug in a module that knows how
to turn .pyx files into .c files, without requiring
arcane internal knowledge and without interfering
with any other extensions I might want to use at
the same time.

-- 
Greg

From david.lyon at preisshare.net  Tue Apr 21 07:32:04 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 01:32:04 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
Message-ID: <0556c54a1cea8f61e6f50cfce2ca81e1@preisshare.net>

Hi Lennart,

Sorry for my late posting...

I'm working on a gui python package manager on sourceforge...

> Executive summary:
> I have catch 22 problems in finishing the setuptools port to Python 3,
> and unless somebody can come up with alternatives, I may rip out
> setuptools dependency on itself completely, and only depend on
> distutils.

...

It's not easy I agree...

I am a GUI junky myself... :-) I don't use chainsaw (code) unless i need to

I'm trying out all the installers on one machine...

einstaller won't install because I already installed setuptools....

ez_installer won't de-install setuptools...

pip doesn't want to go on either...

I will figure it out... I am sympathising saying this package stuff
is hard going....

> I don't have a good solution to this, unless we can drop setuptools
> dependency on setuptools completely, and just use plain distutils for
> installing and testing setuptools. This will mean no more setuptools
> source eggs, but then I don't see the purpose of those, really.
> Arguments for not doing this and alternative solutions are highly
> welcome.

:-)

me too



From ziade.tarek at gmail.com  Tue Apr 21 08:44:05 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 08:44:05 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49ED5776.3020304@canterbury.ac.nz>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
Message-ID: <94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>

Please Greg, add your use case here with David's one, at the end of the page,
since we are trying to work on a solution.

http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft

On Tue, Apr 21, 2009 at 7:19 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lennart Regebro wrote:
>
>> We need a plan. Somebody please make a plan
>
> Whatever plan you come up with, I'd like to see the
> basic architecture built around a dependency graph,
> at least for the parts concerned with building
> extension modules from sources.
>
> My use case for handling Pyrex sources. I'd like
> to be able to plug in a module that knows how
> to turn .pyx files into .c files, without requiring
> arcane internal knowledge and without interfering
> with any other extensions I might want to use at
> the same time.
>
> --
> Greg
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From regebro at gmail.com  Tue Apr 21 08:54:22 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 21 Apr 2009 08:54:22 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
Message-ID: <319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>

On Tue, Apr 21, 2009 at 08:44, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Please Greg, add your use case here with David's one, at the end of the page,
> since we are trying to work on a solution.
>
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft

I realized another usecase, which may or may not be covered already,
should I add it to the page?


== Optional extensions ==

Some modules, such as zope.interface, have C-extensions which are only
performance enhancing and have pure-python fallbacks. It should be
easy to write an a command that if it fails doesn't stop the rest of
the build, and also includes the right files if it succeeds, but
doesn't fail when these files can't be built.



-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From david.lyon at preisshare.net  Tue Apr 21 09:11:12 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 03:11:12 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
Message-ID: <1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>


>> Please Greg, add your use case here with David's one, at the end of the
> page,
>> since we are trying to work on a solution.
>>
>> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft

I can identify three major areas of concern:

 1) Platform testing/builds (via cloud computing)
 2) External build tools
 3) too many versions of python

One thing you guys should also consider is automated building/testing
of packages. On different platforms.

Let me give you a practical example...

package win32...

latest version.. no longer works on windows 2000...

only works on xp... or vista...

Bad for me.. takes me time to figure out whats wrong...

Only solution.. go back and find an older release.. yes.. i know..
I can do that... that works...

With cloud computing... you can get test environments for every platform

Amazon web services as one example offer this 

Perl does nightly "builds" and tests....

their automated test system "tests everything"

Another big issue i dislike with many packages... they depend on
having msvc installed to be able to build them.

In many cases because I don't have MSC installed... this prevents
me from building some packages. I don't like that...

I don't believe building "everything" on the 5-10 major platforms
we have (or however many there are) is too much.

Plus now, we have all these different versions of python.. and
instead of getting better it is only seeming to get more worse...

David






From ziade.tarek at gmail.com  Tue Apr 21 09:21:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 09:21:12 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
	<1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
Message-ID: <94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>

> One thing you guys should also consider is automated building/testing
> of packages. On different platforms.
>
> Let me give you a practical example...
>
> package win32...
>
> latest version.. no longer works on windows 2000...
>
> only works on xp... or vista...
>
> Bad for me.. takes me time to figure out whats wrong...
>
> Only solution.. go back and find an older release.. yes.. i know..
> I can do that... that works...
>
> With cloud computing... you can get test environments for every platform
>
> Amazon web services as one example offer this
>
> Perl does nightly "builds" and tests....
>
> their automated test system "tests everything"
>
> Another big issue i dislike with many packages... they depend on
> having msvc installed to be able to build them.
>
> In many cases because I don't have MSC installed... this prevents
> me from building some packages. I don't like that...
>
> I don't believe building "everything" on the 5-10 major platforms
> we have (or however many there are) is too much.
>
> Plus now, we have all these different versions of python.. and
> instead of getting better it is only seeming to get more worse...

I had this idea of running a test server that would listen to PyPI and
get every new package to build it and test it on various platform/python.

I think this could be done right now, with how distutils works,

It requires creating virtual machines and destroying them every time
for security reasons.

But it's quite a work...

Cheers
Tarek
-- 
Tarek Ziad? | http://ziade.org

From mihamina at lab.vectoris.fr  Tue Apr 21 08:45:24 2009
From: mihamina at lab.vectoris.fr (Mihamina Rakotomandimby (R12y))
Date: Tue, 21 Apr 2009 09:45:24 +0300
Subject: [Distutils] mirror plone buildout
Message-ID: <49ED6B84.9090905@lab.vectoris.fr>

Hi,
I use a quite bad internet connection:
- Small bandwidth (128KBps)
- Many micro stalls
- 1s latency to the world...

I would like to first download the Plone (+deps) locally, with a dedicated
FTP client, that can handle more or less correctly the stalls.

Have you got some related documentation somewhere in your bookmarks?
-- 
                              Chef de projet chez Vectoris
                                  Phone: +261 33 11 207 36
System: xUbuntu 8.10 with almost all from package install
    http://www.google.com/search?q=mihamina+rakotomandimby

From hannosch at hannosch.eu  Tue Apr 21 09:31:47 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Tue, 21 Apr 2009 09:31:47 +0200
Subject: [Distutils] mirror plone buildout
In-Reply-To: <49ED6B84.9090905@lab.vectoris.fr>
References: <49ED6B84.9090905@lab.vectoris.fr>
Message-ID: <49ED7663.3010507@hannosch.eu>

Mihamina Rakotomandimby (R12y) wrote:
> I would like to first download the Plone (+deps) locally, with a dedicated
> FTP client, that can handle more or less correctly the stalls.
> 
> Have you got some related documentation somewhere in your bookmarks?

Plone offers buildout based installers for all platforms. Look at
http://plone.org/products/plone/releases/3.2.2 to get the most recent
stable release. You download one file which includes all dependencies in it.

Hanno


From ziade.tarek at gmail.com  Tue Apr 21 09:34:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 09:34:06 +0200
Subject: [Distutils] Distutils work, roadmap
Message-ID: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>

Remainder about Distutils:

- we did a summit at Pycon about distutils
- some people started to work on various tasks (PEPs, etc)

Can everyone here who says "distutils sucks" "setuptools sucks",
"let's rewrite them from scratch" hear that:

Just join the work in progress, help on the PEP, uses cases, and stop
whining about the situation !!!!

Thank you

-- 
Tarek Ziad? | http://ziade.org

From eric at trueblade.com  Tue Apr 21 09:54:34 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 21 Apr 2009 03:54:34 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	<49EC8883.10106@ar.media.kyoto-u.ac.jp>	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
Message-ID: <49ED7BBA.60605@trueblade.com>

Tarek Ziad? wrote:
> Please Greg, add your use case here with David's one, at the end of the page,
> since we are trying to work on a solution.
> 
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft
>

Could the people adding the use cases put their names by them, so if we 
have questions later we know who to ask?

From david at ar.media.kyoto-u.ac.jp  Tue Apr 21 10:06:30 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 21 Apr 2009 17:06:30 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49ED7BBA.60605@trueblade.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	<49EC8883.10106@ar.media.kyoto-u.ac.jp>	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>	<49ED5776.3020304@canterbury.ac.nz>	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<49ED7BBA.60605@trueblade.com>
Message-ID: <49ED7E86.8000806@ar.media.kyoto-u.ac.jp>

Eric Smith wrote:
> Tarek Ziad? wrote:
>> Please Greg, add your use case here with David's one, at the end of
>> the page,
>> since we are trying to work on a solution.
>>
>> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft
>>
>
> Could the people adding the use cases put their names by them, so if
> we have questions later we know who to ask?

Good idea, I have added my names to 'my' examples,

David

From david.lyon at preisshare.net  Tue Apr 21 11:11:09 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 05:11:09 -0400
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
Message-ID: <6da942828b47e111fd9de4c00cd20791@preisshare.net>


Tarek,

I wouldn't take "feedback" as all bad....

Indirectly, it seems like you are telling people not to report their
experience. Indirectly, it seems like you are discouraging thinking
about what could be done to improve things.

Here, we have a popular saying "Nothing ever gets fixed if people
don't complain."

A roadmap is a good idea...

I think distutils needs one...

David

On Tue, 21 Apr 2009 09:34:06 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> Remainder about Distutils:
> 
> - we did a summit at Pycon about distutils
> - some people started to work on various tasks (PEPs, etc)
> 
> Can everyone here who says "distutils sucks" "setuptools sucks",
> "let's rewrite them from scratch" hear that:
> 
> Just join the work in progress, help on the PEP, uses cases, and stop
> whining about the situation !!!!
> 
> Thank you
> 
>


From david.lyon at preisshare.net  Tue Apr 21 11:17:19 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 05:17:19 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
	<1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
	<94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>
Message-ID: <529b5563c114eaa55df6e25b396eb8ea@preisshare.net>


On Tue, 21 Apr 2009 09:21:12 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
 
> I had this idea of running a test server that would listen to PyPI and
> get every new package to build it and test it on various platform/python.
> 
> I think this could be done right now, with how distutils works,
> 
> It requires creating virtual machines and destroying them every time
> for security reasons.
> 
> But it's quite a work...

I think it is very important to put this software infrastructure into
place.

or at least include it in some sort of plan going forward....

have the server farm report build bugs back to the maintainers...

I am prepared to assist with this... and perphaps cover some of the costs.

Amazon web services provide the very same virtual machines that you are
describing at extremely low cost. Otherwise it is just two much work.

Google also offer a similar thing.


David


From ben+python at benfinney.id.au  Tue Apr 21 11:20:20 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 21 Apr 2009 19:20:20 +1000
Subject: [Distutils] Distutils work, roadmap
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
Message-ID: <87ab6ajrcb.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> Just join the work in progress, help on the PEP, uses cases, and stop
> whining about the situation !!!!

URLs please.

> Thank you

And you.

-- 
 \        ?You can be a victor without having victims.? ?Harriet Woods |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Tue Apr 21 11:21:37 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 11:21:37 +0200
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <6da942828b47e111fd9de4c00cd20791@preisshare.net>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<6da942828b47e111fd9de4c00cd20791@preisshare.net>
Message-ID: <94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>

On Tue, Apr 21, 2009 at 11:11 AM, David Lyon <david.lyon at preisshare.net> wrote:
>
> Tarek,
>
> I wouldn't take "feedback" as all bad....
>
> Indirectly, it seems like you are telling people not to report their
> experience. Indirectly, it seems like you are discouraging thinking
> about what could be done to improve things.

I am encouraging people to work with the people that have started
to work on various topics durong Pycon

>
> Here, we have a popular saying "Nothing ever gets fixed if people
> don't complain."
>
> A roadmap is a good idea...
>
> I think distutils needs one...
>


The rough roadmap was expressed during pycon

main page : http://wiki.python.org/moin/Distutils

main topics:

- have a standard version comparison API -
http://wiki.python.org/moin/DistutilsVersionFight
- standardize egg-info: http://wiki.python.org/moin/Distutils/StandardizeEggInfo
- finalize the new PEP 345 - http://wiki.python.org/moin/Distutils/Metadata
- work on the static metadata concept -
http://wiki.python.org/moin/Distutils/StaticMetadata
- work on extendding commands -
http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft


> David
>
> On Tue, 21 Apr 2009 09:34:06 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>> Remainder about Distutils:
>>
>> - we did a summit at Pycon about distutils
>> - some people started to work on various tasks (PEPs, etc)
>>
>> Can everyone here who says "distutils sucks" "setuptools sucks",
>> "let's rewrite them from scratch" hear that:
>>
>> Just join the work in progress, help on the PEP, uses cases, and stop
>> whining about the situation !!!!
>>
>> Thank you
>>
>>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Apr 21 11:23:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 11:23:55 +0200
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <87ab6ajrcb.fsf@benfinney.id.au>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<87ab6ajrcb.fsf@benfinney.id.au>
Message-ID: <94bdd2610904210223q3192081bv8de7b18d9cf5bb18@mail.gmail.com>

On Tue, Apr 21, 2009 at 11:20 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>> Just join the work in progress, help on the PEP, uses cases, and stop
>> whining about the situation !!!!
>
> URLs please.
>
>> Thank you
>
> And you.

See my answer to David,

I'll work on the wiki to make things more clear in the coming days.
(for instance adding the name of the "leader"
in front of each task)

Feel free to add some tasks, I'll eventually make sure they are not
part of an existing task

From ziade.tarek at gmail.com  Tue Apr 21 11:30:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 11:30:55 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <529b5563c114eaa55df6e25b396eb8ea@preisshare.net>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
	<1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
	<94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>
	<529b5563c114eaa55df6e25b396eb8ea@preisshare.net>
Message-ID: <94bdd2610904210230j28b4e88bv7d0143fa179694dc@mail.gmail.com>

On Tue, Apr 21, 2009 at 11:17 AM, David Lyon <david.lyon at preisshare.net> wrote:
>> But it's quite a work...
>
> I think it is very important to put this software infrastructure into
> place.
>
> or at least include it in some sort of plan going forward....
>
> have the server farm report build bugs back to the maintainers...
>
> I am prepared to assist with this... and perphaps cover some of the costs.
>
> Amazon web services provide the very same virtual machines that you are
> describing at extremely low cost. Otherwise it is just two much work.
>
> Google also offer a similar thing.

Wonderful ! Money-wise, I think the PSF would probably cover these
expenses if we come up
with a good, well-defined project. I have tried to make it a GSoc
topic but I didn't find any student
picking it up because I didn't find the time to describe it deeply enough.

I have created a page here :
http://wiki.python.org/moin/Distutils/TestingInfrastructure

If you want to start there on the topic, that could be great.

Tarek

>
>
> David
>
>



-- 
Tarek Ziad? | http://ziade.org

From regebro at gmail.com  Tue Apr 21 11:32:31 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 21 Apr 2009 11:32:31 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904210230j28b4e88bv7d0143fa179694dc@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
	<1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
	<94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>
	<529b5563c114eaa55df6e25b396eb8ea@preisshare.net>
	<94bdd2610904210230j28b4e88bv7d0143fa179694dc@mail.gmail.com>
Message-ID: <319e029f0904210232q59c69fd1p6a5cf941c8c509f9@mail.gmail.com>

Maybe we should stop talking about how to improve distutils under the
topic of what the problem is with setuptools under Python 3? It's
pretty unrelated. ;-)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From david.lyon at preisshare.net  Tue Apr 21 12:13:22 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 06:13:22 -0400
Subject: [Distutils] The problem with Setuptools on Python 1*, 2.* 3.*
In-Reply-To: <49EC8007.6090103@ar.media.kyoto-u.ac.jp>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200126j1955a7d9m28052a2b20485bb7@mail.gmail.com>
	<87myabmzgq.fsf@benfinney.id.au>
	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
Message-ID: <34bfbe1fc11dd205a00ba08c5eab667e@preisshare.net>


I really second this...

after all.. we're only talking about a file copying program..

Surely after so many years... we must have a clear handle on what
files need to be copied where with what permissions....

On Mon, 20 Apr 2009 23:00:39 +0900, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
 
> IMHO, adding plugins systems will not change the fundamental
> deficiencies of distutils, though. You may be able to mitigate some
> problems, but I don't think you will be able to solve the fundamental
> issues, because they are fundamental design issues. The only way to fix
> them would induce serious breakage, and in that case, given the quality
> of distutils code, starting from scratch would be easier - I have yet
> encountered a single major distutils abstraction which was not poorly
> conceived and implemented (my own experience is related to build issues,
> for which numpy has requirements which go much beyond the usual python
> package, though). Commands classes, compiler classes, distributions, all
> this is very wrong for someone who wants more than compiling a couple of
> C files and tight control.



From p.f.moore at gmail.com  Tue Apr 21 12:37:46 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 21 Apr 2009 11:37:46 +0100
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<6da942828b47e111fd9de4c00cd20791@preisshare.net>
	<94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>
Message-ID: <79990c6b0904210337q44808e23m26c20970794ae3ab@mail.gmail.com>

2009/4/21 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Tue, Apr 21, 2009 at 11:11 AM, David Lyon <david.lyon at preisshare.net> wrote:
>>
>> Tarek,
>>
>> I wouldn't take "feedback" as all bad....
>>
>> Indirectly, it seems like you are telling people not to report their
>> experience. Indirectly, it seems like you are discouraging thinking
>> about what could be done to improve things.
>
> I am encouraging people to work with the people that have started
> to work on various topics durong Pycon

Given that I have not got the time or experience with setuptools or
distutils to contribute effort to fixing things, you seem to be saying
that my experience as an end user of these tools, and my spending my
time trying to understand the implications of the proposals so that I
can offer useful feedback, is not of interest.

So be it. I'll stop commenting.

Please let me know when "the work" is complete, and end user feedback
(or "whining", if you prefer) is welcome again.

Paul.

From a.cavallo at acm.org  Tue Apr 21 13:25:30 2009
From: a.cavallo at acm.org (Antonio Cavallo)
Date: Tue, 21 Apr 2009 12:25:30 +0100
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904210230j28b4e88bv7d0143fa179694dc@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<319e029f0904202354i240cb376w1a2491789407068e@mail.gmail.com>
	<1b57b134ab4a83e0cdf9e000a7dad109@preisshare.net>
	<94bdd2610904210021q442eea0fjaf88a433e346e092@mail.gmail.com>
	<529b5563c114eaa55df6e25b396eb8ea@preisshare.net>
	<94bdd2610904210230j28b4e88bv7d0143fa179694dc@mail.gmail.com>
Message-ID: <a71153f0904210425y6f235f04m95cbda5f2b250fbb@mail.gmail.com>

SuSE runs a build service for free and support automatic rebuild of
packages from sources:


https://build.opensuse.org


For anyone interested you can find the lates svn python snapshot under:

http://download.opensuse.org/repositories/home:/cavallo71:/python-opt/

Each subdirectory (CentOS_5, RHEL_5 etc) contains a yum repository
to download the packages opt-python-XXXX

These python packages install under their own separate directory tree
(/opt/opt-python-2.7a0 is the latest snapshot).

In order to use one of them all it is needed is:
$> . /opt/opt-python-2.7a0/opt-python-env.sh
$> python

Python 2.7a0 (trunk, Apr 20 2009, 12:45:44)
[GCC 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os; print os.__file__
/opt/opt-python-2.7a0/lib/python2.7/os.pyc


I hope this helps in testing,
Regards,
Antonio

On Tue, Apr 21, 2009 at 10:30 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Tue, Apr 21, 2009 at 11:17 AM, David Lyon <david.lyon at preisshare.net> wrote:
>>> But it's quite a work...
>>
>> I think it is very important to put this software infrastructure into
>> place.
>>
>> or at least include it in some sort of plan going forward....
>>
>> have the server farm report build bugs back to the maintainers...
>>
>> I am prepared to assist with this... and perphaps cover some of the costs.
>>
>> Amazon web services provide the very same virtual machines that you are
>> describing at extremely low cost. Otherwise it is just two much work.
>>
>> Google also offer a similar thing.
>
> Wonderful ! Money-wise, I think the PSF would probably cover these
> expenses if we come up
> with a good, well-defined project. I have tried to make it a GSoc
> topic but I didn't find any student
> picking it up because I didn't find the time to describe it deeply enough.
>
> I have created a page here :
> http://wiki.python.org/moin/Distutils/TestingInfrastructure
>
> If you want to start there on the topic, that could be great.
>
> Tarek
>
>>
>>
>> David
>>
>>
>
>
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From ronaldoussoren at mac.com  Tue Apr 21 14:00:10 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 21 Apr 2009 14:00:10 +0200
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <20090419131032.GC19409@fridge.pov.lt>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<20090419131032.GC19409@fridge.pov.lt>
Message-ID: <0C774AF0-D53B-4E70-A1F0-7EFFC53BE5BE@mac.com>



> On Sat, Apr 18, 2009 at 11:09:11AM +0200, Lennart Regebro wrote:
>> I don't actually know what features of setuptools people use.
>

* Dependency specification (install_requires, etc)

* Virtualenv + easy_install for installation

* Entrypoints, both for specifying console_scripts and as an easy way  
to provide plugable hooks in an application

* Specifying new distutils commands as setuptools entrypoints

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090421/cfa4454d/attachment.bin>

From ronaldoussoren at mac.com  Tue Apr 21 14:15:45 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 21 Apr 2009 14:15:45 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
Message-ID: <94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>


On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
>
>> I'm sorry you feel that way, as I've been *trying* to help.  I just  
>> still
>> don't get what the problem is.  If I were porting setuptools to  
>> Python 3, I
>> would *want* it to be circular, even if I had to hack on it a  
>> little at
>> first.  So I have a hard time understanding why you don't.
>
> But it CAN NOT be circular under Python 3.

I don't understand why not, doing it may be not entirely trivial but  
it should be possible with some trickery.  As PJE noted one way to do  
this is to explicitly convert setuptools to python 3.x syntax before  
actually running setup.py (e.g. his setup3.py file).  With some care  
this could even be done in setup.py itself.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090421/d7f2ca17/attachment-0001.bin>

From regebro at gmail.com  Tue Apr 21 14:26:23 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 21 Apr 2009 14:26:23 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
Message-ID: <319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>

On Tue, Apr 21, 2009 at 14:15, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> I don't understand why not, doing it may be not entirely trivial but it
> should be possible with some trickery.  As PJE noted one way to do this is
> to explicitly convert setuptools to python 3.x syntax before actually
> running setup.py (e.g. his setup3.py file).  With some care this could even
> be done in setup.py itself.

I'm sorry, I don't think I'm able to explain it more in detail than
what I did in the first post in this discussion, unless you are more
explicit about what I need to explain.

Here it is again, for convenience:

"Currently for the Python 3 version, I've made a script that copies the
source tree and runs 2to3 on it. Then you can use that new tree to run
tests, install, make eggs and releases etc. The problems with this is:

a) The script is made for my computer only. It wouldn't work on
Windows, and it requires you to have lib2to3 checked out in a
particular place (although I think that's only if you use 3.0.0, and
not 3.0.1, but I haven't tested).

b) It's slow, as it copies all files, even those who hasn't changed
when you fixed a bug. Making a script that only copies the changed
files and runs 2to3 on them is possible, but I'd like not too, it
takes time and is likely to break. It seems like a waste.

Obviously, this is exactly the things distutils are meant to solve.
And indeed, distutils in Python 3 has a command called build_2to3
which solves exactly these problems. And indeed, this is the technique
I've used for the Python 3 branch of zope.interfaces. There I just run
setup.py install, and if I'm doing that under Python 3, it will first
run 2to3 on the code before installing. Same thing with running
setup.py test. It will sync changes to build/, run 2to3 on changed
files and then run the tests in build. It makes porting to Python 3
much easier, and it makes installing from source painless.

So why don't I use that for setuptools? Well, because:

c) The setup of setuptools requires setuptools. So to be able to do
the 2to3 conversion in the setup, I need to first convert the source
with 2to3. Yes, catch 22.

I've tried to get around this problem, but failed. Solutions I tried was:

I) Fall back to distutils if setuptools can't be imported. This means
that I can with some effort make a setup that will work under Python 3
and install setuptools under Python 3. Problem solved? No. Because
running setup.py again will just mean that it tries to import
setuptools from the local Python2 location, and it will fail. So in
this case I can't run tests of build eggs for setuptool.

II) Moving the source into a src directory. This means that when
setup.py runs, it will use the version of setuptools installed under
I), and building eggs etc is possible (I think, I haven't tested that
the eggs are correct). Problem solved? No, because you still can't
test it. Which leads us to problem d:

d) When running the tests, the setuptools module will already be
imported. But it will have imported the one in site-packages, *not*
the one in src, and it will therefore test the one in site-packages.
And that one doesn't have the api_tests.txt copied in, so that will
fail, and

e) even if api_tests.txt was copied, this would mean you had to
install setuptools before you test it"


-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From eric at trueblade.com  Tue Apr 21 14:31:24 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 21 Apr 2009 08:31:24 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>	<20090420161437.9FD583A4070@sparrow.telecommunity.com>	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>	<20090420194947.377203A4070@sparrow.telecommunity.com>	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
Message-ID: <49EDBC9C.8020906@trueblade.com>

Ronald Oussoren wrote:
> 
> On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
>>
>>> I'm sorry you feel that way, as I've been *trying* to help.  I just 
>>> still
>>> don't get what the problem is.  If I were porting setuptools to 
>>> Python 3, I
>>> would *want* it to be circular, even if I had to hack on it a little at
>>> first.  So I have a hard time understanding why you don't.
>>
>> But it CAN NOT be circular under Python 3.
> 
> I don't understand why not, doing it may be not entirely trivial but it 
> should be possible with some trickery.  As PJE noted one way to do this 
> is to explicitly convert setuptools to python 3.x syntax before actually 
> running setup.py (e.g. his setup3.py file).  With some care this could 
> even be done in setup.py itself.

But even if it's possible to make it circular, is that really a good 
design? I think not.

Eric.

From ronaldoussoren at mac.com  Tue Apr 21 14:57:35 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 21 Apr 2009 14:57:35 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EDBC9C.8020906@trueblade.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420150508.5A56D3A4070@sparrow.telecommunity.com>
	<319e029f0904200902u1b07e255p4dbce69aa696266a@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<49EDBC9C.8020906@trueblade.com>
Message-ID: <D99D7361-A555-4AD9-923D-BB0B46A4BD73@mac.com>


On 21 Apr, 2009, at 14:31, Eric Smith wrote:

> Ronald Oussoren wrote:
>> On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
>>>
>>>> I'm sorry you feel that way, as I've been *trying* to help.  I  
>>>> just still
>>>> don't get what the problem is.  If I were porting setuptools to  
>>>> Python 3, I
>>>> would *want* it to be circular, even if I had to hack on it a  
>>>> little at
>>>> first.  So I have a hard time understanding why you don't.
>>>
>>> But it CAN NOT be circular under Python 3.
>> I don't understand why not, doing it may be not entirely trivial  
>> but it should be possible with some trickery.  As PJE noted one way  
>> to do this is to explicitly convert setuptools to python 3.x syntax  
>> before actually running setup.py (e.g. his setup3.py file).  With  
>> some care this could even be done in setup.py itself.
>
> But even if it's possible to make it circular, is that really a good  
> design? I think not.

Given what setuptools tries to do I don't think the circular design is  
a problem. A standalone distutils would have had the same problem.

In the long run the useful features of setuptools should incorperated  
into distutils, instead of monkeypatching distutils. That is, of  
course, an entirely different issue and one that's already being  
worked on.

Ronald
>
> Eric.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090421/c855d70c/attachment.bin>

From pje at telecommunity.com  Tue Apr 21 15:03:50 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 21 Apr 2009 09:03:50 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com
 >
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
Message-ID: <20090421130118.911BA3A40AC@sparrow.telecommunity.com>

At 02:26 PM 4/21/2009 +0200, Lennart Regebro wrote:
>So why don't I use that for setuptools? Well, because:
>
>c) The setup of setuptools requires setuptools. So to be able to do
>the 2to3 conversion in the setup, I need to first convert the source
>with 2to3. Yes, catch 22.

What I still don't get is why it can't work in a 2-stage process, 
running one setup.py with distutils to do the build_2to3, and then 
running a different setup.py to do the tests.

I imagine that, if I were trying to support Python 3, what I would do 
first is make a Python 2 setuptools command that ran 2to3 on a 
setuptools-based Python 2 project, and generated a new source tree -- 
with all sdist-targeted content copied over, and all .py files 
converted (including setup.py itself)...  and then ran whatever extra 
commands you gave, running Python 3 on the resulting setup.py, such that:

   python2 setup.py 2to3 test

would automatically do the equivalent of

   cd build/2to3; python3 setup.py test

after creating a converted distribution in build/2to3.  That way, you 
could also do things like:

   python2 setup.py test 2to3 test

to run the tests in Python 2 before converting and running them in Python 3.

If somebody wants to create this command, perhaps that would be a 
good idea.  It can of course be implemented as a plugin, so a change 
to setuptools itself is not required.

In the simplest case, the command could just derive from sdist and 
build an sdist tree in build/2to3 each time, and then run 2to3 in 
place.  Or it could reuse sdist and unpack the sdist into the build 
tree.  This would be slower, but easier to code.  A more advanced 
version could check for changes in SOURCES.txt between the original 
and the build/2to3 directory in order to find files to add/remove, 
and only run 2to3 on changed .py files.

Something like:

    if not os.path.exists(target_sources_txt):
        # wipe build tree, build sdist and unpack

    target_manifest = load_manifest(target_sources_txt)
    for filename in target_manifest:
        if filename not in original_manifest:
            # delete file
            continue
        if is_older_or_nonexistent(filename):
            # copy or run 2to3
    for filename in original_manifest:
        if filename not in target_manifest:
            # copy or run 2to3

...but with a lot more os.path.join operations and a lot less 
handwaving.  ;-)  And the initial version of this could just always 
do the wipe-and-unpack step, although it'd still need to loop over 
the files to run 2to3 anyway, I suppose.

Anyway, I know this is a fair amount of work; It just seems to me 
that it has more uses than converting setuptools; i.e., it'd be a 
useful rig for anybody doing 2-to-3 porting work.


From tseaver at palladion.com  Tue Apr 21 15:04:41 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 21 Apr 2009 09:04:41 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <a71153f0904202031n24924440x1f822faf9e9c6837@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	<49EC8883.10106@ar.media.kyoto-u.ac.jp>	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<a71153f0904202031n24924440x1f822faf9e9c6837@mail.gmail.com>
Message-ID: <gskg9b$jb2$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antonio Cavallo wrote:
>> And, as I frequently run into walls that make me thing setuptools
>> should be completely ignored, and then after fiddling about quite a
>> bit, find a way around it, and then run into the next wall, etc, etc,
>> etc. And these walls are getting more and more frequent... I'm
>> beginning to think that indeed rewrites are the only way.
>>
>> We need a plan. Somebody please make a plan, maybe I can help. But
>> probably not. But I can try.
> 
> I second that especially because I think setuptools looks like an over
> engineering excercise: I have many doubts about about setuptools,
> especially from a system administrative point of view.
> 
> Why a build system (and setup.py is) should have a scm support in it?

Setuptools builds on distutils, which is primarily aimed at distributing
software;  "building" is a minor part of the exercise (a huge
supermajority of distutils packages require no real "build" effort at all).

> Why the plugin entry points should be part of such a system?

Distributing plugins for Chandler was PJE's driving use case in the
original development of setuptools.

> Why the dependecied should be part of it?

Because distributing interrelated software sanely requires it.

> The scm support is part of a "development" stage, possibly useful to
> the developers only: that'd be better covered by a python script not
> tighted to the "building system".

I don't know where you get the idea that setuptools / distutils is
primarily about "building" software.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ7cRp+gerLs4ltQ4RAikNAJ43+6Cs8aCE0WufUsR/FiG3Xk8Z2gCgkIR1
46oMc5ShfytLkqppE2jN4do=
=Vhj/
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Tue Apr 21 15:08:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Apr 2009 15:08:08 +0200
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <79990c6b0904210337q44808e23m26c20970794ae3ab@mail.gmail.com>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<6da942828b47e111fd9de4c00cd20791@preisshare.net>
	<94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>
	<79990c6b0904210337q44808e23m26c20970794ae3ab@mail.gmail.com>
Message-ID: <94bdd2610904210608l63079430i5e16e300930c2200@mail.gmail.com>

On Tue, Apr 21, 2009 at 12:37 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/4/21 Tarek Ziad? <ziade.tarek at gmail.com>:
>> On Tue, Apr 21, 2009 at 11:11 AM, David Lyon <david.lyon at preisshare.net> wrote:
>>>
>>> Tarek,
>>>
>>> I wouldn't take "feedback" as all bad....
>>>
>>> Indirectly, it seems like you are telling people not to report their
>>> experience. Indirectly, it seems like you are discouraging thinking
>>> about what could be done to improve things.
>>
>> I am encouraging people to work with the people that have started
>> to work on various topics durong Pycon
>
> Given that I have not got the time or experience with setuptools or
> distutils to contribute effort to fixing things, you seem to be saying
> that my experience as an end user of these tools, and my spending my
> time trying to understand the implications of the proposals so that I
> can offer useful feedback, is not of interest.

I have never said that. On the contrary, I encourage you to fill your
use case as an end user
in the wiki.

>
> So be it. I'll stop commenting.
>
> Please let me know when "the work" is complete, and end user feedback
> (or "whining", if you prefer) is welcome again.

I think you misunderstood my point. I am reacting to the people that
are most of the time quite
involved in this area and are discussing in the various threads about
how Distutils is very bad code and should
be replaced from a new tool from scratch. This is going on for over a
year, and since Pycon
we are trying to push forward and write things down to try to fix things.

While discussing in this list is vital, having plans/use case written
down in the wiki, everyone can look at, work on,
is the way to go imho.

At the summit, we discussed about the raw roadmap validated by Guido,
and people have started to sprint.
Currently, various people are working on various pre-PEP, use cases, etc.

Some threads are going a little backward and I am scared to loose the momentum.

So my point is : please don't be offended by my previous mail, and
please don't stop to provide feedback.
That was a bad reaction I admit, I am sorry. I'll try to make things
clearer by updating the wiki page maybe,
or the python.org page



>
> Paul.
>



-- 
Tarek Ziad? | http://ziade.org

From zooko at zooko.com  Tue Apr 21 03:06:03 2009
From: zooko at zooko.com (Zooko O'Whielacronx)
Date: Mon, 20 Apr 2009 18:06:03 -0700
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
Message-ID: <FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>

Roughly in order of most important to least important:

1.  a formal, machine-readable way to specify our dependencies; see  
XYZ [2]

2.  automatic installation of dependencies;  This reduces the  
"dependencies" section in the instructions for installing Tahoe [1]  
from having sixteen separate software packages with separate  
versioning requirements to having one: Python 2.4.2 through 2.6.

3.  Obviously having even one "dependency" in your installation  
instructions is too many, so we try to provide .deb's, .rpm's, gentoo  
ebuilds, Windows installers (e.g. as produced by py2exe or bbfreeze),  
Macintosh applications, etc.  Many of the tools that produce these  
other formats rely on the above-mentioned feature #1 and/or feature #2.

4.  Some people like to use "easy_install $PACKAGE".  In fact, quite  
a lot of people -- if I recall the results of Tarek's packaging  
survey correctly, it was more popular among the responding Python  
programmers than any other method of installing packages [3].

5.  Using plugins so that you can add features to your build system,  
such as revision control integration, unit testing, versioning,  
pyflakes, uploading, etc. which you re-use as black box modules  
maintained by someone else instead of copying code into your setup.py  
or your GNUmakefile. [4, 5, 6, 7, 8].

6.  Using the console scripts feature of setuptools to make it so  
your scripts that you wrote for Unix also work on Windows.

7.  python setup.py sdist upload register

8.  revision control integration; I want all files under revision  
control to be included in packages.

Regards,

Zooko

From p.f.moore at gmail.com  Tue Apr 21 16:06:21 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 21 Apr 2009 15:06:21 +0100
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <94bdd2610904210608l63079430i5e16e300930c2200@mail.gmail.com>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<6da942828b47e111fd9de4c00cd20791@preisshare.net>
	<94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>
	<79990c6b0904210337q44808e23m26c20970794ae3ab@mail.gmail.com>
	<94bdd2610904210608l63079430i5e16e300930c2200@mail.gmail.com>
Message-ID: <79990c6b0904210706q5f076ecbuaf7aa5685be5c15c@mail.gmail.com>

2009/4/21 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Tue, Apr 21, 2009 at 12:37 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/4/21 Tarek Ziad? <ziade.tarek at gmail.com>:
>>> On Tue, Apr 21, 2009 at 11:11 AM, David Lyon <david.lyon at preisshare.net> wrote:
>>>>
>>>> Tarek,
>>>>
>>>> I wouldn't take "feedback" as all bad....
>>>>
>>>> Indirectly, it seems like you are telling people not to report their
>>>> experience. Indirectly, it seems like you are discouraging thinking
>>>> about what could be done to improve things.
>>>
>>> I am encouraging people to work with the people that have started
>>> to work on various topics durong Pycon
>>
>> Given that I have not got the time or experience with setuptools or
>> distutils to contribute effort to fixing things, you seem to be saying
>> that my experience as an end user of these tools, and my spending my
>> time trying to understand the implications of the proposals so that I
>> can offer useful feedback, is not of interest.
>
> I have never said that. On the contrary, I encourage you to fill your
> use case as an end user
> in the wiki.
[...]
> I think you misunderstood my point.

Hmm, OK. If that's the case then I should apologise. I am struggling
to understand the details of all these discussion threads, and that
probably coloured my response.

One concrete point - The categories on the wiki don't really give me
much help in knowing where to put my use case. I'm a user of Python
packages - all I care about is running the installer, or if I must
typing python setup.py bdist_wininst. So these aren't relevant, as
they are developer-oriented:

    DistutilsVersionFight
    Distutils/StandardizeEggInfo
    Distutils/Metadata
    Distutils/StaticMetadata
    Distutils/ManifestPluginSystem
    Distutils/TestingInfrastructure

This might be relevant, as it sounds like it relates to ensuring that
bdist_wininst installers are available. But it sounds more like it's
about maintaining bdist_wininst outside of the core - so I'm not sure
where I'd state that as a user I don't want to have to download a
3rd-party bdist_wininst before I can build my own packages if there
isn't a Windows installer. It sounds like there's an implicit
assumption there that I disagree with :-(

    Distutils/Friends : the goal is to try to find a project, a person
or a group of person on each platform that is willing to maintain a
third-party tool that build system-specific distros out of python
package.

Ultimately, though, my "use case" is simple. I want to download and
install Windows platform-specific installers for each package I use,
without needing to care about distutils or setuptools. To the extent
that I am exposed to the existence of distutils/setuptools (beyond the
utterly bare minimum of "python setup.py bdist_wininst"), my
requirement isn't being met - and I'll happily discuss whether I
should expect to lower my expectations, and what benefits there are
(to me, or others) in doing so. But that's it.

The sticking point here is that before setuptools, *my requirements
were largely being met*. Since setuptools appeared on the scene, they
are being eroded (witness projects which have switched to distributing
eggs rather than bdist_wininst installers). From that fact stems all
of my anti-setuptools views and comments (which I won't repeat now).

FWIW, I did post my use case here some days ago, with a request that
someone suggest/find a home for it on the wiki, as I wasn't sure where
it went. To my knowledge, no-one ever did that. I'd still appreciate
it if someone could copy it to the wiki. But trying to understand the
various threads and arguments on the list has sapped most of my
energy, so I'm not going to try to guess where it should go.

Paul.

From regebro at gmail.com  Tue Apr 21 16:06:25 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 21 Apr 2009 16:06:25 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090421130118.911BA3A40AC@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
Message-ID: <319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.com>

On Tue, Apr 21, 2009 at 15:03, P.J. Eby <pje at telecommunity.com> wrote:
> ?python2 setup.py 2to3 test

Well, yes, but it should be

   python 3 setup.py 2to3 test

Otherwise it can't reasonably have any idea of which python to use.
And when you run it with python3, the 2to3 command isn't needed, as
the 2to3 step can be automatically inserted through version detection.
In fact, this is exactly what I'm doing for zope.interface, as
explained. For zope.interface, you run
   python2.5 setup.py test
and it will test it with python2.5 You run
   python3.0 setup.py test
and it will do exactly what you say: copy the files to build, run
2to3, and then run the test on that in-place.
   python3.0 setup.py install
will copy the files to build, run 2to3 and then install it.

> Anyway, I know this is a fair amount of work; It just seems to me that it
> has more uses than converting setuptools; i.e., it'd be a useful rig for
> anybody doing 2-to-3 porting work.

Yes. That's is exactly what I'm trying to achieve. And I aim to extend
setuptools with build_2to3 and test_2to3 commands to do exactly that.
But to do that I need a good setuptools distribution that supports
Python3. And to do that, I need to make setuptools support Python 3 in
a way that is acceptable to somebody who can merge it to trunk and
make an official release. As far as I know that's you. If I can't find
a solution that is both acceptable to you, and makes Python 3 support
relatively painless, then my efforts have been in vain.

Your comment about setuptools using itself as a good test case did
make me think that this is the case. It explains why the test cases
could run even though setuptools still couldn't install itself. I
thought it was badly tested, now I know it's by design. Removing the
self-dependency then means we have to write a whole bunch of new test
cases.

I do not have the energy to do this. As mentioned I have already put
down a lot of effort into this, and I'm expac...experc... desperate
and have given up. Even writing this email makes me all physically
tired, besides taking a lot of time I should use to make money. I'm
getting too old for this. Somebody else can do it.

Over and out.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Tue Apr 21 19:57:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 21 Apr 2009 13:57:31 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.com>
Message-ID: <20090421175500.B00393A40AC@sparrow.telecommunity.com>

At 04:06 PM 4/21/2009 +0200, Lennart Regebro wrote:
>On Tue, Apr 21, 2009 at 15:03, P.J. Eby <pje at telecommunity.com> wrote:
> >  python2 setup.py 2to3 test
>
>Well, yes, but it should be
>
>    python 3 setup.py 2to3 test
>
>Otherwise it can't reasonably have any idea of which python to use.

Why not?  The 2to3 command could simply take an option for the 
python3 executable, and be set from the standard config files (e.g. setup.cfg).


>And when you run it with python3, the 2to3 command isn't needed, as
>the 2to3 step can be automatically inserted through version detection.

Um...  I'm still not following you.  You sound like you're describing 
something very different from what I outlined.


>In fact, this is exactly what I'm doing for zope.interface, as
>explained. For zope.interface, you run
>    python2.5 setup.py test
>and it will test it with python2.5 You run
>    python3.0 setup.py test
>and it will do exactly what you say: copy the files to build, run
>2to3, and then run the test on that in-place.
>    python3.0 setup.py install
>will copy the files to build, run 2to3 and then install it.

Right -- but this is totally *not* what I'm saying.  Because the 
strategy that I outlined doesn't require setuptools to be ported to 
python 3 first.


>Yes. That's is exactly what I'm trying to achieve. And I aim to extend
>setuptools with build_2to3 and test_2to3 commands to do exactly that.
>But to do that I need a good setuptools distribution that supports
>Python3.

No, you don't.  You only need one command, implemented as a plugin 
for the Python2 version of setuptools, and you drive everything from python2.

Once you get to the point where setuptools is ported enough to be 
able to have the Python3 version actually run its tests without 
crashing, you can go self-hosted.  But before that point, you need 
Python2 as the bootstrap/host environment for the build process.


>It explains why the test cases
>could run even though setuptools still couldn't install itself. I
>thought it was badly tested, now I know it's by design.

No, it's also badly tested; that's kind of inherited from the 
distutils.  And by the time I added the 'sandbox' virtualization 
tool, there was already tons of stuff that wasn't tested, such that 
it didn't seem worth it to do something else.


From zooko at zooko.com  Wed Apr 22 01:20:21 2009
From: zooko at zooko.com (Zooko O'Whielacronx)
Date: Tue, 21 Apr 2009 16:20:21 -0700
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
Message-ID: <948AB8BE-DCBA-4442-9C14-0A0A2CCD80AD@zooko.com>

Ugh.  Sorry -- I was on a plane when I composed this and I  
accidentally sent it the next time I connected to the Net before  
adding the references.  Here they are:

On Apr 20, 2009, at 18:06 PM, Zooko O'Whielacronx wrote:

> Roughly in order of most important to least important:
>
> 1.  a formal, machine-readable way to specify our dependencies; see  
> XYZ [2]

[2] http://allmydata.org/trac/tahoe/browser/_auto_deps.py

> 2.  automatic installation of dependencies;  This reduces the  
> "dependencies" section in the instructions for installing Tahoe [1]  
> from having sixteen separate software packages with separate  
> versioning requirements to having one: Python 2.4.2 through 2.6.

[1] http://allmydata.org/source/tahoe/trunk/docs/install.html

> 3.  Obviously having even one "dependency" in your installation  
> instructions is too many, so we try to provide .deb's, .rpm's,  
> gentoo ebuilds, Windows installers (e.g. as produced by py2exe or  
> bbfreeze), Macintosh applications, etc.  Many of the tools that  
> produce these other formats rely on the above-mentioned feature #1  
> and/or feature #2.
>
> 4.  Some people like to use "easy_install $PACKAGE".  In fact,  
> quite a lot of people -- if I recall the results of Tarek's  
> packaging survey correctly, it was more popular among the  
> responding Python programmers than any other method of installing  
> packages [3].

[3] http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first- 
results

> 5.  Using plugins so that you can add features to your build  
> system, such as revision control integration, unit testing,  
> versioning, pyflakes, uploading, etc. which you re-use as black box  
> modules maintained by someone else instead of copying code into  
> your setup.py or your GNUmakefile. [4, 5, 6, 7, 8].

[4] http://pypi.python.org/pypi/setuptools_darcs
[5] http://pypi.python.org/pypi/setuptools_bzr
[6] http://pypi.python.org/pypi/setuptools_hg
[7] http://pypi.python.org/pypi/setuptools_pyflakes
[8] http://pypi.python.org/pypi/setuptools_trial
[9] http://pypi.python.org/pypi/collective.releaser
[10] http://pypi.python.org/pypi/django-reusableapps
[11] http://pypi.python.org/pypi/eggchecker
[12] http://pypi.python.org/pypi/iw.releaser
[13] http://pypi.python.org/pypi/peggy
[14] http://pypi.python.org/pypi/CodeTools
[15] http://pypi.python.org/pypi/collective.dist
[16] http://pypi.python.org/pypi/disthelper
[17] http://pypi.python.org/pypi/bbfreeze
[18] http://pypi.python.org/pypi/buildutils
[19] http://pypi.python.org/pypi/eggtestinfo
[20] http://pypi.python.org/pypi/pip
[21] http://pypi.python.org/pypi/pydeploy32
[22] http://pypi.python.org/pypi/py2app
[23] http://pypi.python.org/pypi/pyinstall
[24] http://pypi.python.org/pypi/pyrun
[25] http://pypi.python.org/pypi/PythonEggTools
[26] http://pypi.python.org/pypi/setuphelper
[27] http://pypi.python.org/pypi/virtualenv
[28] http://pypi.python.org/pypi/yolk
[29] http://pypi.python.org/pypi/yolk-portage
[30] http://pypi.python.org/pypi/zc.buildout
[31] http://pypi.python.org/pypi/Enstaller

I should emphasize that this list is by no means exhaustive -- I just  
scanned through the results of searching pypi for the word  
"setuptools" and pulled out a few that seemed to be vaguely relevant.

I'm think that there might a danger of us distutils-sig folks  
underestimating the value offered by compatibility with setuptools.   
Most of the authors of these packages are probably not reading  
distutils-sig, and so most of them are probably not answering the  
questions like "Why do you use setuptools?".  (For one thing, this  
mailing list is just insanely high volume -- it is hard to make time  
to keep up.)  Many of the authors of these packages and the other  
packages like these are not going to go to much, if any, effort to  
change their tools to support the new Distutils.  To the degree to  
which new authors of new projects which use the new Distutils can  
get, for free, compatibility with projects like these ones that  
already use setuptools, then everyone can benefit.

Regards,

Zooko


> 6.  Using the console scripts feature of setuptools to make it so  
> your scripts that you wrote for Unix also work on Windows.
>
> 7.  python setup.py sdist upload register
>
> 8.  revision control integration; I want all files under revision  
> control to be included in packages.

From ben+python at benfinney.id.au  Wed Apr 22 02:20:33 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 22 Apr 2009 10:20:33 +1000
Subject: [Distutils] Questionnaire: Why do you use setuptools?
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
Message-ID: <87ab69ilny.fsf@benfinney.id.au>

Zooko O'Whielacronx <zooko at zooko.com> writes:

> Roughly in order of most important to least important:

My response is essentially a subset of Zooko's.

> 1.  a formal, machine-readable way to specify our dependencies
[?]

> 2.  automatic installation of dependencies
[?]

> 3. Obviously having even one "dependency" in your installation
> instructions is too many, so we try to provide [system packages] [?]
> Many of the tools that produce these other formats rely on the
> above-mentioned feature #1 and/or feature #2.
[?]

> 7.  python setup.py sdist upload register

All of the above.

-- 
 \        ?Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.? ?Richard |
_o__)                                                         Stallman |
Ben Finney


From david.lyon at preisshare.net  Wed Apr 22 02:49:38 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 20:49:38 -0400
Subject: [Distutils] =?utf-8?q?Location_of_Packages=2E=2E=2E_The_Directory?=
 =?utf-8?q?_location=2C_plus_Documentation_and_Examples_within_a_system_?=
 =?utf-8?q?=3F?=
In-Reply-To: <20090421130118.911BA3A40AC@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420161437.9FD583A4070@sparrow.telecommunity.com>
	<319e029f0904200930x53740822v82e34a4386b1ab8e@mail.gmail.com>
	<20090420170005.C3CB63A4070@sparrow.telecommunity.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
Message-ID: <2f1bd9c57890c8ba2af1ed20eedcc546@preisshare.net>


Hi All,

Once a package is installed on a system... is there any way to find out
where it was installed to?

I am looking for both metadata and a directory name/path.

In my Package Manager GUI, I want to be able to "reveal" all the 
hidden "goodies" that seem to dissapear into the subsystem.

Either "Documentation" or "Demos"

Here is an example....

C:\Python25\Lib\site-packages\win32\Demos

or

C:\Python25\Lib\site-packages\win32com\HTML

Under Linux, it would be just as valuable. If not more.

Regards

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot-XP.PNG
Type: image/png
Size: 19442 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090421/a539a528/attachment-0001.png>

From david.lyon at preisshare.net  Wed Apr 22 03:43:07 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 21:43:07 -0400
Subject: [Distutils] =?utf-8?q?What=27s_missing_from_easy=5Finstall?=
In-Reply-To: <gs35o4$m3a$1@ger.gmane.org>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
	<gs35o4$m3a$1@ger.gmane.org>
Message-ID: <04662699d4fcbcf6fdb13bbdc2915ca5@preisshare.net>


The uninstaller -m option, doesn't seem to want to
work for me.

I haven't so far been able to get it to uninstall
any packages.

David


From greg.ewing at canterbury.ac.nz  Wed Apr 22 01:57:02 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 22 Apr 2009 11:57:02 +1200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>
	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>
	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>
	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>
	<49EC8883.10106@ar.media.kyoto-u.ac.jp>
	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>
	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>
	<49ED5776.3020304@canterbury.ac.nz>
	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
Message-ID: <49EE5D4E.3030607@canterbury.ac.nz>

Tarek Ziad? wrote:
> Please Greg, add your use case here with David's one, at the end of the page,
> since we are trying to work on a solution.
> 
> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft

Is this really the best place for it? My use case
doesn't have anything to do with manifests.

We need a page like this concerning overall distutils
reorganisation.

-- 
Greg

From david at ar.media.kyoto-u.ac.jp  Wed Apr 22 03:31:37 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 22 Apr 2009 10:31:37 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <gskg9b$jb2$1@ger.gmane.org>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<319e029f0904200335y2d7d1889wb9c064abb8f6bbc8@mail.gmail.com>	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	<49EC8883.10106@ar.media.kyoto-u.ac.jp>	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>	<a71153f0904202031n24924440x1f822faf9e9c6837@mail.gmail.com>
	<gskg9b$jb2$1@ger.gmane.org>
Message-ID: <49EE7379.9020702@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:


> I don't know where you get the idea that setuptools / distutils is
> primarily about "building" software.

The main documentation of distutils says "building and installing python
modules". A large fraction of distutils code is related to building.

David

From david at ar.media.kyoto-u.ac.jp  Wed Apr 22 03:38:26 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 22 Apr 2009 10:38:26 +0900
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <49EE5D4E.3030607@canterbury.ac.nz>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>	<79990c6b0904200521y48c9973w7b15f714004b8e10@mail.gmail.com>	<49EC774D.9030808@ar.media.kyoto-u.ac.jp>	<94bdd2610904200650h51c6b80eyb7a575f138a0456e@mail.gmail.com>	<49EC8007.6090103@ar.media.kyoto-u.ac.jp>	<94bdd2610904200726l5e5783bam1146a4d0891e64bd@mail.gmail.com>	<49EC8883.10106@ar.media.kyoto-u.ac.jp>	<319e029f0904201047g58b263en5d79f0766b1cd202@mail.gmail.com>	<319e029f0904201316n63bef828pbe59c8c4a08c7b52@mail.gmail.com>	<49ED5776.3020304@canterbury.ac.nz>	<94bdd2610904202344k4fe6c1e7h6ff1853af3102d72@mail.gmail.com>
	<49EE5D4E.3030607@canterbury.ac.nz>
Message-ID: <49EE7512.4060102@ar.media.kyoto-u.ac.jp>

Greg Ewing wrote:
> Tarek Ziad? wrote:
>> Please Greg, add your use case here with David's one, at the end of
>> the page,
>> since we are trying to work on a solution.
>>
>> http://wiki.python.org/moin/Distutils/ManifestPluginSystem/Draft
>
> Is this really the best place for it? My use case
> doesn't have anything to do with manifests.

Mine neither. I agree the title is confusing, but OTOH, moving things in
wiki is easy, whereas waiting for things to be at the right place
increases the risk of losing the information. That's how wiki work.

We need more people who care about the building issues (vs the
distribution issues) if we want to be taken in account for distutils
improvements :)

David

From david at ar.media.kyoto-u.ac.jp  Wed Apr 22 03:54:30 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 22 Apr 2009 10:54:30 +0900
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <948AB8BE-DCBA-4442-9C14-0A0A2CCD80AD@zooko.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>	<FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
	<948AB8BE-DCBA-4442-9C14-0A0A2CCD80AD@zooko.com>
Message-ID: <49EE78D6.6070307@ar.media.kyoto-u.ac.jp>

Zooko O'Whielacronx wrote:
>
> Many of the authors of these packages and the other packages like
> these are not going to go to much, if any, effort to change their
> tools to support the new Distutils.

agreed on this.

> To the degree to which new authors of new projects which use the new
> Distutils can get, for free, compatibility with projects like these
> ones that already use setuptools, then everyone can benefit.

The current distutils needs to be improved, this at least seems to be
consensual. From the Pycon summary (I was not there, just using the wiki
and Tarek emails as a reference), it seems that some things from
setuptools will be incorporated in concept if not in code (install
requirements, better extensibility, etc...). The technical details are
not all set, but they will be at some point.

The problem with a new "Distutils2" 100 % compatible with the current
one is that it is impossible to do without recreating all the things
which it does wrong, because distutils has some serious design issues
which cannot be solved without breaking things, and some packages depend
on distutils internals quite a bit - thing about all the ones which are
broken with setuptools for example.

Doing something which is incompatible and requires new scripts from
scratch is not an option either. Something which could convert simple
setup.py files to the new distutils could be more reasonable (a bit like
2to3), and I think it is doable for a significant proportion of current
packages, through an extended set of static metadata. Such a proposal is
being worked on (http://wiki.python.org/moin/Distutils/StaticMetadata);
I think the suggestion of making those metadata independent of distutils
design (being more high level and independent of distutils commands)
should be given serious thought. It can be done gradually, too.

David


From pje at telecommunity.com  Wed Apr 22 05:02:54 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 21 Apr 2009 23:02:54 -0400
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <04662699d4fcbcf6fdb13bbdc2915ca5@preisshare.net>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
	<gs35o4$m3a$1@ger.gmane.org>
	<04662699d4fcbcf6fdb13bbdc2915ca5@preisshare.net>
Message-ID: <20090422030023.28B933A40AC@sparrow.telecommunity.com>

At 09:43 PM 4/21/2009 -0400, David Lyon wrote:

>The uninstaller -m option, doesn't seem to want to
>work for me.
>
>I haven't so far been able to get it to uninstall
>any packages.

That's not an uninstall option; it simply ensures that the package is 
no longer listed in easy-install.pth file.  You must manually remove 
the .egg file or directory at present.


From david.lyon at preisshare.net  Wed Apr 22 05:50:14 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 21 Apr 2009 23:50:14 -0400
Subject: [Distutils] =?utf-8?q?What=27s_missing_from_easy=5Finstall?=
In-Reply-To: <20090422030023.28B933A40AC@sparrow.telecommunity.com>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
	<gs35o4$m3a$1@ger.gmane.org>
	<04662699d4fcbcf6fdb13bbdc2915ca5@preisshare.net>
	<20090422030023.28B933A40AC@sparrow.telecommunity.com>
Message-ID: <38a3c50257efddcecb8b455ce8777ca2@preisshare.net>


On Tue, 21 Apr 2009 23:02:54 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> At 09:43 PM 4/21/2009 -0400, David Lyon wrote:
> 
>>The uninstaller -m option, doesn't seem to want to
>>work for me.
>>
>>I haven't so far been able to get it to uninstall
>>any packages.
> 
> That's not an uninstall option; it simply ensures that the package is
> no longer listed in easy-install.pth file.  You must manually remove
> the .egg file or directory at present.

Oh ok...

That doesn't seem so hard....

I will try to implement that...

Thank you for your assistance..

David


From pje at telecommunity.com  Wed Apr 22 06:01:48 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 22 Apr 2009 00:01:48 -0400
Subject: [Distutils] What's missing from easy_install
In-Reply-To: <38a3c50257efddcecb8b455ce8777ca2@preisshare.net>
References: <grfism$7j5$1@ger.gmane.org> <grfq4a$2df$1@ger.gmane.org>
	<364299f40904122045k2ecd7970vb3451daf6ba49b67@mail.gmail.com>
	<20090413045619.GA21812@ayn.mi.celestial.com>
	<49E2C580.5010704@ar.media.kyoto-u.ac.jp>
	<gs215l$q5m$1@ger.gmane.org>
	<011DAAA0-373B-43F4-BE73-9167DC2E5784@zooko.com>
	<gs35o4$m3a$1@ger.gmane.org>
	<04662699d4fcbcf6fdb13bbdc2915ca5@preisshare.net>
	<20090422030023.28B933A40AC@sparrow.telecommunity.com>
	<38a3c50257efddcecb8b455ce8777ca2@preisshare.net>
Message-ID: <20090422035916.464933A40AC@sparrow.telecommunity.com>

At 11:50 PM 4/21/2009 -0400, David Lyon wrote:

>On Tue, 21 Apr 2009 23:02:54 -0400, "P.J. Eby" <pje at telecommunity.com>
>wrote:
> > At 09:43 PM 4/21/2009 -0400, David Lyon wrote:
> >
> >>The uninstaller -m option, doesn't seem to want to
> >>work for me.
> >>
> >>I haven't so far been able to get it to uninstall
> >>any packages.
> >
> > That's not an uninstall option; it simply ensures that the package is
> > no longer listed in easy-install.pth file.  You must manually remove
> > the .egg file or directory at present.
>
>Oh ok...
>
>That doesn't seem so hard....
>
>I will try to implement that...

If you're implementing an uninstall using easy_install, you probably 
also want to use -N and -x, to prevent dependencies and scripts from 
being reinstalled.  You may also want to check what scripts a package 
provides, and uninstall those as well.


From ziade.tarek at gmail.com  Wed Apr 22 10:42:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 22 Apr 2009 10:42:29 +0200
Subject: [Distutils] Location of Packages... The Directory location,
	plus 	Documentation and Examples within a system ?
In-Reply-To: <2f1bd9c57890c8ba2af1ed20eedcc546@preisshare.net>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<2f1bd9c57890c8ba2af1ed20eedcc546@preisshare.net>
Message-ID: <94bdd2610904220142k6cfe54c9mafba5fc89a4e0389@mail.gmail.com>

On Wed, Apr 22, 2009 at 2:49 AM, David Lyon <david.lyon at preisshare.net> wrote:
>
> Hi All,
>
> Once a package is installed on a system... is there any way to find out
> where it was installed to?

Not at the moment. PEP 376 though, proposes a new API to be able to reach
the metadata of an installed package, browsing the directories from the PATH
(that's how setuptools does)

>
> I am looking for both metadata and a directory name/path.
>
> In my Package Manager GUI, I want to be able to "reveal" all the
> hidden "goodies" that seem to dissapear into the subsystem.
>
> Either "Documentation" or "Demos"
>
> Here is an example....
>
> C:\Python25\Lib\site-packages\win32\Demos
>
> or
>
> C:\Python25\Lib\site-packages\win32com\HTML
>
> Under Linux, it would be just as valuable. If not more.

There's no explicit way at distutils level to point things like documentation,
maybe that could be a metadata.

Maybe you can propose this, by commenting ths page :

http://wiki.python.org/moin/Distutils/Metadata

Regards
TArek


>
> Regards
>
> David
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Wed Apr 22 11:05:49 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 22 Apr 2009 05:05:49 -0400
Subject: [Distutils]
 =?utf-8?q?Location_of_Packages=2E=2E=2E_The_Directory?=
 =?utf-8?q?_location=2C_plus_Documentation_and_Examples_within_a_system_?=
 =?utf-8?q?=3F?=
In-Reply-To: <94bdd2610904220142k6cfe54c9mafba5fc89a4e0389@mail.gmail.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<319e029f0904201013h2d495910q58b8bcd602781ab5@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<2f1bd9c57890c8ba2af1ed20eedcc546@preisshare.net>
	<94bdd2610904220142k6cfe54c9mafba5fc89a4e0389@mail.gmail.com>
Message-ID: <6725e87625134ab382383636840969d7@preisshare.net>



On Wed, 22 Apr 2009 10:42:29 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> There's no explicit way at distutils level to point things like
> documentation,

Well I thank you for the answer. At least I have an answer from an expert..

> maybe that could be a metadata.
> 
> Maybe you can propose this, by commenting ths page :
> 
> http://wiki.python.org/moin/Distutils/Metadata

I would like to help but I'm not an expert in that area.

A simple answer is that it would be good if the developer of a package
could:

 - Register their documentation file ie index.html etc...

 - Register their examples directory

 - Register program executeables - (an example - ez_install.exe)
   otherwise it's easy for them to get lost.

All other information seems to be available on Pypi in the DOAP record.

Regards David






From ziade.tarek at gmail.com  Wed Apr 22 11:12:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 22 Apr 2009 11:12:22 +0200
Subject: [Distutils] RFC : Version comparison
Message-ID: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>

Hi,

We worked during Pycon on version comparisons. Distutils has one but
it is a bit strict, setuptools has another one, but it's a bit
heuristic.

We would like to propose the inclusion for Python 2.7 of a new version
comparison algorithm, based on the discussion
Fedora, Ubuntu and Python people had. The plan would be to deprecate
the current one (which is not really used anyway)
and provide, promote this one.

Trent Mick took the lead on this work at the end of Pycon, and worked
on a prototype.

It's explained here, and there's an implementation  (I've put it at
the top of the page for conveniency):

http://wiki.python.org/moin/Distutils/VersionComparison

Please comment,

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Wed Apr 22 11:20:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 22 Apr 2009 11:20:20 +0200
Subject: [Distutils] Distutils work, roadmap
In-Reply-To: <79990c6b0904210706q5f076ecbuaf7aa5685be5c15c@mail.gmail.com>
References: <94bdd2610904210034l308108dds3f0f89083f40f86c@mail.gmail.com>
	<6da942828b47e111fd9de4c00cd20791@preisshare.net>
	<94bdd2610904210221l17f16e00tc5f20051d92c425b@mail.gmail.com>
	<79990c6b0904210337q44808e23m26c20970794ae3ab@mail.gmail.com>
	<94bdd2610904210608l63079430i5e16e300930c2200@mail.gmail.com>
	<79990c6b0904210706q5f076ecbuaf7aa5685be5c15c@mail.gmail.com>
Message-ID: <94bdd2610904220220w1ba2833do88a0d26c3e118cbc@mail.gmail.com>

On Tue, Apr 21, 2009 at 4:06 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> One concrete point - The categories on the wiki don't really give me
> much help in knowing where to put my use case. I'm a user of Python
> packages - all I care about is running the installer, or if I must
> typing python setup.py bdist_wininst. So these aren't relevant, as
> they are developer-oriented:
>
> ? ?DistutilsVersionFight
> ? ?Distutils/StandardizeEggInfo
> ? ?Distutils/Metadata
> ? ?Distutils/StaticMetadata
> ? ?Distutils/ManifestPluginSystem
> ? ?Distutils/TestingInfrastructure

Right, I have started to work on its reorganization

> FWIW, I did post my use case here some days ago, with a request that
> someone suggest/find a home for it on the wiki, as I wasn't sure where
> it went. To my knowledge, no-one ever did that. I'd still appreciate
> it if someone could copy it to the wiki. But trying to understand the
> various threads and arguments on the list has sapped most of my
> energy, so I'm not going to try to guess where it should go.

I'll push it into the wiki, thanks

Tarek


-- 
Tarek Ziad? | http://ziade.org

From zooko at zooko.com  Wed Apr 22 14:38:30 2009
From: zooko at zooko.com (Zooko O'Whielacronx)
Date: Wed, 22 Apr 2009 05:38:30 -0700
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <49EE78D6.6070307@ar.media.kyoto-u.ac.jp>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>	<FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
	<948AB8BE-DCBA-4442-9C14-0A0A2CCD80AD@zooko.com>
	<49EE78D6.6070307@ar.media.kyoto-u.ac.jp>
Message-ID: <E8D5597E-2121-44D2-9F2E-21C5AFEE7C8F@zooko.com>

On Apr 21, 2009, at 18:54 PM, David Cournapeau wrote:

> The problem with a new "Distutils2" 100 % compatible with the current
> one is that it is impossible to do without recreating all the things
> which it does wrong,

I'm sorry, I was unclear.  I do not think that it is super important  
for a programmer who currently uses setuptools to package their  
software to be able to switch to distutils2 to package their software.

I do think that it is super important for a programmer who uses  
distutils2 to be able to easily re-use the software written by  
*other* people who use setuptools, and vice versa.  This means the  
sdist format should be interoperable, probably the bdist_egg format  
should be interoperable, and the "install_requires" metadata should  
be interoperable.

It would also be nice if a programmer who is choosing whether to use  
setuptools or distutils2 to package their own software could consider  
the library of development tools plugins as equally available for  
both, instead of thinking "Oh, I could use the new distutils2, which  
has a cleaner design and is in Python 2.7, but if I use setuptools  
then I can continue to use all these development plugins that already  
exist".  So if distutils2 can re-use extant setuptools plugins, then  
that would be handy.

Regards,

Zooko

From david at ar.media.kyoto-u.ac.jp  Wed Apr 22 14:43:26 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 22 Apr 2009 21:43:26 +0900
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <E8D5597E-2121-44D2-9F2E-21C5AFEE7C8F@zooko.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>	<FB2198AD-21CF-4FAD-9B84-EDBBE8851502@zooko.com>
	<948AB8BE-DCBA-4442-9C14-0A0A2CCD80AD@zooko.com>
	<49EE78D6.6070307@ar.media.kyoto-u.ac.jp>
	<E8D5597E-2121-44D2-9F2E-21C5AFEE7C8F@zooko.com>
Message-ID: <49EF10EE.3060109@ar.media.kyoto-u.ac.jp>

Zooko O'Whielacronx wrote:
> On Apr 21, 2009, at 18:54 PM, David Cournapeau wrote:
>
>> The problem with a new "Distutils2" 100 % compatible with the current
>> one is that it is impossible to do without recreating all the things
>> which it does wrong,
>
> I'm sorry, I was unclear.  I do not think that it is super important
> for a programmer who currently uses setuptools to package their
> software to be able to switch to distutils2 to package their software.

Ah, ok, then I indeed partly misunderstood what you said.

> I do think that it is super important for a programmer who uses
> distutils2 to be able to easily re-use the software written by *other*
> people who use setuptools, and vice versa.  This means the sdist
> format should be interoperable, probably the bdist_egg format should
> be interoperable, and the "install_requires" metadata should be
> interoperable.

Oh, yes, definitely. I would not even imagine designing something where
eggs or sdist formats are not *exactly* the same, whether they are
produced by distutils, setuptools, distutils2 or whatever. This is why
the formats have to be specified, and the code to produce them separated
from the rest (e.g. there should be a library to produce an egg from the
metadata without using the rest of setuptools, same for sdist, etc...).
And this can be done for the current distutils as well anyway.

Ideally, the static metadata format should be the same, so that you
could really, practically, support both systems 'for free'. At least for
simple packages (by simple I mean packages without difficult C code, or
more generally specificities which require setup magic, etc...).

> It would also be nice if a programmer who is choosing whether to use
> setuptools or distutils2 to package their own software could consider
> the library of development tools plugins as equally available for
> both, instead of thinking "Oh, I could use the new distutils2, which
> has a cleaner design and is in Python 2.7, but if I use setuptools
> then I can continue to use all these development plugins that already
> exist".  So if distutils2 can re-use extant setuptools plugins, then
> that would be handy.

This seems more difficult to do (although I have never thought about
this problem to be honest, contrary to the other ones mentioned above -
I don't use setuptools that much myself). Many plugins work on the
assumption of commands - and if I were to design a new tool, commands
would be *the* thing I would like to get rid of (it does not play well
with the idea of handling dependencies, for once).

cheers,

David

From carlos.tejo at fundacionctic.org  Wed Apr 22 15:18:17 2009
From: carlos.tejo at fundacionctic.org (Carlos Tejo Alonso)
Date: Wed, 22 Apr 2009 15:18:17 +0200
Subject: [Distutils] [Catalog-sig]  Metadata-Version in PKG-INFO
References: <09700B613C4DD84FA9F2FEA5218828190595094A@ayalga.fundacionctic.org>
	<94bdd2610904160347n71122936wf118bf73f7f1f713@mail.gmail.com>
	<09700B613C4DD84FA9F2FEA52188281905950B7D@ayalga.fundacionctic.org>
	<94bdd2610904160734m936fd87r3bcceaa0c7977a0a@mail.gmail.com>
	<49E79E30.4000904@v.loewis.de>
	<94bdd2610904161449r221a3609t2a046ee844754e2e@mail.gmail.com>
Message-ID: <09700B613C4DD84FA9F2FEA52188281905AD79F0@ayalga.fundacionctic.org>

> Maybe we could ask someone at the FSF so we have the best versions in
> our Trove classifier.

There is a list [1] of OSI approved licenses. Maybe it could be useful for the trove classifier.

-------------------------------------
Carlos Tejo Alonso
Research & Development Department
CTIC Foundation [Asturias, Spain]
www.fundacionctic.org
------------------------------------- 

[1] http://www.opensource.org/licenses/category

From exarkun at divmod.com  Wed Apr 22 15:38:15 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 22 Apr 2009 09:38:15 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
Message-ID: <20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>

On Wed, 22 Apr 2009 11:12:22 +0200, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>Hi,
>
>We worked during Pycon on version comparisons. Distutils has one but
>it is a bit strict, setuptools has another one, but it's a bit
>heuristic.
>
>We would like to propose the inclusion for Python 2.7 of a new version
>comparison algorithm, based on the discussion
>Fedora, Ubuntu and Python people had. The plan would be to deprecate
>the current one (which is not really used anyway)
>and provide, promote this one.
>
>Trent Mick took the lead on this work at the end of Pycon, and worked
>on a prototype.
>
>It's explained here, and there's an implementation  (I've put it at
>the top of the page for conveniency):
>
>http://wiki.python.org/moin/Distutils/VersionComparison
>

Clearly being able to parse the string representation is important, since
this information will largely reside in non-Python text files.  I suppose
that many people will also be very comfortable with and happy about writing
RationalVersion(somestring) and ">=somestring".  However, it would also be
great if there were a way to construct a version object out of structured
data instead.  Can a constructor which takes each part of the version data
as a separate object be added?

Jean-Paul

From pje at telecommunity.com  Wed Apr 22 16:18:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 22 Apr 2009 10:18:41 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904212327p31c0dd0fm4cd9f1a29d3b03a8@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420185839.67ACB3A4070@sparrow.telecommunity.com>
	<319e029f0904201211t1d985b53j6bcc59edf3ed4f25@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.com>
	<20090421175500.B00393A40AC@sparrow.telecommunity.com>
	<319e029f0904212327p31c0dd0fm4cd9f1a29d3b03a8@mail.gmail.com>
Message-ID: <20090422141610.30BD73A4070@sparrow.telecommunity.com>

At 08:27 AM 4/22/2009 +0200, Lennart Regebro wrote:
>On Tue, Apr 21, 2009 at 19:57, P.J. Eby <pje at telecommunity.com> wrote:
> > At 04:06 PM 4/21/2009 +0200, Lennart Regebro wrote:
> >>
> >> On Tue, Apr 21, 2009 at 15:03, P.J. Eby <pje at telecommunity.com> wrote:
> >> >  python2 setup.py 2to3 test
> >>
> >> Well, yes, but it should be
> >>
> >>   python 3 setup.py 2to3 test
> >>
> >> Otherwise it can't reasonably have any idea of which python to use.
> >
> > Why not?  The 2to3 command could simply take an option for the python3
> > executable, and be set from the standard config files (e.g. setup.cfg).
>
>Because that would mean that Python 2 needs to be installed to use
>Python 3. It also means all programs that do any sort of installing
>need to either know the position of the python 3 executable to use
>when installing, and be run with Python 2, or they need to be run with
>Python 3 and know the position of a Python 2 interpreter to run
>setup.py with.

Er, no.  It only means that you need Python 2 to be installed *while 
porting a package* to Python 3.


From pje at telecommunity.com  Wed Apr 22 16:29:01 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 22 Apr 2009 10:29:01 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.co
 m>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
Message-ID: <20090422142628.C05483A4070@sparrow.telecommunity.com>

At 11:12 AM 4/22/2009 +0200, Tarek Ziad? wrote:
>Hi,
>
>We worked during Pycon on version comparisons. Distutils has one but
>it is a bit strict, setuptools has another one, but it's a bit
>heuristic.
>
>We would like to propose the inclusion for Python 2.7 of a new version
>comparison algorithm, based on the discussion
>Fedora, Ubuntu and Python people had. The plan would be to deprecate
>the current one (which is not really used anyway)
>and provide, promote this one.
>
>Trent Mick took the lead on this work at the end of Pycon, and worked
>on a prototype.
>
>It's explained here, and there's an implementation  (I've put it at
>the top of the page for conveniency):
>
>http://wiki.python.org/moin/Distutils/VersionComparison
>
>Please comment,

I don't see how it can manage, e.g. a development version of a 
postrelease, with an SVN rev or date stamp on it.  Such versions 
might not be found on PyPI or on RPMs, but would be needed in development.

(Btw, the wiki page pseudo-regex doesn't match what the code actually 
parses, either.)


From regebro at gmail.com  Wed Apr 22 16:52:14 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 22 Apr 2009 16:52:14 +0200
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <20090422141610.30BD73A4070@sparrow.telecommunity.com>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.com>
	<20090421175500.B00393A40AC@sparrow.telecommunity.com>
	<319e029f0904212327p31c0dd0fm4cd9f1a29d3b03a8@mail.gmail.com>
	<20090422141610.30BD73A4070@sparrow.telecommunity.com>
Message-ID: <319e029f0904220752w192af274te256c2070a940793@mail.gmail.com>

On Wed, Apr 22, 2009 at 16:18, P.J. Eby <pje at telecommunity.com> wrote:
> Er, no. ?It only means that you need Python 2 to be installed *while porting
> a package* to Python 3.

No. It means it needs to be installed when installing the package from
a source distribution. Which is the normal way of distributing
modules.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From pje at telecommunity.com  Wed Apr 22 18:04:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 22 Apr 2009 12:04:51 -0400
Subject: [Distutils] The problem with Setuptools on Python 3.
In-Reply-To: <319e029f0904220752w192af274te256c2070a940793@mail.gmail.co
 m>
References: <319e029f0904200119h60b03d53q4fb4bbb7a641a407@mail.gmail.com>
	<20090420194947.377203A4070@sparrow.telecommunity.com>
	<319e029f0904201312p11661231pdf3592e4f65b4370@mail.gmail.com>
	<94174C97-4FAD-4696-9CB2-2DC9770C29E2@mac.com>
	<319e029f0904210526nabfb422p4a441fb1256cae9b@mail.gmail.com>
	<20090421130118.911BA3A40AC@sparrow.telecommunity.com>
	<319e029f0904210706t437211fewe70e53c0474cbbf7@mail.gmail.com>
	<20090421175500.B00393A40AC@sparrow.telecommunity.com>
	<319e029f0904212327p31c0dd0fm4cd9f1a29d3b03a8@mail.gmail.com>
	<20090422141610.30BD73A4070@sparrow.telecommunity.com>
	<319e029f0904220752w192af274te256c2070a940793@mail.gmail.com>
Message-ID: <20090422160220.22D9E3A40AC@sparrow.telecommunity.com>

At 04:52 PM 4/22/2009 +0200, Lennart Regebro wrote:
>On Wed, Apr 22, 2009 at 16:18, P.J. Eby <pje at telecommunity.com> wrote:
> > Er, no.  It only means that you need Python 2 to be installed 
> *while porting
> > a package* to Python 3.
>
>No. It means it needs to be installed when installing the package from
>a source distribution. Which is the normal way of distributing
>modules.

I don't understand you.  Here is what I understand:

1. Setuptools requires setuptools
2. Setuptools doesn't run on Python 3 (yet)
3. There needs to be a way to build a Py3 version of setuptools in 
order to fix #2

Therefore, adding a new setuptools comand to do #3, that runs under 
either Py2 or Py3, fixes #1 in the context of #2.

However, once setuptools *does* run on Python 3, then there is no 
longer a need for the build process to run exclusively under...

Aha!  Now I (finally) get what you're talking about!  In order for 
this to work, there'd have to be a separate Py3 source distro for 
setuptools, or else setup.py would need to have a (non-setuptools 
depending) way to build its own Python3 version.

Okay, now that I actually understand the problem, I will give it some 
more thought.  I see now that what I was proposing works only for the 
porting process and for non-self-dependent packages, but not for 
distribution of self-dependent packages like setuptools.  Either the 
sdist would need to ship with a Python3 version already included (or 
have a distinct Py3 sdist), or there'd need to be a 
non-setuptools-dependent bootstrap process.

I'll have to think about this one a bit more.


From setuptools at bugs.python.org  Thu Apr 23 05:03:02 2009
From: setuptools at bugs.python.org (Chad)
Date: Thu, 23 Apr 2009 03:03:02 +0000
Subject: [Distutils] [issue68] install local package with extras with
	easy_install
In-Reply-To: <1240455782.86.0.828196896528.issue68@psf.upfronthosting.co.za>
Message-ID: <1240455782.86.0.828196896528.issue68@psf.upfronthosting.co.za>


New submission from Chad <chadrik at gmail.com>:

1) easy_install can be used to install "extra" dependencies, using the following
syntax (undocumented on the PEAK website, as far as i can tell):

$ easy_install mypackage[extraFeature]

where "extraFeature" is a feature specified by the package's extras_require
keyword in setup.py.  

2) easy_install can be used to install a package in the current directory (as of
0.5a9) :

$ easy_install .  

however, these two features can't be combined: 

$ easy_install .[extraFeature]
running easy_install
Searching for .[extraFeature]
Reading http://pypi.python.org/simple/./
No local packages or download links found for .[extraFeature]
error: Could not find suitable distribution for Requirement.parse('.[extraFeature]')

I feel that the above is a bug. '.' should be treated as a path not a package
name, since '.' is not a valid package name.  i haven't looked at the code yet,
but it seems that a regular expression should be able to determine between a
valid package name, an http address, and a path on disk.  it seems like '.' was
added as a special case, because other relative paths don't seem to be supported:

$ cd /path/to/mypackage
$ easy_install ../mypackage[extraFeature] 
error: Not a URL, existing file, or requirement spec: '../mypackage[extraFeature] '


I'd be glad to submit a patch, if you just point me to the proper repository to
checkout and the proper procedures for getting it integrated into the main branch.

-chad

----------
messages: 279
nosy: chadrik
priority: bug
status: unread
title: install local package with extras with easy_install

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue68>
_______________________________________________

From pje at telecommunity.com  Thu Apr 23 07:10:43 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 23 Apr 2009 01:10:43 -0400
Subject: [Distutils] [issue68] install local package with extras with
 easy_install
In-Reply-To: <1240455782.86.0.828196896528.issue68@psf.upfronthosting.co .za>
References: <1240455782.86.0.828196896528.issue68@psf.upfronthosting.co.za>
	<1240455782.86.0.828196896528.issue68@psf.upfronthosting.co.za>
Message-ID: <20090423050811.97BAD3A40AC@sparrow.telecommunity.com>

At 03:03 AM 4/23/2009 +0000, Chad wrote:
>I feel that the above is a bug. '.' should be treated as a path not a package
>name, since '.' is not a valid package name.

Actually, it is.


>   i haven't looked at the code yet,
>but it seems that a regular expression should be able to determine between a
>valid package name, an http address, and a path on disk.  it seems 
>like '.' was
>added as a special case, because other relative paths don't seem to 
>be supported:
>
>$ cd /path/to/mypackage
>$ easy_install ../mypackage[extraFeature]
>error: Not a URL, existing file, or requirement spec: 
>'../mypackage[extraFeature] '

As it says in the error message, arguments are checked for a URL, 
existing file, or a requirement spec -- and in that order.  URL-ness 
is checked by regex.  Filename-ness is checked by checking for the 
existence of the file or directory, not by parsing.  If the file 
doesn't exist, an attempt is made to parse it as a requirement 
spec.  '.[foo]' is neither a URL nor an existing file, but it *is* a 
valid requirement spec, for a package named '.'.

I don't know of any way to change any of this without changing the 
command-line API.  The recommended workaround for building a local 
package with extras is:

     easy_install . packagename[extras]

as this will first install the local package, then install the 
specified extras.


From ziade.tarek at gmail.com  Thu Apr 23 11:11:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 23 Apr 2009 11:11:30 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>
Message-ID: <94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>

On Wed, Apr 22, 2009 at 3:38 PM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> Clearly being able to parse the string representation is important, since
> this information will largely reside in non-Python text files. ?I suppose
> that many people will also be very comfortable with and happy about writing
> RationalVersion(somestring) and ">=somestring". ?However, it would also be
> great if there were a way to construct a version object out of structured
> data instead. ?Can a constructor which takes each part of the version data
> as a separate object be added?

Sounds good, I'd be in favor of making RationalVersion using explicit
version bits,
and having some kind of function:

str2version(somestring) -> RationalVersion instance

I have pushed the prototype in a bitbucket project so everyone can
work it out (just join the project)

http://bitbucket.org/tarek/distutilsversion/

I'll wait for Trent feedback to make that change,

Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Apr 23 11:19:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 23 Apr 2009 11:19:22 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090422142628.C05483A4070@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
Message-ID: <94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>

2009/4/22 P.J. Eby <pje at telecommunity.com>:
> I don't see how it can manage, e.g. a development version of a postrelease,
> with an SVN rev or date stamp on it. ?Such versions might not be found on
> PyPI or on RPMs, but would be needed in development.

So, instead of having 'dev' and 'post', we would require a third case
for "pos+dev" version

so  (dev|post)N+ could become, ((dev|post)N+)|(postN+devN+)

example:

1.0.dev459 < 1.0 < 1.0.post456dev463 < 1.0.post456 < 1.0.post489


>
> (Btw, the wiki page pseudo-regex doesn't match what the code actually
> parses, either.)
>

i'll check it thx

>



-- 
Tarek Ziad? | http://ziade.org

From rupert.thurner at gmail.com  Sun Apr 26 21:39:53 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Sun, 26 Apr 2009 12:39:53 -0700 (PDT)
Subject: [Distutils] how to set cc correctly?
Message-ID: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>

how could one set the compiler correctly?

http://opencsw has a setuptools package, which does not use the cc in
the path, but in /opt/studio/SOS11/SUNWspro/bin/cc

so the following happens:

# CC=/opt/SUNWspro/bin/cc easy_install genshi
Searching for genshi
Reading http://pypi.python.org/simple/genshi/
Reading http://genshi.edgewall.org/
Reading http://genshi.edgewall.org/wiki/Download
Best match: Genshi 0.5.1
Downloading http://ftp.edgewall.com/pub/genshi/Genshi-0.5.1.zip
Processing Genshi-0.5.1.zip
Running Genshi-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/
easy_install-7yiT65/Genshi-0.5.1/egg-dist-tmp-Cp42JQ
warning: no previously-included files found matching 'doc/
2000ft.graffle'
warning: no previously-included files matching '*' found under
directory 'doc/logo.lineform'
unable to execute /opt/studio/SOS11/SUNWspro/bin/cc: No such file or
directory
**********************************************************************
WARNING:
An optional C extension could not be compiled, speedups will not be
available.
**********************************************************************
Adding Genshi 0.5.1 to easy-install.pth file

Installed /opt/csw/lib/python2.6/site-packages/Genshi-0.5.1-py2.6-
solaris-2.10-sun4v.egg
Processing dependencies for genshi
Finished processing dependencies for genshi

From yanegomi at gmail.com  Mon Apr 27 04:05:22 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Sun, 26 Apr 2009 19:05:22 -0700
Subject: [Distutils] Questionnaire: Why do you use setuptools?
In-Reply-To: <94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
Message-ID: <364299f40904261905k26e95e04l49356c9f050c8fe@mail.gmail.com>

Tarek,

On Sat, Apr 18, 2009 at 2:23 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sat, Apr 18, 2009 at 11:09 AM, Lennart Regebro <regebro at gmail.com> wrote:
>> Setuptools non-support for Python 3 is currently a serious hindrance
>> towards Python 3 aceptance. I'm trying to figure out what to do as a
>> next step in the Python 3 support for setuptools. And I have
>> encountered some obstacles. The first one is that setuptools requires
>> itself for installing and running tests. That makes it hard to install
>> it under Python 3. There are various solutions to this, but the next
>> obstacle I encounter in choosing the right solution is that the code
>> is hard to understand, and it makes me want to just rip it out and
>> start over, or in even more frustrated moments, avoid the problems by
>> not using setuptools at all. But the third obstacle for that is that I
>> don't actually know what features of setuptools people use.
>>
>> I personally use setuptools for these reasons:
>>
>> 1. When I create projects with paster, it uses setuptools.
>> 2. Setuptools makes it possible to specify requirements, which is then
>> used by buildout.
>> 3. Namespace packages require pkg_resources?
>> 4. The test command.
>>
>> What are the other major reasons people use setuptools?

I ideally would like to use setuptools like pkg_resources for
establishing the requires functionality to ensure that the versions of
the packages I'm dealing with are legitimate and tested.

Having a uniform versioning mechanism would greatly simplify bug
reporting and triage, not just within the python community, but also
in any communities outside of python that use / distribute python
utilities / modules.

> entry points (in the "how to extend commands" topic I might propose to
> include it/use it but it would need to be separated
> from pkg_resources)
>
>> Is there any good reason to not extract the namespace package support
>> into a separate package?
>
> It's on its way http://svn.python.org/projects/peps/trunk/pep-0382.txt
>
> and PEP 345 is being reworked to include requires
> http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt

I realize that PEP 345 isn't your PEP, but if you could relay this to
the authors, it would be much appreciated:

The section...

Supported-Platform (multiple use)

    Binary distributions containing a PKG-INFO file will use the
Supported-Platform field in their metadata to specify the OS and CPU
for which the binary package was compiled. The semantics of the
Supported-Platform field are not specified in this PEP.

    Example:

    Supported-Platform: RedHat 7.2
    Supported-Platform: i386-win32-2791

... isn't 100% complete in my mind. Will version checking of the OS be
properly supported so I can say:

    Supported-Platform: RedHat (>7.2)

... to say that Redhat 7.2+ is supported? I say this because certain
versions of FreeBSD have certain featuresets that were introduced in
later versions (most notably the ones that are basically fixing
pthread and rt support with multiprocessing), and specifying a version
range would be helpful. If it's not within the scope of the PEP at all
to address this point, this point should be referenced wherever it
exists in other documentation.

> Anyways, you can probably help if you worked on pkg_resources, by
> splitting all its components in pep-8-ied, readable elements
> in your branch

Thanks,
-Garrett

From ziade.tarek at gmail.com  Mon Apr 27 10:52:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 27 Apr 2009 10:52:12 +0200
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
Message-ID: <94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>

On Sun, Apr 26, 2009 at 9:39 PM, rupert.thurner
<rupert.thurner at gmail.com> wrote:
> how could one set the compiler correctly?
> http://opencsw has a setuptools package, which does not use the cc in
> the path, but in /opt/studio/SOS11/SUNWspro/bin/cc
>
> so the following happens:
>
> # CC=/opt/SUNWspro/bin/cc easy_install genshi
> Searching for genshi
> Reading http://pypi.python.org/simple/genshi/
> Reading http://genshi.edgewall.org/
> Reading http://genshi.edgewall.org/wiki/Download
> Best match: Genshi 0.5.1
> Downloading http://ftp.edgewall.com/pub/genshi/Genshi-0.5.1.zip
> Processing Genshi-0.5.1.zip
> Running Genshi-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/
> easy_install-7yiT65/Genshi-0.5.1/egg-dist-tmp-Cp42JQ
> warning: no previously-included files found matching 'doc/
> 2000ft.graffle'
> warning: no previously-included files matching '*' found under
> directory 'doc/logo.lineform'
> unable to execute /opt/studio/SOS11/SUNWspro/bin/cc: No such file or
> directory

I am not sur to understand what you are trying to achieve.

This indicates your environment variable CC was properly set and sent
to the compiler.

Are you sure "/opt/studio/SOS11/SUNWspro/bin/cc" exists and is executable ?

If cc is located somewhere else, try:

# CC=/its/here/cc easy_install genshi

If you don't provide any CC, it will use the one defined in the
Makefile of your Python
installation.

Tarek
-- 
Tarek Ziad? | http://ziade.org

From trentm at gmail.com  Mon Apr 27 18:17:27 2009
From: trentm at gmail.com (Trent Mick)
Date: Mon, 27 Apr 2009 09:17:27 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090422142628.C05483A4070@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
Message-ID: <6db0ea510904270917y6c83ef65hba3fea8f3324af18@mail.gmail.com>

> (Btw, the wiki page pseudo-regex doesn't match what the code actually parses, either.)

Current: N.N[.N]+[abc]N[.N]+[.(dev|post)N+]

Admittedly that is just trying to be illustrative without being too
"syntax-y", but perhaps it is more confusing than helpful. Ultimately
I think the docs might be most helpful by just giving examples of
valid version strings for different use cases. Something like:

Basic versions:
   1.0    # there must be at least two numbers (e.g. just "1" is not allowed)
   1.5.3.2   # any number of dotted segments more than two are allowed

Alphas, betas and release candidates (they must be numbered):
  1.0a1    # the first alpha on the way to "1.0"
  3.1b1    # the first beta
  5.0c3    # the third release candidate for "5.0"
Any number of additional dotted segments are allowed
  1.0a1.1    # a quick update to the "1.0a1" was necessary (personal
note: I'd tend to do a "1.0a2" release, but sometimes that might not
be feasible)

... and so on describing the use cases for "dev" (pre-) releases and
"post" releases.





-- 
Trent Mick
trentm at gmail.com

From trentm at gmail.com  Mon Apr 27 18:25:02 2009
From: trentm at gmail.com (Trent Mick)
Date: Mon, 27 Apr 2009 09:25:02 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090422142628.C05483A4070@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
Message-ID: <6db0ea510904270925q11575ea4re43e662bfbf40826@mail.gmail.com>

> I don't see how it can manage, e.g. a development version of a postrelease, with an SVN rev or date stamp on it. ?Such versions might not be found on PyPI or on RPMs, but would be needed in development.

I had thought the use cases for dev- and post- releases *was* to use
an SVN (or other SCC systems') rev or a date stamp, thus meaning there
would be no need for handling "a development version of a
post-release".

Perhaps I'm just not seeing it though, I haven't been involved with a
project that does post-releases. Can someone describe a concrete
example?


Trent

-- 
Trent Mick
trentm at gmail.com

From trentm at gmail.com  Mon Apr 27 18:36:24 2009
From: trentm at gmail.com (Trent Mick)
Date: Mon, 27 Apr 2009 09:36:24 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>
	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>
Message-ID: <6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>

>>?Can a constructor which takes each part of the version data
>> as a separate object be added?
>
> Sounds good, I'd be in favor of making RationalVersion using explicit
> version bits,
> and having some kind of function:
>
> str2version(somestring) -> RationalVersion instance

Or we could have:

   RationalVersion(...version bits...)
   RationalVersion.from_string(s)      # this is a @classmethod

> I have pushed the prototype in a bitbucket project so everyone can
> work it out (just join the project)
>
> http://bitbucket.org/tarek/distutilsversion/

Thanks for putting this up.

If we provide a way to construct a RationalVersion with version bits,
then we need to discuss what those "version bits" look like. Currently
the internal tuple data structure (`self.info`) looks like this (from
verlib.py's description of the 'f' marker used to help with sort
ordering, http://bitbucket.org/tarek/distutilsversion/src/cc93f5e1df3f/verlib.py#cl-163):

    # A marker used in the second and third parts of the `info` tuple, for
    # versions that don't have those segments, to sort properly. A example
    # of versions in sort order ('highest' last):
    #   1.0b1           ((1,0), ('b',1), ('f',))
    #   1.0.dev345      ((1,0), ('f',),  ('dev', 345))
    #   1.0             ((1,0), ('f',),  ('f',))
    #   1.0.post345     ((1,0), ('f',),  ('post', 345))
    #                           ^        ^
    #   'f' < 'b' -------------/         |
    #                                    |
    #   'dev' < 'f' < 'post' -----------/
    # Other letters would do, bug 'f' for 'final' is kind of nice.

Is this what we would want the constructor to take?

  RationalVersion( (1,0), ('b', 1) )     # 1.0b1
  RationalVersion( (1,0), ('f',), ('post', 345) )   # 1.0.post345

It seems a little too low-level. We could allow something a little nicer:

  RationalVersion( (1, 0, 'b', 1) )   # 1.0b1
  RationalVersion( (1, 0, 'post', 345) ) # 1.0.post345

If we did this, then we'd probably want these attributes on RationalVersion:

>>> v = RationalVersion((1,0,'b',1))
>>> str(v)
'1.0b1'
>>> v.info
(1, 0, 'b', 1)
>>> v._cmp_info   # use for comparison
((1, 0), ('b', 1), ('f',))


Thoughts?


Trent

-- 
Trent Mick
trentm at gmail.com

From exarkun at divmod.com  Mon Apr 27 18:55:50 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 27 Apr 2009 12:55:50 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
Message-ID: <20090427165550.24697.295875833.divmod.quotient.12452@henry.divmod.com>

On Mon, 27 Apr 2009 09:36:24 -0700, Trent Mick <trentm at gmail.com> wrote:
>>>?Can a constructor which takes each part of the version data
>>> as a separate object be added?
>>
>> Sounds good, I'd be in favor of making RationalVersion using explicit
>> version bits,
>> and having some kind of function:
>>
>> str2version(somestring) -> RationalVersion instance
>
>Or we could have:
>
>   RationalVersion(...version bits...)
>   RationalVersion.from_string(s)      # this is a @classmethod

I like the sound of that.

>
>> I have pushed the prototype in a bitbucket project so everyone can
>> work it out (just join the project)
>>
>> http://bitbucket.org/tarek/distutilsversion/
>
>Thanks for putting this up.
>
>If we provide a way to construct a RationalVersion with version bits,
>then we need to discuss what those "version bits" look like. Currently
>the internal tuple data structure (`self.info`) looks like this (from
>verlib.py's description of the 'f' marker used to help with sort
>ordering, http://bitbucket.org/tarek/distutilsversion/src/cc93f5e1df3f/verlib.py#cl-163):
>
>    # A marker used in the second and third parts of the `info` tuple, for
>    # versions that don't have those segments, to sort properly. A example
>    # of versions in sort order ('highest' last):
>    #   1.0b1           ((1,0), ('b',1), ('f',))
>    #   1.0.dev345      ((1,0), ('f',),  ('dev', 345))
>    #   1.0             ((1,0), ('f',),  ('f',))
>    #   1.0.post345     ((1,0), ('f',),  ('post', 345))
>    #                           ^        ^
>    #   'f' < 'b' -------------/         |
>    #                                    |
>    #   'dev' < 'f' < 'post' -----------/
>    # Other letters would do, bug 'f' for 'final' is kind of nice.
>
>Is this what we would want the constructor to take?
>
>  RationalVersion( (1,0), ('b', 1) )     # 1.0b1
>  RationalVersion( (1,0), ('f',), ('post', 345) )   # 1.0.post345
>
>It seems a little too low-level. We could allow something a little nicer:
>
>  RationalVersion( (1, 0, 'b', 1) )   # 1.0b1
>  RationalVersion( (1, 0, 'post', 345) ) # 1.0.post345
>
>If we did this, then we'd probably want these attributes on RationalVersion:
>
>>>> v = RationalVersion((1,0,'b',1))
>>>> str(v)
>'1.0b1'
>>>> v.info
>(1, 0, 'b', 1)
>>>> v._cmp_info   # use for comparison
>((1, 0), ('b', 1), ('f',))
>
>
>Thoughts?
>

I'd be in favor of replacing the 'b' and 'post' and so forth with
symbolic constants - eg, RationalVersion((1, 0, RationalVersion.POST, 345)).
If not that, then I'd at least like 'final' instead of 'f', 'beta' instead
of 'b' (is that what 'b' means?) - since this doesn't necessarily need to
be tied to how the version is serialized: it's for programmers to type, so
the more obvious it is what things mean, the better.

Jean-Paul

From trentm at gmail.com  Mon Apr 27 19:53:53 2009
From: trentm at gmail.com (Trent Mick)
Date: Mon, 27 Apr 2009 10:53:53 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090427165550.24697.295875833.divmod.quotient.12452@henry.divmod.com>
References: <6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
	<20090427165550.24697.295875833.divmod.quotient.12452@henry.divmod.com>
Message-ID: <6db0ea510904271053h2c03c32bl58139df2aec28c25@mail.gmail.com>

> I'd be in favor of replacing the 'b' and 'post' and so forth with
> symbolic constants - eg, RationalVersion((1, 0, RationalVersion.POST, 345)).
> If not that, then I'd at least like 'final' instead of 'f', 'beta' instead
> of 'b' (is that what 'b' means?) - since this doesn't necessarily need to
> be tied to how the version is serialized: it's for programmers to type, so
> the more obvious it is what things mean, the better.

Yes, it doesn't necessarily need to be tied to the version
serialization, but there is some elegance if this is the case. Granted
the 'f' for 'final' isn't obvious, but in the context of version
strings 'b' for 'beta' and 'a' for 'alpha' is pretty obvious, no? The
'f' is just internal, anyway. I'd rather not get into the situation
where we start allowing both "b" and "beta" as arguments to the
constructor to mean the same thing.

If people thought it would help, I'd be cool with:

  RationalVersion.BETA = 'b'

et al.

Trent

-- 
Trent Mick
trentm at gmail.com

From tseaver at palladion.com  Mon Apr 27 20:16:33 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 27 Apr 2009 14:16:33 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>
	<6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
Message-ID: <gt4sqi$1as$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trent Mick wrote:
>>>  Can a constructor which takes each part of the version data
>>> as a separate object be added?
>> Sounds good, I'd be in favor of making RationalVersion using explicit
>> version bits,
>> and having some kind of function:
>>
>> str2version(somestring) -> RationalVersion instance
> 
> Or we could have:
> 
>    RationalVersion(...version bits...)
>    RationalVersion.from_string(s)      # this is a @classmethod

I would prefer keeping the string version of __init__, with the "I'm in
control, dammit" version reserved for the non-default factory method.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ9faB+gerLs4ltQ4RAsGtAJ9p6z3WjdM8AxLs6kfBgxcHOuheHQCeNSra
JXKlglWpco8D1uFhSj0CLcA=
=EdQY
-----END PGP SIGNATURE-----


From patf at well.com  Mon Apr 27 21:20:32 2009
From: patf at well.com (patrick flaherty)
Date: Mon, 27 Apr 2009 12:20:32 -0700
Subject: [Distutils] add new compiler to distutils
Message-ID: <49F60580.8040905@well.com>


Hi,

Relatively new to python (and very new to distutils) but experienced 
programmer.

I'm trying to do a 64 bit build of 
http://sourceforge.net/projects/pyopenssl/

I'm on Windows Server 2008 R2.  First had to acquaint myself with 
mingw32.  That built but the DLLs (or rather PYDs) that are created 
won't load at runtime.  I transferred the build to a 32 bit machine; the 
PYDs loaded; and the program ran successfully.  Thus, as I said, I'm 
looking to rebuild in 64 bits.

It's seems that there's a 64 bit version of mingw in development on 
sourceforge:  http://sourceforge.net/projects/mingw-w64.  You download a 
toolchain; add that to your Windows PATH;  and I'm able to build a 
sample hello.c successfully.

The problem is with Python and setup.py.  When I try to specify compiler 
i86_64-mingw32 or some variant to setup.py, I get the following:

> L:\pf\Python\pyOpenSSL>setup.py build_ext -Ic:\openssl\include -c 
> x86_64-pc-mingw32
> running build_ext
> error: don't know how to compile C/C++ code on platform 'nt' with 
> 'x86_64-pc-mingw32' compiler
So after fumbling around a bit, where I've arrived at is that in the 
distutils package ( c:\python26\lib\distutils ) there are various files 
such as msvccompiler.py, msvccompiler9.py (I have VS 2008 installed), 
ccompiler.py and cygwincompiler.py that would seem to indicate that 
'new' compilers have to have support built into distutils.

Is this correct, and if so, how does one go about the task of adding a 
new compiler?

thanx - pat


From trentm at gmail.com  Tue Apr 28 01:25:34 2009
From: trentm at gmail.com (Trent Mick)
Date: Mon, 27 Apr 2009 16:25:34 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <gt4sqi$1as$1@ger.gmane.org>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>
	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>
	<6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
	<gt4sqi$1as$1@ger.gmane.org>
Message-ID: <6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>

[Tarek]
>>> Sounds good, I'd be in favor of making RationalVersion using explicit
>>> version bits,
>>> and having some kind of function:
>>>
>>> str2version(somestring) -> RationalVersion instance

[Trent]
>> Or we could have:
>>
>> ? ?RationalVersion(...version bits...)
>> ? ?RationalVersion.from_string(s) ? ? ?# this is a @classmethod

[Tres]
> I would prefer keeping the string version of __init__, with the "I'm in
> control, dammit" version reserved for the non-default factory method.

I'd tend to agree, but I don't feel strongly about it.

Tarek,
Why would you prefer the default constructor argument be the version bits form.


Trent

-- 
Trent Mick
trentm at gmail.com

From eric at trueblade.com  Tue Apr 28 02:09:45 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 27 Apr 2009 20:09:45 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>	<6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>	<gt4sqi$1as$1@ger.gmane.org>
	<6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>
Message-ID: <49F64949.2020608@trueblade.com>

> [Trent]
>>> Or we could have:
>>>
>>>    RationalVersion(...version bits...)
>>>    RationalVersion.from_string(s)      # this is a @classmethod
> 
> [Tres]
>> I would prefer keeping the string version of __init__, with the "I'm in
>> control, dammit" version reserved for the non-default factory method.
> 
> I'd tend to agree, but I don't feel strongly about it.

I agree. The more common one will be the string version, it should be 
__init__.


From sridharr at activestate.com  Tue Apr 28 02:52:16 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Mon, 27 Apr 2009 17:52:16 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <49F64949.2020608@trueblade.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>	<6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>	<gt4sqi$1as$1@ger.gmane.org>	<6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>
	<49F64949.2020608@trueblade.com>
Message-ID: <49F65340.8090602@activestate.com>

On 09-04-27 05:09 PM, Eric Smith wrote:
>> [Trent]
>>>> Or we could have:
>>>>
>>>>    RationalVersion(...version bits...)
>>>>    RationalVersion.from_string(s)      # this is a @classmethod
>>
>> [Tres]
>>> I would prefer keeping the string version of __init__, with the "I'm in
>>> control, dammit" version reserved for the non-default factory method.
>>
>> I'd tend to agree, but I don't feel strongly about it.
>
> I agree. The more common one will be the string version, it should be
> __init__.

I prefer `from_string` for the following reasons:

   - `__init__` is typically expected to accept object components as 
arguments, rather than a parse-able string.
   - `RationalVersion.from_string('...')` is much more explicit than 
`RationalVersion('...')` and explicit is better than implicit. Besides, 
the standard library has several instances of `from_string`.

From david at ar.media.kyoto-u.ac.jp  Tue Apr 28 05:23:43 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 28 Apr 2009 12:23:43 +0900
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
Message-ID: <49F676BF.4000100@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
>
> I am not sur to understand what you are trying to achieve.
>
> This indicates your environment variable CC was properly set and sent
> to the compiler.

I don't think it does: the user requires /opt/SUNWspro/bin/cc as a C
compiler, and easy_install uses /opt/studio/SOS11/SUNWspro/bin/cc which
are different compilers, unless one is a softlink to the other.

cheers,

David

From ziade.tarek at gmail.com  Tue Apr 28 07:39:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 28 Apr 2009 07:39:26 +0200
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <49F676BF.4000100@ar.media.kyoto-u.ac.jp>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>

On Tue, Apr 28, 2009 at 5:23 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Tarek Ziad? wrote:
>>
>> I am not sur to understand what you are trying to achieve.
>>
>> This indicates your environment variable CC was properly set and sent
>> to the compiler.
>
> I don't think it does: the user requires /opt/SUNWspro/bin/cc as a C
> compiler, and easy_install uses /opt/studio/SOS11/SUNWspro/bin/cc which
> are different compilers, unless one is a softlink to the other.

Oops, correct, I didn't notice there were different.

Rupert, can you try downloading that package, uncompress it and run :

# CC=/opt/SUNWspro/bin/cc python setup.py install

If this fails too, there's a problem in Distutils.sysconfig.customize_compiler
we need to look at. (basically, it looks at os.environ if your
compiler type is "unix"
and not mingw or bcpp, or msvc etc..)

If it works, the problem is on easy_install itself.


>
> cheers,
>
> David
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Apr 28 07:53:27 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 28 Apr 2009 07:53:27 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422133815.24697.334482636.divmod.quotient.10511@henry.divmod.com>
	<94bdd2610904230211y657c5363y79696b43a06a7ac0@mail.gmail.com>
	<6db0ea510904270936q4871ecc0r331d1cbc51b7145@mail.gmail.com>
	<gt4sqi$1as$1@ger.gmane.org>
	<6db0ea510904271625i18dc4232s74a1cd45865408c3@mail.gmail.com>
Message-ID: <94bdd2610904272253qedd16eal9259d6b667dc66b1@mail.gmail.com>

On Tue, Apr 28, 2009 at 1:25 AM, Trent Mick <trentm at gmail.com> wrote:
> [Tarek]
>>>> Sounds good, I'd be in favor of making RationalVersion using explicit
>>>> version bits,
>>>> and having some kind of function:
>>>>
>>>> str2version(somestring) -> RationalVersion instance
>
> [Trent]
>>> Or we could have:
>>>
>>> ? ?RationalVersion(...version bits...)
>>> ? ?RationalVersion.from_string(s) ? ? ?# this is a @classmethod
>
> [Tres]
>> I would prefer keeping the string version of __init__, with the "I'm in
>> control, dammit" version reserved for the non-default factory method.
>
> I'd tend to agree, but I don't feel strongly about it.
>
> Tarek,
> Why would you prefer the default constructor argument be the version bits form.

The example that came in mind to me was datetime, that uses the bits
in the constructor, plus provides conversion function or class methods
to handle the string form:

datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])

So like Sridhar, I'd rather have an detailed constructor.

I'm 0- on Tres proposal though, because the string form will be the
most used form.

Cheers
Tarek

From david.lyon at preisshare.net  Tue Apr 28 07:52:44 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 28 Apr 2009 01:52:44 -0400
Subject: [Distutils] =?utf-8?b?R2VuZXJhbCBRdWVzdGlvbi4uLiAuRUdHcy4uLiBh?=
	=?utf-8?q?ny_good=3F?=
In-Reply-To: <364299f40904261905k26e95e04l49356c9f050c8fe@mail.gmail.com>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
	<364299f40904261905k26e95e04l49356c9f050c8fe@mail.gmail.com>
Message-ID: <ba0eea27bc693a8051693576beda0670@preisshare.net>


Hi all, me again..

I think I have a handle on how to implement package deinstallation
in my gui package manager.

One thing I came accross is .EGG files...

I'm wondering what people think of these?

are they good?

how do people see them fitting in with distutils.. ? etc

Somebody mentioned they don't work to well under gentoo but i
don't know what the exact complaint is.

Are source distributions (ie distutils) competitive with .EGG
packaging?

Regards

David


From chris at simplistix.co.uk  Tue Apr 28 14:51:13 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 28 Apr 2009 13:51:13 +0100
Subject: [Distutils] [Catalog-sig] Setuptools + Subversion 1.6 - new
 setuptools release necessary
In-Reply-To: <49F69462.6050309@zopyx.com>
References: <49F69462.6050309@zopyx.com>
Message-ID: <49F6FBC1.8080001@simplistix.co.uk>

Andreas Jung wrote:
> it is known that the latest setuptools version produces broken
> packages with SVN 1.6 checkouts. Could we get a fixed setuptools
> version asap - fixing this issue is essential
> (there is some patch floating around).

I think you meant this to go to the distutils sig...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From lists at zopyx.com  Tue Apr 28 14:58:27 2009
From: lists at zopyx.com (Andreas Jung)
Date: Tue, 28 Apr 2009 14:58:27 +0200
Subject: [Distutils] [Catalog-sig] Setuptools + Subversion 1.6 - new
 setuptools release necessary
In-Reply-To: <49F6FBC1.8080001@simplistix.co.uk>
References: <49F69462.6050309@zopyx.com> <49F6FBC1.8080001@simplistix.co.uk>
Message-ID: <49F6FD73.10809@zopyx.com>

On 28.04.2009 14:51 Uhr, Chris Withers wrote:
> Andreas Jung wrote:
>> it is known that the latest setuptools version produces broken
>> packages with SVN 1.6 checkouts. Could we get a fixed setuptools
>> version asap - fixing this issue is essential
>> (there is some patch floating around).
>
> I think you meant this to go to the distutils sig...

Ups, sorry.

Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090428/7439dc4f/attachment.vcf>

From pje at telecommunity.com  Tue Apr 28 15:04:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 28 Apr 2009 09:04:52 -0400
Subject: [Distutils] General Question... .EGGs... a ny good?
In-Reply-To: <ba0eea27bc693a8051693576beda0670@preisshare.net>
References: <319e029f0904180209t56e35c4ew96c91b63f3e7da20@mail.gmail.com>
	<94bdd2610904180223r57513eaaj7057de387a83f0b0@mail.gmail.com>
	<364299f40904261905k26e95e04l49356c9f050c8fe@mail.gmail.com>
	<ba0eea27bc693a8051693576beda0670@preisshare.net>
Message-ID: <20090428130219.F2B923A4070@sparrow.telecommunity.com>

At 01:52 AM 4/28/2009 -0400, David Lyon wrote:
>Are source distributions (ie distutils) competitive with .EGG
>packaging?

Yes.  Really the main reason to distribute egg files is if you want 
end-users to be able to download a single file as a plugin for your 
application, or certain other special cases.


From marius at pov.lt  Tue Apr 28 21:03:57 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 28 Apr 2009 22:03:57 +0300
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
Message-ID: <20090428190356.GE19312@fridge.pov.lt>

On Wed, Apr 22, 2009 at 11:12:22AM +0200, Tarek Ziad? wrote:
> We worked during Pycon on version comparisons. Distutils has one but
> it is a bit strict, setuptools has another one, but it's a bit
> heuristic.
> 
> We would like to propose the inclusion for Python 2.7 of a new version
> comparison algorithm, based on the discussion Fedora, Ubuntu and
> Python people had. The plan would be to deprecate the current one
> (which is not really used anyway) and provide, promote this one.

I assume by "not really used" you refer to the distutils algorithm
rather than the setuptools one?
> 
> Trent Mick took the lead on this work at the end of Pycon, and worked
> on a prototype.
> 
> It's explained here, and there's an implementation  (I've put it at
> the top of the page for conveniency):
> 
> http://wiki.python.org/moin/Distutils/VersionComparison

I find the dot separating the "1.0" from "dev" in "1.0.dev456"
unappealing.  Why was it added?

The current common setuptools usage is "dev" with no dot and with no
requirement to have at least one digit following the "dev".  (The common
usage also implies tarring and feathering anyone who releases a package
with 'dev' in the version number into PyPI).

I'm also kind of wondering why 'dev' rather than 'pre' if we want to
unambiguously distinguish prereleases from postreleases.  If it's for
compatibility with setuptools, then why the extra dot in front of dev?

(Personally, I like 1.2 < 1.2+svn1234 < 1.2.1 as opposed to the
prevailing norm of 1.2 < 1.2.1dev < 1.2.1.)

> Please comment,

-0.5 for inventing a new algorithm

+0.5 for using an existing one (and an additional +0.1 on top of that if
it's the one used by Debian)

Marius Gedminas
-- 
I've been in the sun for a week.  I took the bold step of leaving my
laptop at home.  I found only 4K messages pending when I returned.
        -- Keith Packard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090428/af7d75ac/attachment.pgp>

From trentm at gmail.com  Tue Apr 28 21:47:49 2009
From: trentm at gmail.com (Trent Mick)
Date: Tue, 28 Apr 2009 12:47:49 -0700
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090428190356.GE19312@fridge.pov.lt>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090428190356.GE19312@fridge.pov.lt>
Message-ID: <6db0ea510904281247i4e159112vcd88c2b11357be17@mail.gmail.com>

>> http://wiki.python.org/moin/Distutils/VersionComparison
>
> I find the dot separating the "1.0" from "dev" in "1.0.dev456"
> unappealing. ?Why was it added?

I believe the consensus at the PyCon open spaces was that it separated
better in general. For example:

  1.0a2.1dev1234
  1.0a2.1.dev1234

> I'm also kind of wondering why 'dev' rather than 'pre' if we want to
> unambiguously distinguish prereleases from postreleases.

Just seemed to be in more common usage for that (around 1000 of the
about 8000 versions currently on PyPI used "dev"). "pre" was also
discussed at PyCon.

>
> (Personally, I like 1.2 < 1.2+svn1234 < 1.2.1 as opposed to the
> prevailing norm of 1.2 < 1.2.1dev < 1.2.1.)

In what is being proposed you'd need to use:

  1.2 < 1.2.post1234 < 1.2.1

Trent

-- 
Trent Mick
trentm at gmail.com

From marius at pov.lt  Wed Apr 29 01:35:17 2009
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 29 Apr 2009 02:35:17 +0300
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <6db0ea510904281247i4e159112vcd88c2b11357be17@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090428190356.GE19312@fridge.pov.lt>
	<6db0ea510904281247i4e159112vcd88c2b11357be17@mail.gmail.com>
Message-ID: <20090428233517.GC21497@fridge.pov.lt>

On Tue, Apr 28, 2009 at 12:47:49PM -0700, Trent Mick wrote:
> >> http://wiki.python.org/moin/Distutils/VersionComparison
> >
> > I find the dot separating the "1.0" from "dev" in "1.0.dev456"
> > unappealing. ?Why was it added?
> 
> I believe the consensus at the PyCon open spaces was that it separated
> better in general. For example:
> 
>   1.0a2.1dev1234
>   1.0a2.1.dev1234

I respectfully disagree with the applicability of the word "better" for
these examples.  "Less worse", maybe.

Picking a less extreme example:

  2.1.5dev1234
  2.1.5.dev1234

Hm.  Let me stare at it a bit more.  Hm.  Why not

  2.1.5.dev.1234

then?

Marius Gedminas
-- 
MCSE == Minesweeper Consultant / Solitaire Expert
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090429/2e3a987e/attachment.pgp>

From ziade.tarek at gmail.com  Wed Apr 29 11:18:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 29 Apr 2009 11:18:48 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090428233517.GC21497@fridge.pov.lt>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090428190356.GE19312@fridge.pov.lt>
	<6db0ea510904281247i4e159112vcd88c2b11357be17@mail.gmail.com>
	<20090428233517.GC21497@fridge.pov.lt>
Message-ID: <94bdd2610904290218i589c4b8bp2d09938cbc768810@mail.gmail.com>

On Wed, Apr 29, 2009 at 1:35 AM, Marius Gedminas <marius at pov.lt> wrote:
> Picking a less extreme example:
>
> ?2.1.5dev1234
> ?2.1.5.dev1234
>
> Hm. ?Let me stare at it a bit more. ?Hm. ?Why not
>
> ?2.1.5.dev.1234
>
> then?

-1,

imho the semantic of the "." here is to separate the atomic parts of
the version, and "dev1234" is one
atomic part with its own meaning. (1234 doesn't mean much alone)



>
> Marius Gedminas
> --
> MCSE == Minesweeper Consultant / Solitaire Expert
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iD8DBQFJ95K1kVdEXeem148RAqLkAKCBsSlQ9zZXUjP3oUJD+ne1joy5pgCfVH6p
> BTC/i9/MRo8I6FWKMf1g38w=
> =Trbk
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | http://ziade.org

From mihamina at lab.vectoris.fr  Thu Apr 30 08:18:36 2009
From: mihamina at lab.vectoris.fr (Mihamina Rakotomandimby (R12y))
Date: Thu, 30 Apr 2009 09:18:36 +0300
Subject: [Distutils] buildout and distribution packaging
Message-ID: <49F942BC.90800@lab.vectoris.fr>

Hi all
More and more python projects are "switching" to buildout.
The one I face directly is Plone.

I also administer servers and really love using their package management 
tools: If I use a Fedora (resp. Debian, Gentoo,...), I insist using yum/rpm 
(resp. apt/dpkgn, emerge,...) to install a software, and if the software 
needs some tunning, I act on the source package, then compile+install my 
tuned package.

The question is to know wether the current "fashion" to switch to buildout 
will make the distribution packager work easier or harder.

Taking the example of Perl and CPAN, it seems there is no nig problem, as 
well as there seems to be (in Debian at least) a specific tool to package 
Perl modules. Do you thing it will evolve that way for Python "buildouted" 
applications?

Thank you.

-- 
                              Chef de projet chez Vectoris
                                  Phone: +261 33 11 207 36
System: xUbuntu 8.10 with almost all from package install
    http://www.google.com/search?q=mihamina+rakotomandimby

From ziade.tarek at gmail.com  Thu Apr 30 09:16:15 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 09:16:15 +0200
Subject: [Distutils] buildout and distribution packaging
In-Reply-To: <49F942BC.90800@lab.vectoris.fr>
References: <49F942BC.90800@lab.vectoris.fr>
Message-ID: <94bdd2610904300016x1b6786e4x8131509473b70581@mail.gmail.com>

On Thu, Apr 30, 2009 at 8:18 AM, Mihamina Rakotomandimby (R12y)
<mihamina at lab.vectoris.fr> wrote:
> Hi all
> More and more python projects are "switching" to buildout.
> The one I face directly is Plone.
>
> I also administer servers and really love using their package management
> tools: If I use a Fedora (resp. Debian, Gentoo,...), I insist using yum/rpm
> (resp. apt/dpkgn, emerge,...) to install a software, and if the software
> needs some tunning, I act on the source package, then compile+install my
> tuned package.
>
> The question is to know wether the current "fashion" to switch to buildout
> will make the distribution packager work easier or harder.

A whole lot easier if your packager agrees that you are delivering an
*application*
composed of code, and not a list of packages.

You can build a big RPM out of your buildout and provide it,
or a .deb, with the right spec file so your various files are placed
in the right
place in the system tree.

The fact that this application might have a package present elsewhere
on your system, maybe another version, is unavoidable today and your
packager might say that it's a security whole, and that it makes it harder
for him to upgrade your app in case a package must be updated.

But your application *has* to be a black box and you have to provide upgrades
yourself.

That said imho, one day, Python will evolve and provide multi-version
support, and a feature for an application to pick the versions its needs.
It's just too fuzzy and too controversial right now.

Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Apr 30 09:34:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 09:34:26 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
Message-ID: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>

Hi,

I have reworked the PEP a little bit with people feedback.

It needs more feedback : http://svn.python.org/projects/peps/trunk/pep-0376.txt

- install/uninstall script

  I think the best solution is not to provide an install script since
third-party tool do it. Furthermore,
  there's already the simplest install script available today: you can
run "python setup.py install" on a given package
  so it gets installed.

  So what about adding just a global uninstall feature, that
uninstalls the files installed for a package, using the record
  file, and let the third party tool have better features.

- MANIFEST (SOURCES.txt)

  Ronald pointed out that it was not necessary to have a MANIFEST file
included if we are going to have a RECORD
  file, as described in the PEP.

  The MANIFEST (which in a way is equivalent to SOURCES.txt pip or
easy_install adds) is the list of source files,
  whereas the RECORD is the list if installed files (which might
include more elements in case of compilations)

  What would be the interest of having the list of source files in egg-info ?


Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Apr 30 09:46:39 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 09:46:39 +0200
Subject: [Distutils] Distutils roadmap : 8th may
Message-ID: <94bdd2610904300046s29fd70c9s6ec84639190cf695@mail.gmail.com>

Hello,

I would like to make some decisions and move forward on three topics
for Distutils :

1/ version comparison : http://wiki.python.org/moin/Distutils/VersionComparison

2/ standardization of egg-info : PEP 376 +
http://wiki.python.org/moin/Distutils/StandardizeEggInfo

3/ changes in PKG-INFO : PEP 345 +
http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt
+ http://wiki.python.org/moin/Distutils/Metadata

I propose to settle on these three topics and set next friday (8th
may) as a deadline.

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu Apr 30 13:53:15 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 07:53:15 -0400
Subject: [Distutils] buildout and distribution packaging
In-Reply-To: <94bdd2610904300016x1b6786e4x8131509473b70581@mail.gmail.com>
References: <49F942BC.90800@lab.vectoris.fr>
	<94bdd2610904300016x1b6786e4x8131509473b70581@mail.gmail.com>
Message-ID: <d4ea0dbc00ec8828cd3b8b798dfa0033@preisshare.net>



On Thu, 30 Apr 2009 09:16:15 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> The fact that this application might have a package present elsewhere
> on your system, maybe another version, is unavoidable today and your
> packager might say that it's a security whole, and that it makes it
harder
> for him to upgrade your app in case a package must be updated.
> 
... 
> That said imho, one day, Python will evolve and provide multi-version
> support, and a feature for an application to pick the versions its needs.
> It's just too fuzzy and too controversial right now.

Hi Tarek,

I have actually been debugging a lot of the old packaging code within
python
and I feel like I am slowly coming to understand it.

>From what I understand of the code, multiple versions seem to be inherently
ok, it all just depends on the order of the python-path code.

So if a package is sitting right in the application directory, it will get
picked up first. And can be used. Provided the python-path points there...

Furthermore, it seems that the .PTH files are really the key to the whole
packaging system. They are all loaded in site.py (really old code) and they
tell the underlying interpretor where to look for any of the packages that
it needs.

I can see where you are trying to drive things... and it makes sense...

David




From david.lyon at preisshare.net  Thu Apr 30 14:11:10 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 08:11:10 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
Message-ID: <86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>



On Thu, 30 Apr 2009 09:34:26 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> - install/uninstall script
> 
>   I think the best solution is not to provide an install script since
> third-party tool do it. Furthermore,
>   there's already the simplest install script available today: you can
> run "python setup.py install" on a given package
>   so it gets installed.
> 
>   So what about adding just a global uninstall feature, that
> uninstalls the files installed for a package, using the record
>   file, and let the third party tool have better features.

Today I just was writing a package-deinstallation routine and
doing small tests on the routine... :-) for my gui package
manager.

There are one or two issues here for you to think about...

 1) - distutils already exists on the installed system

    - how can you install over it?

 2) - The new distutils code won't be retrofitted to
      old systems. 

As far as I can see, it's just so much better to have a
package uninstaller in a seperate gui tool. That can be
installed over an existing installation.

> - MANIFEST (SOURCES.txt)
> 
>   Ronald pointed out that it was not necessary to have a MANIFEST file
> included if we are going to have a RECORD
>   file, as described in the PEP.
> 
>   The MANIFEST (which in a way is equivalent to SOURCES.txt pip or
> easy_install adds) is the list of source files,
>   whereas the RECORD is the list if installed files (which might
> include more elements in case of compilations)

I have done some limited testing on deinstalling packages over the
last week.

The big problem with using a manifest of installed files is that
the setup.py in distutils can generate more files than were originally
in the source package. Doing a build... for instance... leaves .o
or .obj files.

The way to solve the problem is to recursively delete the whole
package directory. That gets rid of everything. Intermediate files
and all...

> 
>   What would be the interest of having the list of source files in
> egg-info ?

Not neccessary - recursively delete the whole package directory or
.egg file.

David


From ziade.tarek at gmail.com  Thu Apr 30 14:23:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 14:23:41 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
Message-ID: <94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>

On Thu, Apr 30, 2009 at 2:11 PM, David Lyon <david.lyon at preisshare.net> wrote:
> As far as I can see, it's just so much better to have a
> package uninstaller in a seperate gui tool. That can be
> installed over an existing installation.

We are planning to propose a backport for previous versions


> The big problem with using a manifest of installed files is that
> the setup.py in distutils can generate more files than were originally
> in the source package. Doing a build... for instance... leaves .o
> or .obj files.

And creates scripts files, etc..

>
> The way to solve the problem is to recursively delete the whole
> package directory. That gets rid of everything. Intermediate files
> and all...
>
>>
>> ? What would be the interest of having the list of source files in
>> egg-info ?
>
> Not neccessary - recursively delete the whole package directory or
> .egg file.

No because you can have files installed anywhere

The way to handle de-installation is to use the recorded list of file
that can be created by the 'sdist' command at installation time

It's the --record option and we want to put its output in  "RECORD" in
the egg.info

Now my point is : do we want to add the MANIFEST (source file list)
file as well.
Setuptools and pip are adding a SOURCES.txt file at this point.

Cheers
Tarek

From david.lyon at preisshare.net  Thu Apr 30 14:35:13 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 08:35:13 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
Message-ID: <6023049d001a7fcc4e9aa09489217fb2@preisshare.net>



On Thu, 30 Apr 2009 14:23:41 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> We are planning to propose a backport for previous versions

ok

>> Not neccessary - recursively delete the whole package directory or
>> .egg file.
> 
> No because you can have files installed anywhere

That's true that they can be installed anywhere. But there always
needs to be an entry in a .PTH file along the python path to
specify where the files were installed to.

> The way to handle de-installation is to use the recorded list of file
> that can be created by the 'sdist' command at installation time
> 
> It's the --record option and we want to put its output in  "RECORD" in
> the egg.info
> 
> Now my point is : do we want to add the MANIFEST (source file list)
> file as well.
> Setuptools and pip are adding a SOURCES.txt file at this point.

Well, you can certainly try that...

But your logic fails in this case....

 - you installed a package on a system in 1998...

 - that system didn't generate all of the manifest info that
   was just invented now. (2009)

 - therefore... the code you run now cannot deinstall that
   old package.

Obviously.. there's ways around it...

They're just my observations...

David

   



From ziade.tarek at gmail.com  Thu Apr 30 14:43:42 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 14:43:42 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
Message-ID: <94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>

On Thu, Apr 30, 2009 at 2:35 PM, David Lyon <david.lyon at preisshare.net> wrote:
>>
>> No because you can have files installed anywhere
>
> That's true that they can be installed anywhere. But there always
> needs to be an entry in a .PTH file along the python path to
> specify where the files were installed to.

You don't specify in this pth file that the package "foo" installed
the script "bar"
in the bin/ directory of your python installation.

>
>> The way to handle de-installation is to use the recorded list of file
>> that can be created by the 'sdist' command at installation time
>>
>> It's the --record option and we want to put its output in ?"RECORD" in
>> the egg.info
>>
>> Now my point is : do we want to add the MANIFEST (source file list)
>> file as well.
>> Setuptools and pip are adding a SOURCES.txt file at this point.
>
> Well, you can certainly try that...
>
> But your logic fails in this case....
>
> ?- you installed a package on a system in 1998...
>
> ?- that system didn't generate all of the manifest info that
> ? was just invented now. (2009)
>
> ?- therefore... the code you run now cannot deinstall that
> ? old package.

There's no plan to provide the uninstall feature for old packages,
but to provide a new egg-info standard that can be used on previous
Python version. (and therefore the uninstall feature that goes with it)

>
> Obviously.. there's ways around it...

I don't see how, since there were no standard way back then to keep
track of installed files

Tarek

-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu Apr 30 15:32:42 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 09:32:42 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
Message-ID: <1a27e131b5d7ae18a041cdf802130281@preisshare.net>



On Thu, 30 Apr 2009 14:43:42 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
>> That's true that they can be installed anywhere. But there always
>> needs to be an entry in a .PTH file along the python path to
>> specify where the files were installed to.
> 
> You don't specify in this pth file that the package "foo" installed
> the script "bar"
> in the bin/ directory of your python installation.

But during package installation, this information will be written
into a .PTH file somewhere along the python path...

> There's no plan to provide the uninstall feature for old packages,
> but to provide a new egg-info standard that can be used on previous
> Python version. (and therefore the uninstall feature that goes with it)
> 

but if you can't uninstall an older package from the system... then
how can you install the newer version with all the new information..

this leaves the user quite stuck...

>> Obviously.. there's ways around it...
> 
> I don't see how, since there were no standard way back then to keep
> track of installed files

That's my point....

Having a system where new packages can be deinstalled and old packages
cannot be deinstalled will be totally confusing for users. 

It will drive most users crazy - if you think about it.

If you have a deinstallation facility, then it must work for new
packages as well as old. Otherwise, imho there's just no point.

David




From david at ar.media.kyoto-u.ac.jp  Thu Apr 30 15:24:07 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 30 Apr 2009 22:24:07 +0900
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <1a27e131b5d7ae18a041cdf802130281@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
Message-ID: <49F9A677.6090809@ar.media.kyoto-u.ac.jp>

David Lyon wrote:
> If you have a deinstallation facility, then it must work for new
> packages as well as old. Otherwise, imho there's just no point.
>   

It is impossible to uninstall a package if you don't have a recording of
what was installed. Removing every directories is a bad idea, as it may
remove files which were not installed by the package. No packaging
system does that. That's actually one of the very few rules which is
followed by any installation method, be it windows installer, linux
packages, etc...

cheers,

David

From ziade.tarek at gmail.com  Thu Apr 30 15:50:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 15:50:19 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <1a27e131b5d7ae18a041cdf802130281@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
Message-ID: <94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>

On Thu, Apr 30, 2009 at 3:32 PM, David Lyon <david.lyon at preisshare.net> wrote:
>
>
> On Thu, 30 Apr 2009 14:43:42 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>>> That's true that they can be installed anywhere. But there always
>>> needs to be an entry in a .PTH file along the python path to
>>> specify where the files were installed to.
>>
>> You don't specify in this pth file that the package "foo" installed
>> the script "bar"
>> in the bin/ directory of your python installation.
>
> But during package installation, this information will be written
> into a .PTH file somewhere along the python path...

No, you just have a list of relative paths to installed package there that's it.

> If you have a deinstallation facility, then it must work for new
> packages as well as old. Otherwise, imho there's just no point.
>

See David C. comment. How do you expect such a system to properly install
a project that was installed in an unknown manner in the past ?

You don't even know what packages were installed for your project
unless you digg into the source distribution.

For example, if a project installs two packages in your Python, how do
you know it ?

A project is not a self-contained directory we can remove recursively

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu Apr 30 16:17:42 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 10:17:42 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <49F9A677.6090809@ar.media.kyoto-u.ac.jp>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<49F9A677.6090809@ar.media.kyoto-u.ac.jp>
Message-ID: <b1bad02cee7d01b32556d080a4664e6e@preisshare.net>


Hi David,

On Thu, 30 Apr 2009 22:24:07 +0900, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:

> It is impossible to uninstall a package if you don't have a recording of
> what was installed. 

Not true.

> Removing every directories is a bad idea, as it may
> remove files which were not installed by the package. 

In python, packages go into their own unique directories
or .egg files.

> No packaging system does that. That's actually one of the very few 
> rules which is followed by any installation method, be it windows 
> installer, linux packages, etc...

I don't disagree with your statement.

However, the packaging system you are talking about didn't follow
the abovementioned rules when it was designed.

Forcing it to "behave" like others when it wasn't designed to
work that way in the first place won't give you the desired
results. 

I've been tracing through the old code....

There is much inner beauty in that old code...

It seems illogical to me that on the one hand, programmers are
told at conferences to delete package directories manually, and
then here... people are discouraged from doing it programmatically.

Anyway, I'm not going to discuss this with you guys too much more.

All I was doing was sharing my experience because I have been
debugging the python interpreter lately and seeing how the
packaging stuff works.

In summary... packages are just directories with an __init__.py
file in them. Sometimes they are zipped into eggs.

If directories get deleted, the python interpreter doesn't care.

It moves right along. (down the python-path list)

You can try putting more formality and burdens on the developer
when they are trying to build a package (a directory). But
somehow it goes against the simplicity of the 'old code'.

Don't forget....

the whole point of "python setup.py" is to tell python to
go "build a whole lot of stuff".. "and put it away"

Now.. what you are saying is that you don't want that paradigm
anymore... you want it to be changed to "here are all the
files in a nice manifest" paradigm.

The "new" paradigm is not pythonic....

It only works when you have all the files/drivers already
built and you want them "plonked" in the right places.

Anyway... I'm just bouncing feedback to your ideas...

Discussing in the spirit of sharing ideas...

David


From david.lyon at preisshare.net  Thu Apr 30 16:34:14 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 10:34:14 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
Message-ID: <e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>

Hi Tarek,

On Thu, 30 Apr 2009 15:50:19 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
>> But during package installation, this information will be written
>> into a .PTH file somewhere along the python path...
> 
> No, you just have a list of relative paths to installed package there
> that's it.

It means the same thing.

> You don't even know what packages were installed for your project
> unless you digg into the source distribution.

Yes I do...

I have the following code:

--code-------

import pkg_resources

ws = pkg_resources.WorkingSet()

for i in ws:
    s = str(i)
    print s

--end code---

That tells me what packages I have...

> For example, if a project installs two packages in your Python, how do
> you know it ?

The above code will tell me...

both packages will appear in the list.

David



From ziade.tarek at gmail.com  Thu Apr 30 16:48:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 16:48:32 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
Message-ID: <94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>

On Thu, Apr 30, 2009 at 4:34 PM, David Lyon <david.lyon at preisshare.net> wrote:
> Hi Tarek,
>
> On Thu, 30 Apr 2009 15:50:19 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>>> But during package installation, this information will be written
>>> into a .PTH file somewhere along the python path...
>>
>> No, you just have a list of relative paths to installed package there
>> that's it.
>
> It means the same thing.
>

No, you don't have the exaustive list of the files installed.

>> You don't even know what packages were installed for your project
>> unless you digg into the source distribution.
>
> Yes I do...
>
> I have the following code:
>
> --code-------
>
> import pkg_resources
>
> ws = pkg_resources.WorkingSet()
>
> for i in ws:
> ? ?s = str(i)
> ? ?print s
>
> --end code---

That doesn't give you a list of installed package for your *project*.

Remember that a Distutils project (one setup.py file) can install
several packages in your python, and depending on the way you install(ed)
it, you might not be abe to get back your packages.

Try it yourself, here's such a package:

create a dir with a setup.py file :

"""
from distutils.core import setup

setup(name='ok', packages=['one', 'two'])
"""

now create a "one" subdirectory with a __init__ file in it, same for "two"

next, run :

$ python setup.py install

Now tell me how you know, looking at your Python installation
(site-packages), that "ok" installed "one" and "two"'... ;)



Tarek
-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu Apr 30 17:08:26 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 11:08:26 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
Message-ID: <c7c43dc617a0bd08c241a4a297edc51e@preisshare.net>



On Thu, 30 Apr 2009 16:48:32 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> No, you don't have the exaustive list of the files installed.

My testing indicates that pkg_resources works ok.

>> --code-------
>>
>> import pkg_resources
>>
>> ws = pkg_resources.WorkingSet()
>>
>> for i in ws:
>> ? ?s = str(i)
>> ? ?print s
>>
>> --end code---
> 
> That doesn't give you a list of installed package for your *project*.

I am writing a package manager gui. I don't have a project.

If there are projects or things like that, I tell the user to go
get zc.buildout or pip.

> Remember that a Distutils project (one setup.py file) can install
> several packages in your python, and depending on the way you install(ed)
> it, you might not be abe to get back your packages.

I am driving easy_install..

Seems to work reasonably well in many cases....

> Try it yourself, here's such a package:
> 
> create a dir with a setup.py file :
> 
> """
> from distutils.core import setup
> 
> setup(name='ok', packages=['one', 'two'])
> """
> 
> now create a "one" subdirectory with a __init__ file in it, same for
"two"
> 
> next, run :
> 
> $ python setup.py install
> 
> Now tell me how you know, looking at your Python installation
> (site-packages), that "ok" installed "one" and "two"'... ;)

That isn't a typical use case in my scenario.

A typical user just wants to install package x,y and z from
pypi. They want them to go "into python" and it is as simple
as that. Those packages will probably end up in site-packages
but as likely as not, the user won't care where they go as
long as they are available when they are needed.

When they are finished, or they don't like them.. they want
to deinstall them by clicking the remove button.

In what I am doing, the nested directories would have been
happily removed.

I appreciate your feedback and it is nice discussing these
things with you.

Best Regards

David




From ziade.tarek at gmail.com  Thu Apr 30 17:29:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 30 Apr 2009 17:29:40 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <c7c43dc617a0bd08c241a4a297edc51e@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
	<c7c43dc617a0bd08c241a4a297edc51e@preisshare.net>
Message-ID: <94bdd2610904300829n1221dcc4yebad5f3ea9abfd26@mail.gmail.com>

On Thu, Apr 30, 2009 at 5:08 PM, David Lyon <david.lyon at preisshare.net> wrote:
> A typical user just wants to install package x,y and z from
> pypi. They want them to go "into python" and it is as simple
> as that. Those packages will probably end up in site-packages
> but as likely as not, the user won't care where they go as
> long as they are available when they are needed.
>
> When they are finished, or they don't like them.. they want
> to deinstall them by clicking the remove button.
>
> In what I am doing, the nested directories would have been
> happily removed.
>

What is your definition of a "package" and a "project" ?

If by "package" you mean the packages that are available at PyPI
for instance, they may have several packages included in them.

If you use easy_install, they will stay nested in the same directory
or zip file in your site-packages, but you still may have some
files created outside.

for instance, a console script generated by setuptools console_script
entry point.

In your case, you may find everything needed to uninstall though, by
looking inside
the EGG-INFO directory of the project.

Now back to the initial discussion : if some projects are not
installed with easy_install
you won't have all the info needed to uninstall the various packages
and scripts.

So basically: we are trying to add in Distutils such a standard, (it
differs slighlty from
setuptools but it's the same spirit)

And then, as we said earlier, it will be impossible to uninstall packages that
were previously uninstalled with some other techniques since we won't have
the info required.

You said it yourself : your own uninstall system works only for
packages installed with easy_install,
so I guess you agree that your system cannot uninstall other packages properly

(I show you an example of a package you couldn't uninstall in my previous mail.)

Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Thu Apr 30 20:01:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 30 Apr 2009 14:01:31 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <b1bad02cee7d01b32556d080a4664e6e@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<49F9A677.6090809@ar.media.kyoto-u.ac.jp>
	<b1bad02cee7d01b32556d080a4664e6e@preisshare.net>
Message-ID: <20090430175857.B5B093A4070@sparrow.telecommunity.com>

At 10:17 AM 4/30/2009 -0400, David Lyon wrote:
>In summary... packages are just directories with an __init__.py
>file in them. Sometimes they are zipped into eggs.

You are confusing "Python package" with "Python project".  Projects 
are zipped into eggs, and may contain zero or more 
packages.  Packages, conversely, may contain contents from multiple projects.

A distribution found on PyPI such as "FooBar-2.79" is the 2.79 
release of the *project* "FooBar"...  this tells you absolutely 
nothing about the names of packages or modules contained in that project.


>You can try putting more formality and burdens on the developer
>when they are trying to build a package (a directory). But
>somehow it goes against the simplicity of the 'old code'.

You don't understand packages, so the rest of your email here makes 
no sense.  The simplicity that you are describing *never actually 
existed*, even before setuptools and eggs came on the scene.