From david.lyon at preisshare.net  Sun Feb  1 00:14:31 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 31 Jan 2009 18:14:31 -0500
Subject: [Distutils]
 =?utf-8?q?=5BPython_Language_Summit=5D_Distutils_/_Pa?=
 =?utf-8?q?ckaging=09survey?=
In-Reply-To: <20090131153556.407903A4114@sparrow.telecommunity.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<20090131153556.407903A4114@sparrow.telecommunity.com>
Message-ID: <e38d0137b0c1a766be5a7b9ba3a95b90@preisshare.net>



On Sat, 31 Jan 2009 10:37:56 -0500, "P.J. Eby" <pje at telecommunity.com>
wrote:

> If this is true, then there's no need to distinguish between .py
> files and any other data files - they both belong in /share to begin
> with, not in /lib.  Or else they ALL belong in /lib.  The entire "FHS
> demands they be split" concept is wrong from the get-go, under that
> interpretation.

Not taking away from that....

but in windows things have their own directories too and python
files should go in the right places also.

Best place for packages -> \Program Files\Common Files\Python x.x\Packages

Best place for python -> \Program Files\Python x.x

Under linux - Best place for packages -> /usr/local/Python x.x/Packages.

(a python developer shouldn't go looking in system directories for
a source package he may want to look at)

Putting files where people expect to find them, doesn't cost more. The
only reason things are the way they are is because of history. But at
the same time, if we want our software to "live" in an operating system
we must follow their rules to be regarded as good citizens.

:-)

Regards

David


From ben+python at benfinney.id.au  Sun Feb  1 01:55:16 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 01 Feb 2009 11:55:16 +1100
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
Message-ID: <871vuj562z.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> They are not arbitrary - they come from standard usage and have a
> rationale, at least on Unix

Agreed with this.

> (datadir for arch independent, and libdir for arch dependent, to
> simplify).

I think you've not only simplified, you've done so in the wrong
direction. I'd say instead that ?datadir? is for *non-executable*
files, and ?libdir? for executable.

That something is non-executable usually means that it's
?architecture-independent?, and traditionally, ?executable? has
meant ?architecture-dependent?, but I don't think that's a necessary
distinction here. Viz. the placement of the Python standard library,
including mostly architecture-independent executable files, in a
?libdir?.

> But you mostly do not need to care, as a developer: .py files would
> be considered as data files, [?]

That would lead to a misguided attempt to keep ?foo.py? separate from
the resulting ?foo.pyc?, which would be far more pain than it's worth
AFAICT.

Rather, ?foo.py? (and ?foo.pyc?) files would both be identified as
*executable* resources, and thence installed to the same location.

-- 
 \                   ?Too many Indians spoil the golden egg.? ?Sir Joh |
  `\                                                   Bjelke-Petersen |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Sun Feb  1 02:02:24 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 01 Feb 2009 12:02:24 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<87wsccz8nw.fsf@benfinney.id.au> <49841318.6000800@palladion.com>
	<498412B2.6060703@ar.media.kyoto-u.ac.jp>
	<4984B1E7.6060101@palladion.com>
Message-ID: <87wscb3r6n.fsf@benfinney.id.au>

Tres Seaver <tseaver at palladion.com> writes:

> Calling .py files "data" is another choice which seems (to me)
> arbitrary: in my view, .py files, as well as others, are "software",
> not data, and should be kept together with the other software (e.g.,
> templates used for rendering HTML) that they are distributed with.

Everything in a Python package ? data, programs, templates,
documentation files, etc. ? is software (as opposed to hardware).

I think you mean that ?foo.py? files are program files, rather than
data. I certainly agree with this.

-- 
 \        ?The restriction of knowledge to an elite group destroys the |
  `\                   spirit of society and leads to its intellectual |
_o__)                                impoverishment.? ?Albert Einstein |
Ben Finney


From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 06:15:25 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 14:15:25 +0900
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <871vuj562z.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<871vuj562z.fsf@benfinney.id.au>
Message-ID: <49852FED.2040608@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
>
> I think you've not only simplified, you've done so in the wrong
> direction. I'd say instead that ?datadir? is for *non-executable*
> files, and ?libdir? for executable.
>   

It depends on what you mean by executable: it is obviously wrong when
executable = have the executable bit, so what do you mean by executable ?

The arch independent/dependent is much clearer IMHO, and is the
vocabulary used by the FHS, AFAIK. It is certainly the one used by the
configure script help:

  --bindir=DIR           user executables [EPREFIX/bin]
  --sbindir=DIR          system admin executables [EPREFIX/sbin]
  --libexecdir=DIR       program executables [EPREFIX/libexec]
  --sysconfdir=DIR       read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR   modifiable architecture-independent data
[PREFIX/com]
  --localstatedir=DIR    modifiable single-machine data [PREFIX/var]
  --libdir=DIR           object code libraries [EPREFIX/lib]
  --includedir=DIR       C header files [PREFIX/include]
  --oldincludedir=DIR    C header files for non-gcc [/usr/include]
  --datarootdir=DIR      read-only arch.-independent data root
[PREFIX/share]
  --datadir=DIR          read-only architecture-independent data
[DATAROOTDIR]
  --infodir=DIR          info documentation [DATAROOTDIR/info]
  --localedir=DIR        locale-dependent data [DATAROOTDIR/locale]
  --mandir=DIR           man documentation [DATAROOTDIR/man]
  --docdir=DIR           documentation root [DATAROOTDIR/doc/libsndfile]
  --htmldir=DIR          html documentation [DOCDIR]
  --dvidir=DIR           dvi documentation [DOCDIR]
  --pdfdir=DIR           pdf documentation [DOCDIR]
  --psdir=DIR            ps documentation [DOCDIR]


After all, that's one of the rationale for the FHS: some files are
shared by different computers, some are not.

> That something is non-executable usually means that it's
> ?architecture-independent?, and traditionally, ?executable? has
> meant ?architecture-dependent?,

I don't understand this in the context of python: most python
executables are architecture independent (python scripts). For example,
/usr/bin/ipython is a python script.

>  but I don't think that's a necessary
> distinction here. Viz. the placement of the Python standard library,
> including mostly architecture-independent executable files, in a
> ?libdir?.
>   

I think we should stop thinking, and start looking :) Here are some
files as packaged in the python-numpy Ubuntu package package:

/usr/bin/f2py -> (default) python script
/usr/bin/f2py2.4 -> 2.4 script
/usr/bin/f2py2.4-dbg -> 2.4 debug script
...
/usr/lib/python2.4/site-packages/numpy/core/_dotblas.so -> C extension,
obviously arch dependent.
...
/usr/share/doc/python-numpy/CAPI.txt.gz -> some doc
/usr/share/pyshared/numpy/__config__.py -> the python scripts


So you have things which are supposed to be called from the shell in
bindir, the C extension in libdir, the python code in datadir, the docs
in docdir, etc...

>
> That would lead to a misguided attempt to keep ?foo.py? separate from
> the resulting ?foo.pyc?, which would be far more pain than it's worth
> AFAICT.
>   

There is no .pyc in a debian package: they are computed at installation
time. The .pyc and .so are put in
/usr/lib/python2.5/site-package/package-name, but the .py are not (they
are softlink to the actual .py, which are in
/usr/data/pyshared/package-name).

David

From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 06:32:06 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 14:32:06 +0900
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <20090131165847.GA13200@laurie.devork>
References: <1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<20090131153556.407903A4114@sparrow.telecommunity.com>
	<20090131165847.GA13200@laurie.devork>
Message-ID: <498533D6.9020005@ar.media.kyoto-u.ac.jp>

Floris Bruynooghe wrote:
> On Sat, Jan 31, 2009 at 10:37:56AM -0500, P.J. Eby wrote:
>   
>> At 05:40 PM 1/31/2009 +0900, David Cournapeau wrote:
>>     
>>> But you mostly do not need to care, as a developer: .py files would be
>>> considered as data files, extensions as arch-dependent, etc...
>>>       
>> If this is true, then there's no need to distinguish between .py files 
>> and any other data files - they both belong in /share to begin with, not 
>> in /lib.  Or else they ALL belong in /lib.  The entire "FHS demands they 
>> be split" concept is wrong from the get-go, under that interpretation.
>>     
>
> I don't think trying to split off .py and .so files is a good idea.
>   

That's how it is done in debian and ubuntu packages. So if you want to
make it easier for those distributions, you don't have a choice.

cheers,

David

From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 06:41:18 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 14:41:18 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <20090131153556.407903A4114@sparrow.telecommunity.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<20090131153556.407903A4114@sparrow.telecommunity.com>
Message-ID: <498535FE.2020506@ar.media.kyoto-u.ac.jp>

P.J. Eby wrote:
> At 05:40 PM 1/31/2009 +0900, David Cournapeau wrote:
>> Ian Bicking wrote:
>> > On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
>> > I wouldn't want to use those.  What goes in libdir, what goes in
>> > datadir?  I don't know, and frankly the distinctions start getting
>> > really arbitrary.
>>
>> They are not arbitrary - they come from standard usage and have a
>> rationale, at least on Unix (datadir for arch independent, and libdir
>> for arch dependent, to simplify).
>>
>> But you mostly do not need to care, as a developer: .py files would be
>> considered as data files, extensions as arch-dependent, etc...
>
> If this is true, then there's no need to distinguish between .py files
> and any other data files - they both belong in /share to begin with,
> not in /lib.

True, and that's exactly what is written there:

http://wiki.python.org/moin/Distutils/Proposals/AutoconfLikeOptions

I am sure I forgot some categories - I only put the ones I could think
of right away from looking at one python package I am familiar with (numpy).

> Or else they ALL belong in /lib.  The entire "FHS demands they be
> split" concept is wrong from the get-go, under that interpretation.

nobody made that suggestion AFAICS,

David


From regebro at gmail.com  Sun Feb  1 07:26:01 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 1 Feb 2009 07:26:01 +0100
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <E64C99AC-4E00-4385-83D4-D634F01AB8DB@zooko.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<E64C99AC-4E00-4385-83D4-D634F01AB8DB@zooko.com>
Message-ID: <319e029f0901312226t109110d9gb191aa01e3257528@mail.gmail.com>

On Sat, Jan 31, 2009 at 23:21, zooko <zooko at zooko.com> wrote:
> Many other people, however, are unwilling to let the Python packaging tool
> nor the packages that it is installing write into their system directories.
>  For these people, the system directories are never allowed to be written
> into by any tool other than their operating system packaging tool e.g. apt.
>  For these people, offering a new feature so that, in addition to the Python
> packaging tool scribbling into their system directory at install time, it
> will *also* later scribble into it again attempting to undo the changes that
> it earlier did is only compounding the problem.  :-)
>
> For those people, there are several potential solutions.  One is the path
> from Python package through Python packaging tool (e.g. stdeb) to operating
> system package (e.g. .deb) to operating system tool (e.g. apt) to system
> directories.  Another is GNU stow.
>
> For me, and for some hackers that I've talked to, making setuptools
> compatible with GNU stow would satisfy their need to have a way to cleanly
> uninstall.
>
> So, please consider this a reminder in the distutils development process
> that you are undergoing that GNU stow compatibility has many benefits.

As far as I can figure out, that's just a matter of installing things
with a prefix, like /usr/local/stow/yoursoftware instead of /usr/local
and then using stow to symlink everything into /usr/local/bin and
/usr/local/lib, etc. I may have misunderstood, though, but if not it
should already be compatible...

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From ben+python at benfinney.id.au  Sun Feb  1 07:27:53 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 01 Feb 2009 17:27:53 +1100
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<871vuj562z.fsf@benfinney.id.au>
	<49852FED.2040608@ar.media.kyoto-u.ac.jp>
Message-ID: <871vui4qom.fsf@benfinney.id.au>

David Cournapeau <david at ar.media.kyoto-u.ac.jp> writes:

> Ben Finney wrote:
> > I think you've not only simplified, you've done so in the wrong
> > direction. I'd say instead that ?datadir? is for
> > *non-executable* files, and ?libdir? for executable.
> 
> It depends on what you mean by executable: it is obviously wrong
> when executable = have the executable bit, so what do you mean by
> executable ?

We could diverge into a long discussion about what ?executable?
means; but let's not. I'm willing to allow a common sense meaning to
serve for the purpose of this discussion.

The issue is less about what Ben Finney or David Cornapeau think it
means. The issue is more whether it's a useful distinction.

> > That something is non-executable usually means that it's
> > ?architecture-independent?, and traditionally, ?executable?
> > has meant ?architecture-dependent?,
> 
> I don't understand this in the context of python: most python
> executables are architecture independent (python scripts).

Yes, that's mainly my point. Whether a particular set of bits is
?architecture-dependent? or ?architecture-independent? is rather
orthogonal, in Python, to whether it's an executable program library.

-- 
 \     ?Jealousy: The theory that some other fellow has just as little |
  `\                                         taste.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 07:17:42 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 15:17:42 +0900
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <319e029f0901312226t109110d9gb191aa01e3257528@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>	<E64C99AC-4E00-4385-83D4-D634F01AB8DB@zooko.com>
	<319e029f0901312226t109110d9gb191aa01e3257528@mail.gmail.com>
Message-ID: <49853E86.9080404@ar.media.kyoto-u.ac.jp>

Lennart Regebro wrote:
> On Sat, Jan 31, 2009 at 23:21, zooko <zooko at zooko.com> wrote:
>   
>> Many other people, however, are unwilling to let the Python packaging tool
>> nor the packages that it is installing write into their system directories.
>>  For these people, the system directories are never allowed to be written
>> into by any tool other than their operating system packaging tool e.g. apt.
>>  For these people, offering a new feature so that, in addition to the Python
>> packaging tool scribbling into their system directory at install time, it
>> will *also* later scribble into it again attempting to undo the changes that
>> it earlier did is only compounding the problem.  :-)
>>
>> For those people, there are several potential solutions.  One is the path
>> from Python package through Python packaging tool (e.g. stdeb) to operating
>> system package (e.g. .deb) to operating system tool (e.g. apt) to system
>> directories.  Another is GNU stow.
>>
>> For me, and for some hackers that I've talked to, making setuptools
>> compatible with GNU stow would satisfy their need to have a way to cleanly
>> uninstall.
>>
>> So, please consider this a reminder in the distutils development process
>> that you are undergoing that GNU stow compatibility has many benefits.
>>     
>
> As far as I can figure out, that's just a matter of installing things
> with a prefix, like /usr/local/stow/yoursoftware instead of /usr/local
> and then using stow to symlink everything into /usr/local/bin and
> /usr/local/lib, etc. I may have misunderstood, though, but if not it
> should already be compatible...
>   

There are two problems with stow and setuptools:
    - setuptools refuses to install in a directory not in PYTHONPATH, so
python setup.py install --prefix=/usr/local/stow/my-package does not
work when setup.py uses setuptools. You have to use
--single-version-externally-managed (plus another option required by
this one). That's the most annoying point, since it depends on whether
setup.py uses setuptools or plain distutils. From a user POV, setuptools
broke standard python behavior.
    - namespace packages: the fact that namespace/__init__ are 'owned'
by several packages is a fundamental problem (not only for stow, but
almost any package system). But unless python itself supports namespace
packages, I don't see any solution to this one.

cheers,

Davod


From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 07:31:01 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 15:31:01 +0900
Subject: [Distutils] [Python Language Summit]
	Distutils	/	Packaging	survey
In-Reply-To: <871vui4qom.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<871vuj562z.fsf@benfinney.id.au>	<49852FED.2040608@ar.media.kyoto-u.ac.jp>
	<871vui4qom.fsf@benfinney.id.au>
Message-ID: <498541A5.2000106@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
>
> Yes, that's mainly my point. Whether a particular set of bits is
> ?architecture-dependent? or ?architecture-independent? is rather
> orthogonal, in Python, to whether it's an executable program library.
>   

I don't think I have ever used the term executable program library in
that discussion, so I don't understand your point.
Architecture independent is the wording used by FHS, not mine
(http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA),
which is why I used it in
http://wiki.python.org/moin/Distutils/Proposals/AutoconfLikeOptions.

David

From tarek.ziade at ingeniweb.com  Sun Feb  1 11:12:17 2009
From: tarek.ziade at ingeniweb.com (Tarek Ziade)
Date: Sun, 1 Feb 2009 11:12:17 +0100
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
	explicitly with -L when building extensions?
In-Reply-To: <20090131140143.GA7382@laurie.devork>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork> <49839116.2050801@enthought.com>
	<20090131140143.GA7382@laurie.devork>
Message-ID: <a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>

2009/1/31 Floris Bruynooghe <floris.bruynooghe at gmail.com>:
>> I'm leaning more and more toward this is actually a bug
>> with the distutils source on Solaris.
>
> Yes, I agree now that it is a bug in distutils and I agree with your
> fix.  The if statement should check both that it is SunOS and that it
> is using a shared python, just like the linux one.  If fact the linux
> one could just be modified:
>
> if (sys.paltform.startswith('linux') \
>    or sys.platform.startswith('gnu') \
>    or sys.platform.startswith('sunos')) \
>        and sysconfig.get_config_var('Py_ENABLE_SHARED'):
>    ...
>
> I'll leave the honours of reporting the bug to you if you agree with
> this.

Yes thanks to report this, I'll apply your fix,

Regards
Tarek

-- 
Tarek Ziad? - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une soci?t? du groupe Alter Way

From floris.bruynooghe at gmail.com  Sun Feb  1 12:57:40 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sun, 1 Feb 2009 11:57:40 +0000
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <498533D6.9020005@ar.media.kyoto-u.ac.jp>
References: <498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<20090131153556.407903A4114@sparrow.telecommunity.com>
	<20090131165847.GA13200@laurie.devork>
	<498533D6.9020005@ar.media.kyoto-u.ac.jp>
Message-ID: <20090201115740.GA3987@laurie.devork>

On Sun, Feb 01, 2009 at 02:32:06PM +0900, David Cournapeau wrote:
> Floris Bruynooghe wrote:
> > On Sat, Jan 31, 2009 at 10:37:56AM -0500, P.J. Eby wrote:
> >   
> >> At 05:40 PM 1/31/2009 +0900, David Cournapeau wrote:
> >>     
> >>> But you mostly do not need to care, as a developer: .py files would be
> >>> considered as data files, extensions as arch-dependent, etc...
> >>>       
> >> If this is true, then there's no need to distinguish between .py files 
> >> and any other data files - they both belong in /share to begin with, not 
> >> in /lib.  Or else they ALL belong in /lib.  The entire "FHS demands they 
> >> be split" concept is wrong from the get-go, under that interpretation.
> >>     
> >
> > I don't think trying to split off .py and .so files is a good idea.
> >   
> 
> That's how it is done in debian and ubuntu packages. So if you want to
> make it easier for those distributions, you don't have a choice.

Hmm fair enough, I must have missed that last time I looked at the
implementation of python-support and python-central.  But both take
the burden of having to create a symlink farm because of this though.
And to be honest I think the motivation for this is supporting
multiple python versions without having to duplicate all .py files
rather then the FHS.

But the great thing about the proposal is that it doesn't even matter.
Files will be easily tagged/detected as "python modules" and "python
extension modules" and the tools consuming this information can decide
to do what they want with them.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Sun Feb  1 13:11:59 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sun, 1 Feb 2009 12:11:59 +0000
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49852FED.2040608@ar.media.kyoto-u.ac.jp>
References: <877i4dvl02.fsf@benfinney.id.au> <498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<871vuj562z.fsf@benfinney.id.au>
	<49852FED.2040608@ar.media.kyoto-u.ac.jp>
Message-ID: <20090201121159.GB3987@laurie.devork>

On Sun, Feb 01, 2009 at 02:15:25PM +0900, David Cournapeau wrote:
> The arch independent/dependent is much clearer IMHO, and is the
> vocabulary used by the FHS, AFAIK. It is certainly the one used by the
> configure script help:
> 
>   --libdir=DIR           object code libraries [EPREFIX/lib]
>   --datarootdir=DIR      read-only arch.-independent data root
> [PREFIX/share]
>   --datadir=DIR          read-only architecture-independent data
> [DATAROOTDIR]

The FHS, just like autoconf, does say that datadir must be
arch-independent.  But AFAIK they never say that libdir is not allowed
to be architecture independent.

But I'm suspecting this is very much off-topic by now.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From david at ar.media.kyoto-u.ac.jp  Sun Feb  1 13:08:33 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sun, 01 Feb 2009 21:08:33 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <20090201115740.GA3987@laurie.devork>
References: <498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<20090131153556.407903A4114@sparrow.telecommunity.com>	<20090131165847.GA13200@laurie.devork>	<498533D6.9020005@ar.media.kyoto-u.ac.jp>
	<20090201115740.GA3987@laurie.devork>
Message-ID: <498590C1.7090704@ar.media.kyoto-u.ac.jp>

Floris Bruynooghe wrote:
>
> Hmm fair enough, I must have missed that last time I looked at the
> implementation of python-support and python-central.  But both take
> the burden of having to create a symlink farm because of this though.
> And to be honest I think the motivation for this is supporting
> multiple python versions without having to duplicate all .py files
> rather then the FHS.
>   

Yes, multiple-versions certainly motivated the split as well. I think
Fedora Core for example does not support multiple python versions (I
can't find a reference ATM), but I am not a user of rpm-based
distributions, so don't really know much about it and how they do it.

> But the great thing about the proposal is that it doesn't even matter.
> Files will be easily tagged/detected as "python modules" and "python
> extension modules" and the tools consuming this information can decide
> to do what they want with them.
>   

exactly. FHS is just one example - I suggested following autoconf
categories because that's the maximum flexibility, and can be used for
many cases, since it is a defacto standard (on Unix at least - but I
don't think we need to change much how things are done for Mac OS X and
Windows).

cheers,

David

From zooko at zooko.com  Sun Feb  1 15:43:49 2009
From: zooko at zooko.com (zooko)
Date: Sun, 1 Feb 2009 07:43:49 -0700
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <49853E86.9080404@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>	<E64C99AC-4E00-4385-83D4-D634F01AB8DB@zooko.com>
	<319e029f0901312226t109110d9gb191aa01e3257528@mail.gmail.com>
	<49853E86.9080404@ar.media.kyoto-u.ac.jp>
Message-ID: <68A9BE4A-DDC5-4FA0-AF7D-DD9D5D7D1C63@zooko.com>

On Jan 31, 2009, at 23:17 PM, David Cournapeau wrote:

> There are two problems with stow and setuptools:
>     - setuptools refuses to install in a directory not in  
> PYTHONPATH, so python setup.py install --prefix=/usr/local/stow/my- 
> package does not work when setup.py uses setuptools. You have to  
> use --single-version-externally-managed (plus another option  
> required by this one). That's the most annoying point, since it  
> depends on whether setup.py uses setuptools or plain distutils.  
> From a user POV, setuptools broke standard python behavior.
>     - namespace packages: the fact that namespace/__init__ are  
> 'owned' by several packages is a fundamental problem (not only for  
> stow, but almost any package system). But unless python itself  
> supports namespace packages, I don't see any solution to this one.

also:

     - easy-install.pth; When you install two different packages  
under GNU stow, then stow can install both of those packages (by  
creating symlinks pointing to each file from each package) *unless*  
both packages created a file in the same location.  Two different  
Python packages both installed by setuptools (without --single- 
version-externally-managed) will create a file in the same location  
-- the easy-install.pth file.  If setuptools didn't need to create  
this file (and if it accepted installation into a directory not  
currently on the PYTHONPATH, per http://bugs.python.org/setuptools/ 
issue54 (be more like distutils with regard to --prefix=)), then it  
would be GNU stow-compatible.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From regebro at gmail.com  Sun Feb  1 19:40:19 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 1 Feb 2009 19:40:19 +0100
Subject: [Distutils] Setuptools for Python 3 port ready for testing.
Message-ID: <319e029f0902011040m663c647bn60d08a83c1bc690f@mail.gmail.com>

I've uploaded an sdist of setuptools ported to Python 3 to here:
http://code.google.com/p/python-incompatibility/downloads/list

A little more info on my blog:
http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-python-3/

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From jim at zope.com  Sun Feb  1 19:55:25 2009
From: jim at zope.com (Jim Fulton)
Date: Sun, 1 Feb 2009 13:55:25 -0500
Subject: [Distutils] Setuptools for Python 3 port ready for testing.
In-Reply-To: <319e029f0902011040m663c647bn60d08a83c1bc690f@mail.gmail.com>
References: <319e029f0902011040m663c647bn60d08a83c1bc690f@mail.gmail.com>
Message-ID: <58877397-4EB0-4AD3-9310-E03C45430107@zope.com>

That's awesome.  Thanks Lennart!

Jim

On Feb 1, 2009, at 1:40 PM, Lennart Regebro wrote:

> I've uploaded an sdist of setuptools ported to Python 3 to here:
> http://code.google.com/p/python-incompatibility/downloads/list
>
> A little more info on my blog:
> http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-python-3/
>
> -- 
> Lennart Regebro: Zope and Plone consulting.
> http://www.colliberty.com/
> +33 661 58 14 64
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Jim Fulton
Zope Corporation



From ianb at colorstudy.com  Sun Feb  1 22:29:52 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 1 Feb 2009 15:29:52 -0600
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49840E8C.1090302@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
Message-ID: <b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>

On Sat, Jan 31, 2009 at 2:40 AM, David Cournapeau <
david at ar.media.kyoto-u.ac.jp> wrote:

> Ian Bicking wrote:
> > On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
> > <floris.bruynooghe at gmail.com <mailto:floris.bruynooghe at gmail.com>>
> wrote:
> >
> >     I imagine things like libdir, prefix, datadir, docdir and other
> things
> >     copied from autoconf.  Where the defaults would be something like:
> >
> >     prefix = sys.prefix
> >     libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> >     datadir = sys.prefix/share/mypackage
> >     docdir = sys.prefix/share/doc/mypackage
> >
> >
> > I wouldn't want to use those.  What goes in libdir, what goes in
> > datadir?  I don't know, and frankly the distinctions start getting
> > really arbitrary.
>
> They are not arbitrary - they come from standard usage and have a
> rationale, at least on Unix (datadir for arch independent, and libdir
> for arch dependent, to simplify).
>

I'm just about ready to run screaming from this discussion... so no, I want
no part of determining what the "right" place for these files is, nor do I
find it self-evident.

But you mostly do not need to care, as a developer: .py files would be
> considered as data files, extensions as arch-dependent, etc... The main
> category which needs special care is documentation, and I think I am not
> the only one thinking that's one thing missing in distutils ATM.
>
> >
> > I would rather see something like pkg_resources existing API, where
> > there is some file that maps out how the local names of files (where
> > they'd be in a checkout) map to their installed location, then the
> > pkg_resources code could finds the real location of the file.
>
> I am not sure I understand how this would help OS packagers - this does
> not sound as the same problem at all.


Sorry, I didn't describe what I meant.

I imagine some file like package-data.conf, containing:

data mypackage/templates/
docs docs/_build/

At least in this example the first word is some tag, and the second is the
directory, or files, or maybe a wildcard or something determining what files
that tag applies to.  Everything not declared but present in a package (or
as a module) would default to "lib", and everything outside of that (like
the setup.py file) would be "ignore".

On installation, you'd write something like
mypackage.egg-info/file-locations.txt, that might look like:

mypackage/templates/ -> /usr/share/mypackage/templates/
docs/_build/ -> /usr/share/doc/mypackage

(I'm not sure what the syntax would look like, but whatever.)  Then when I
did something like pkg_resources.resource_string('mypackage',
'mypackage/templates') it'd look up this file to find the location (in the
absence of the file it'd act like it does currently)

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090201/52bee68f/attachment-0001.htm>

From ben+python at benfinney.id.au  Sun Feb  1 23:26:52 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 02 Feb 2009 09:26:52 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
Message-ID: <87tz7d3iab.fsf@benfinney.id.au>

Ian Bicking <ianb at colorstudy.com> writes:

> I'm just about ready to run screaming from this discussion... so no,
> I want no part of determining what the "right" place for these files
> is

As I understand it, you are speaking from the perspective of a
?developer?; that is, someone who intends to share the software work
you've written, but leave the packaging of that work for particular
OSen to other people.

In that case, you're *not* being asked to determine what the ?right?
installation location of files is; that's exactly the shift in control
that's being requested here, by divorcing the decision about what the
*purpose* of a file is from what its installed *location* will be.

Instead, you're being asked only to determine which category a file
belongs to; and those categories are (modulo some historical quirks,
due to David's obvious strong preference for the C-language-focussed
?autoconf?) mostly about the *purpose* of a file ? precisely to allow
that determination to be made without the same person needing to know
about install locations.

> I imagine some file like package-data.conf, containing:
> 
> data mypackage/templates/
> docs docs/_build/
> 
> At least in this example the first word is some tag, and the second
> is the directory, or files, or maybe a wildcard or something
> determining what files that tag applies to. Everything not declared
> but present in a package (or as a module) would default to "lib",
> and everything outside of that (like the setup.py file) would be
> "ignore".

This is (with implementation differences) entirely compatible with
what David's proposing, AFAICT.

The message you responded to, though, was discussing the complementary
part: the install location of (in your example) ?docs?, ?data?, ?lib?,
etc. If you want no part of that, you're welcome to avoid it; and
that's exactly the kind of partition that is being striven for here.

> On installation, you'd write something like
> mypackage.egg-info/file-locations.txt, that might look like:
> 
> mypackage/templates/ -> /usr/share/mypackage/templates/
> docs/_build/ -> /usr/share/doc/mypackage
> 
> (I'm not sure what the syntax would look like, but whatever.)

Here, though, you lose the very useful categorisation you set up
above. AIUI, David is proposing that install location be a mapping not
from file path to file path, but from file *purpose* (as categorised
above) to install file path.

-- 
 \     ?For every complex problem, there is a solution that is simple, |
  `\                               neat, and wrong.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From david at ar.media.kyoto-u.ac.jp  Mon Feb  2 04:31:18 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 02 Feb 2009 12:31:18 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>	
	<498232A0.8060306@palladion.com>	
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	
	<20090129233428.GA10050@laurie.devork>	
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	
	<20090130183951.GA5119@laurie.devork>	
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
Message-ID: <49866906.8000601@ar.media.kyoto-u.ac.jp>

Ian Bicking wrote:
> On Sat, Jan 31, 2009 at 2:40 AM, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp <mailto:david at ar.media.kyoto-u.ac.jp>>
> wrote:
>
>     Ian Bicking wrote:
>     > On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
>     > <floris.bruynooghe at gmail.com
>     <mailto:floris.bruynooghe at gmail.com>
>     <mailto:floris.bruynooghe at gmail.com
>     <mailto:floris.bruynooghe at gmail.com>>> wrote:
>     >
>     >     I imagine things like libdir, prefix, datadir, docdir and
>     other things
>     >     copied from autoconf.  Where the defaults would be something
>     like:
>     >
>     >     prefix = sys.prefix
>     >     libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>     >     datadir = sys.prefix/share/mypackage
>     >     docdir = sys.prefix/share/doc/mypackage
>     >
>     >
>     > I wouldn't want to use those.  What goes in libdir, what goes in
>     > datadir?  I don't know, and frankly the distinctions start getting
>     > really arbitrary.
>
>     They are not arbitrary - they come from standard usage and have a
>     rationale, at least on Unix (datadir for arch independent, and libdir
>     for arch dependent, to simplify).
>
>
> I'm just about ready to run screaming from this discussion... so no, I
> want no part of determining what the "right" place for these files is,
> nor do I find it self-evident.

I never said it was self-evident. And about the part of determining what
the right place is: the whole point is that you, as a developer, won't
have to care :) Only by changing the distutils implementation and adding
some options to the install command, but without changing *anything* in
the setup.py, would many if not most python softwares be easier to
install along the FHS. And where something is missing, the *packager*
could send a patch to fix some metadata upstream, instead of keeping
their changes separately.

If you don't have any experience with autotools, it is a bit hard to go
into details, but you almost never care about what goes where for the
most part. You just tag some files as programs, some as docs, etc...
Most of it being implicit.

>
> Sorry, I didn't describe what I meant.
>
> I imagine some file like package-data.conf, containing:
>
> data mypackage/templates/
> docs docs/_build/
>
> At least in this example the first word is some tag, and the second is
> the directory, or files, or maybe a wildcard or something determining
> what files that tag applies to.  Everything not declared but present
> in a package (or as a module) would default to "lib", and everything
> outside of that (like the setup.py file) would be "ignore".

I think this does not have to be as complicated. We already need to tag
some files in the setup.py file, so we could do:

from distutils.core import setup, Extension

setup(name='example',
        version='1.0',
        description='Example',
        author='You',
        author_email='me at python.net',
        packages=['example', 'example.foo'],
        ext_modules=[Extension('example.foo._bar', ['example/foo/bar.c'])],
        pdfdoc=['doc/doc.pdf'],
        htmldoc=['doc/html'],
        pacakge_data=['example/data/data1.dat', 'example/data/data2.dat']
        )

Note that except pdfdoc and htmldoc, all this is already there in
distutils. Then, you could do

python setup.py install --prefix=tmp

ending up in (on Linux):

tmp/lib/python-$PYVER/site-packages/example/__init__.py
...
tmp/lib/python-$PYVER/site-packages/example/foo/bar/_bar.so
tmp/lib/python-$PYVER/site-packages/example/data/data1.dat
...
tmp/lib/python-$PYVER/site-packages/example/doc/doc.pdf
...

As is currently done. But you could also do:

python setup.py install --libdir=tmp/lib/ --datadir=tmp/share/
--pdfdoc=tmp/share/doc

which would result in:

tmp/share/example/__init__.py
...
tmp/lib/python-$PYVER/site-packages/example/foo/bar/_bar.so
tmp/share/example/data/data1.dat
...
tmp/share/doc/example/doc.pdf
...

I did not try to follow any kind of Debian policy - the point is that
you can decide how to do the mapping category -> location at install
time. And if a mapping is missing/buggy, the packager can fix it, and
send you a patch. You, as the upstream developer, would have nothing to
do - actually, I certainly expect most people not to care at all about
all this, and they will just have to wait for patchs to come in (that's
the optimistic view :) ).

cheers,

David

From david at ar.media.kyoto-u.ac.jp  Mon Feb  2 04:38:20 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 02 Feb 2009 12:38:20 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <87tz7d3iab.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
	<87tz7d3iab.fsf@benfinney.id.au>
Message-ID: <49866AAC.3070202@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
> Instead, you're being asked only to determine which category a file
> belongs to; and those categories are (modulo some historical quirks,
> due to David's obvious strong preference for the C-language-focussed
> ?autoconf?) mostly about the *purpose* of a file ? precisely to allow
> that determination to be made without the same person needing to know
> about install locations.
>   

Just to be clear: I don't have a strong preference for autoconf, and do
not intend to push for it - it is just that it is a de-facto standard,
and much more flexible than distutils in that aspect. It does not mean
we have to do like autoconf - but if we don't, it should be justified,
instead of implementing again something from scratch which does not work.

NIH syndrome and all that, if you want :)

cheers,

David

From david.lyon at preisshare.net  Mon Feb  2 04:57:22 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sun, 01 Feb 2009 22:57:22 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49866906.8000601@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
	<49866906.8000601@ar.media.kyoto-u.ac.jp>
Message-ID: <11b64eca4c74faed1fc3f027f392d20c@preisshare.net>

Hi David,

I am learning a lot from this list....

On Mon, 02 Feb 2009 12:31:18 +0900, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> >>     >     prefix = sys.prefix
>>     >     libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>>     >     datadir = sys.prefix/share/mypackage
>>     >     docdir = sys.prefix/share/doc/mypackage
..
> I never said it was self-evident. And about the part of determining what
> the right place is: the whole point is that you, as a developer, won't
> have to care :) 

As a python developer... I think you do.... :-)

First thing you want to go looking for is the documentation.

omg - that can be impossible to find for most packages.

Lucky I was trained in the school of 'if you don't understand
the program - go debug the code'....

Sorry to say this, but I didn't know packages even included
documentation. It is d&*med difficult to find in linux.

> I think this does not have to be as complicated. We already need to tag
> some files in the setup.py file, so we could do:
> 
> from distutils.core import setup, Extension
> 
> setup(name='example',
>         version='1.0',
>         description='Example',
>         author='You',
>         author_email='me at python.net',
>         packages=['example', 'example.foo'],
>         ext_modules=[Extension('example.foo._bar',
> ['example/foo/bar.c'])],
>         pdfdoc=['doc/doc.pdf'],
>         htmldoc=['doc/html'],
>         pacakge_data=['example/data/data1.dat', 'example/data/data2.dat']
>         )
> 
> Note that except pdfdoc and htmldoc, all this is already there in
> distutils. Then, you could do
> 
> python setup.py install --prefix=tmp
> 
> ending up in (on Linux):
> 
> tmp/lib/python-$PYVER/site-packages/example/__init__.py
> ...
> tmp/lib/python-$PYVER/site-packages/example/foo/bar/_bar.so
> tmp/lib/python-$PYVER/site-packages/example/data/data1.dat
> ...
> tmp/lib/python-$PYVER/site-packages/example/doc/doc.pdf

Very interesting...

Regards

David





From david.lyon at preisshare.net  Mon Feb  2 05:12:10 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sun, 01 Feb 2009 23:12:10 -0500
Subject: [Distutils] Python package Management GUI - New Project on
	Sourceforge
In-Reply-To: <7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
Message-ID: <402ded795e428be151d6e0d19d96f359@preisshare.net>

Hi all,

I am pleased to announce that we have started a new python project
on sourceforge.

   Python Package Manager
   pythonpkgmgr.sourceforge.net

The goal is to provide a cross platform GUI tool that will vastly 
simplify loading and installing packages under python.

 - written in python
 - use WXWidgets for cross compatability
 - utilises distutils
 - provide a GUI wrapper for EasyInstall and pip
 - fetches packages from http://pypi.python.org/pypi using
   their XML-RPC interface.

Feel free to apply to join the project and help us build
the solution that we all need and deserve.

Regards

David Lyon


From ben+python at benfinney.id.au  Mon Feb  2 05:48:48 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 02 Feb 2009 15:48:48 +1100
Subject: [Distutils] Python package Management GUI - New Project on
	Sourceforge
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<402ded795e428be151d6e0d19d96f359@preisshare.net>
Message-ID: <87iqnt1m1b.fsf@benfinney.id.au>

David Lyon <david.lyon at preisshare.net> writes:

> The goal is to provide a cross platform GUI tool that will vastly
> simplify loading and installing packages under python.

Thanks for the announcement.
> 
>  - written in python
>  - use WXWidgets for cross compatability
>  - utilises distutils
>  - provide a GUI wrapper for EasyInstall and pip

This all sounds good.

>  - fetches packages from http://pypi.python.org/pypi using
>    their XML-RPC interface.

Surely this is something that should be done by the package management
library, making it available in one place for each front-end tool; not
implemented in a specific front-end tool?

-- 
 \     ?Nature is trying very hard to make us succeed, but nature does |
  `\       not depend on us. We are not the only experiment.? ?Richard |
_o__)                                   Buckminster Fuller, 1978-04-30 |
Ben Finney


From p.f.moore at gmail.com  Mon Feb  2 09:13:44 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Feb 2009 08:13:44 +0000
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49866906.8000601@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
	<49866906.8000601@ar.media.kyoto-u.ac.jp>
Message-ID: <79990c6b0902020013l316b9df5n6293e71419fb40c9@mail.gmail.com>

2009/2/2 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> I never said it was self-evident. And about the part of determining what
> the right place is: the whole point is that you, as a developer, won't
> have to care :) Only by changing the distutils implementation and adding
> some options to the install command, but without changing *anything* in
> the setup.py, would many if not most python softwares be easier to
> install along the FHS. And where something is missing, the *packager*
> could send a patch to fix some metadata upstream, instead of keeping
> their changes separately.
>
> If you don't have any experience with autotools, it is a bit hard to go
> into details, but you almost never care about what goes where for the
> most part. You just tag some files as programs, some as docs, etc...
> Most of it being implicit.

I suspect that people involved in this discussion don't really
appreciate just how confusing these fine points can be, particularly
for Windows developers who have no concept of the differences (in
terms of a filesystem hierarchy that expresses it). I know that, in
some sense it's "obvious" that a file is documentation or code - but
what about a README? It may be "documentation", but it "obviously", to
me as a Windos developer, wants to go "with the code", not "with the
documentation". And what about sample code? Code or documentation?

As Ian points out, you are asking people to care about something that,
at the moment, we don't have to. "Easy" or not.

Paul.

From joss at debian.org  Mon Feb  2 11:56:08 2009
From: joss at debian.org (Josselin Mouette)
Date: Mon, 02 Feb 2009 11:56:08 +0100
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <20090131141028.GA8616@laurie.devork>
References: <b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<20090130230236.GD5606@laurie.devork>
	<20090131141028.GA8616@laurie.devork>
Message-ID: <1233572168.18512.15.camel@tomoyo>

Le samedi 31 janvier 2009 ? 14:10 +0000, Floris Bruynooghe a ?crit :
> On Fri, Jan 30, 2009 at 11:02:36PM +0000, Floris Bruynooghe wrote:
> > Ok, that was supposed to read:
> > 
> > prefix = sys.prefix
> > libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> > datadir = sys.prefix/share/mypackage
> > docdir = sys.prefix/share/doc/mypackage
> 
> Just how many time will I get this wrong??
> 
> libdir = sys.prefix/lib/pythonX.Y/site-packages
> datadir = sys.prefix/share/<project_name>
> docdir = sys.prefix/share/doc/<project_name>

Please don?t add more confusion by adding a libdir meaning something
different from the existing autoconf macros.

prefix = sys.prefix
libdir = prefix+"/lib"  # not sys.prefix; redefining prefix must lead to redefining the others
pkglibdir = libdir+"/"+pkgname
datadir = prefix+"/share"
pkgdatadir = datadir+"/"+pkgname
pythondir = libdir+"pythonX.Y/site-packages"
pkgpythondir = pythondir+modulename
...

(And you can add the prefix / exec_prefix distinction for systems that
require it.)

-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090202/b99cfc92/attachment.pgp>

From setuptools at bugs.python.org  Mon Feb  2 17:26:54 2009
From: setuptools at bugs.python.org (Andre Nathan)
Date: Mon, 02 Feb 2009 16:26:54 +0000
Subject: [Distutils] [issue58] find_on_path on systems with tight permissions
In-Reply-To: <1233592014.79.0.4268964786.issue58@psf.upfronthosting.co.za>
Message-ID: <1233592014.79.0.4268964786.issue58@psf.upfronthosting.co.za>


New submission from Andre Nathan <andrenth at gmail.com>:

Hello

I'm using setuptools on a server with tight permissions; a normal user is not
allowed to list the contents of certain directories, including /usr/bin, which
setuptools tries to scan when searching for .egg files. Thus, running any
command installed from setuptools fails with

  OSError: [Errno 13] Permission denied: '/usr/bin'

The following change in find_on_path(), in pkg_resources.py, makes it ignore
directories which it cannot read:

  -if os.path.isdir(path_item):
  +if os.path.isdir(path_item) and os.access(path_item, os.R_OK):

Do you think this could be applied?

Thanks,
Andre

----------
messages: 233
nosy: andrenth
priority: bug
status: unread
title: find_on_path on systems with tight permissions

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue58>
_______________________________________________

From dpeterson at enthought.com  Mon Feb  2 19:29:50 2009
From: dpeterson at enthought.com (Dave Peterson)
Date: Mon, 02 Feb 2009 12:29:50 -0600
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
 explicitly with -L when building extensions?
In-Reply-To: <a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>
References: <497E59C6.4020802@enthought.com>	
	<20090130230849.GE5606@laurie.devork>
	<49839116.2050801@enthought.com>	
	<20090131140143.GA7382@laurie.devork>
	<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>
Message-ID: <49873B9E.5060002@enthought.com>

Tarek Ziade wrote:
> 2009/1/31 Floris Bruynooghe <floris.bruynooghe at gmail.com>:
>   
>>> I'm leaning more and more toward this is actually a bug
>>> with the distutils source on Solaris.
>>>       
>> Yes, I agree now that it is a bug in distutils and I agree with your
>> fix.  The if statement should check both that it is SunOS and that it
>> is using a shared python, just like the linux one.  If fact the linux
>> one could just be modified:
>>
>> if (sys.paltform.startswith('linux') \
>>    or sys.platform.startswith('gnu') \
>>    or sys.platform.startswith('sunos')) \
>>        and sysconfig.get_config_var('Py_ENABLE_SHARED'):
>>    ...
>>
>> I'll leave the honours of reporting the bug to you if you agree with
>> this.
>>     
>
> Yes thanks to report this, I'll apply your fix,
>
> Regards
> Tarek
>   

Hi Tarek,

It isn't clear to me if you fixed it already or were waiting for my 
patch, but I just added issue 5132 to the Python issue tracker to 
document the issue and provide a patch file.

-- Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090202/5172c985/attachment.htm>

From ziade.tarek at gmail.com  Mon Feb  2 19:56:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 2 Feb 2009 19:56:24 +0100
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
	explicitly with -L when building extensions?
In-Reply-To: <49873B9E.5060002@enthought.com>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork> <49839116.2050801@enthought.com>
	<20090131140143.GA7382@laurie.devork>
	<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>
	<49873B9E.5060002@enthought.com>
Message-ID: <94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>

On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson at enthought.com> wrote:
>
> Hi Tarek,
>
> It isn't clear to me if you fixed it already or were waiting for my patch,
> but I just added issue 5132 to the Python issue tracker to document the
> issue and provide a patch file.

I was waiting for that, thanks :)

Tarek
>
> -- Dave
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From dpeterson at enthought.com  Mon Feb  2 20:01:18 2009
From: dpeterson at enthought.com (Dave Peterson)
Date: Mon, 02 Feb 2009 13:01:18 -0600
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
 explicitly with -L when building extensions?
In-Reply-To: <94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>
References: <497E59C6.4020802@enthought.com>	
	<20090130230849.GE5606@laurie.devork>
	<49839116.2050801@enthought.com>	
	<20090131140143.GA7382@laurie.devork>	
	<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>	
	<49873B9E.5060002@enthought.com>
	<94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>
Message-ID: <498742FE.6060105@enthought.com>

Tarek Ziad? wrote:
> On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson at enthought.com> wrote:
>   
>> Hi Tarek,
>>
>> It isn't clear to me if you fixed it already or were waiting for my patch,
>> but I just added issue 5132 to the Python issue tracker to document the
>> issue and provide a patch file.
>>     
>
> I was waiting for that, thanks :)
>
> Tarek
>   

I noticed you removed the Python 2.5 version specification.  I wasn't 
aware there were no further planned maintenance releases for the 2.5 
branch.   A few of the libs we use still seem to have problems building 
on 2.6.   Guess it is just a matter of time.

-- Dave


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090202/a6d9ac53/attachment.htm>

From ziade.tarek at gmail.com  Mon Feb  2 20:04:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 2 Feb 2009 20:04:50 +0100
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
	explicitly with -L when building extensions?
In-Reply-To: <498742FE.6060105@enthought.com>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork> <49839116.2050801@enthought.com>
	<20090131140143.GA7382@laurie.devork>
	<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>
	<49873B9E.5060002@enthought.com>
	<94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>
	<498742FE.6060105@enthought.com>
Message-ID: <94bdd2610902021104m1101fae8y743086fd0e9dc905@mail.gmail.com>

On Mon, Feb 2, 2009 at 8:01 PM, Dave Peterson <dpeterson at enthought.com> wrote:
> Tarek Ziad? wrote:
>
> On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson at enthought.com>
> wrote:
>
>
> Hi Tarek,
>
> It isn't clear to me if you fixed it already or were waiting for my patch,
> but I just added issue 5132 to the Python issue tracker to document the
> issue and provide a patch file.
>
>
> I was waiting for that, thanks :)
>
> Tarek
>
>
> I noticed you removed the Python 2.5 version specification.  I wasn't aware
> there were no further planned maintenance releases for the 2.5 branch.   A
> few of the libs we use still seem to have problems building on 2.6.   Guess
> it is just a matter of time.

Yeah, 2.5 is only security bugfixes now. So it will be available in
the next 2.6.x  I guess


Regards
Tarek

>
> -- Dave
>
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From zooko at zooko.com  Mon Feb  2 22:18:18 2009
From: zooko at zooko.com (zooko)
Date: Mon, 2 Feb 2009 14:18:18 -0700
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
	explicitly with -L when building extensions?
In-Reply-To: <498742FE.6060105@enthought.com>
References: <497E59C6.4020802@enthought.com>	
	<20090130230849.GE5606@laurie.devork>
	<49839116.2050801@enthought.com>	
	<20090131140143.GA7382@laurie.devork>	
	<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>	
	<49873B9E.5060002@enthought.com>
	<94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>
	<498742FE.6060105@enthought.com>
Message-ID: <AE636966-4E6B-43E9-A078-74EAFC7012E8@zooko.com>


On Feb 2, 2009, at 12:01 PM, Dave Peterson wrote:
>
> I noticed you removed the Python 2.5 version specification.  I  
> wasn't aware there were no further planned maintenance releases for  
> the 2.5 branch.   A few of the libs we use still seem to have  
> problems building on 2.6.   Guess it is just a matter of time.

I maintain a few libraries with compiled C code.  I don't know how to  
build binaries for Python 2.6, because apparently the method that  
I've been using until now to build binary extension modules (mingw)  
isn't able to produce 2.6-compatible extension modules yet.

For now, I'm just not distributing binaries for Python 2.6 (nor, of  
course, doing a proper job of testing with Python 2.6).  I've been  
kind of hoping that either mingw would catch up or that all my users  
would give up on Python >= 2.6 and go back to using Python 2.5, but  
perhaps I should start making new plans...

Meanwhile I'm watching with interest this ticket: http:// 
bugs.python.org/issue3871

Maybe in the future I can just offer my users a Python executable  
which is compatible with the Free Software tools that I know and use,  
instead of figuring out how to make modules which are compatible with  
the proprietary compiler used to build the official Python 2.6!

Regards,

Zooko

P.S.  Whoops, sorry.  This message was total flame-bait and off-topic  
to boot.  But it has been bugging me, and I feel better for having  
gotten it off my chest.

From floris.bruynooghe at gmail.com  Mon Feb  2 22:30:18 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 2 Feb 2009 21:30:18 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
References: <1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
Message-ID: <20090202213018.GA5818@laurie.devork>

On Sun, Feb 01, 2009 at 03:29:52PM -0600, Ian Bicking wrote:
> I'm just about ready to run screaming from this discussion...

Thanks for not doing so

> Sorry, I didn't describe what I meant.
> 
> I imagine some file like package-data.conf, containing:
> 
> data mypackage/templates/
> docs docs/_build/
> 
> At least in this example the first word is some tag, and the second is the
> directory, or files, or maybe a wildcard or something determining what files
> that tag applies to.  Everything not declared but present in a package (or
> as a module) would default to "lib", and everything outside of that (like
> the setup.py file) would be "ignore".
> 
> On installation, you'd write something like
> mypackage.egg-info/file-locations.txt, that might look like:
> 
> mypackage/templates/ -> /usr/share/mypackage/templates/
> docs/_build/ -> /usr/share/doc/mypackage
> 
> (I'm not sure what the syntax would look like, but whatever.)  Then when I
> did something like pkg_resources.resource_string('mypackage',
> 'mypackage/templates') it'd look up this file to find the location (in the
> absence of the file it'd act like it does currently)

I really like this idea.  It seems to solve most of the issues.  For
some reason my brain kept thinking that categorising data by it's
purpose is a really natural thing to do, but that seems not so.

With this solution both packagers and developers can be happy and the
required patch to be added by a packager is really simple (add a file
with the tags).  It is even entirely possible to provide two APIs with
this system, the one you just described and one for people like me
who'd rather say get_resource('mypackage', 'datafile',
'templates/foo.xml').


Cheers
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From david at ar.media.kyoto-u.ac.jp  Tue Feb  3 06:44:54 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 03 Feb 2009 14:44:54 +0900
Subject: [Distutils] Solaris and distutils: Need to pass
 LIBDIR	explicitly with -L when building extensions?
In-Reply-To: <AE636966-4E6B-43E9-A078-74EAFC7012E8@zooko.com>
References: <497E59C6.4020802@enthought.com>		<20090130230849.GE5606@laurie.devork>	<49839116.2050801@enthought.com>		<20090131140143.GA7382@laurie.devork>		<a26746990902010212i7dacea9br4274e179fcafb346@mail.gmail.com>		<49873B9E.5060002@enthought.com>	<94bdd2610902021056y759da481x9def42458deba45f@mail.gmail.com>	<498742FE.6060105@enthought.com>
	<AE636966-4E6B-43E9-A078-74EAFC7012E8@zooko.com>
Message-ID: <4987D9D6.2090309@ar.media.kyoto-u.ac.jp>

zooko wrote:
>
> On Feb 2, 2009, at 12:01 PM, Dave Peterson wrote:
>>
>> I noticed you removed the Python 2.5 version specification.  I wasn't
>> aware there were no further planned maintenance releases for the 2.5
>> branch.   A few of the libs we use still seem to have problems
>> building on 2.6.   Guess it is just a matter of time.
>
> I maintain a few libraries with compiled C code.  I don't know how to
> build binaries for Python 2.6, because apparently the method that I've
> been using until now to build binary extension modules (mingw) isn't
> able to produce 2.6-compatible extension modules yet.

Building python extensions with mingw on 2.6 is possible, at least on 32
bits arch. We can build numpy with mingw on 2.6; given numpy's reliance
on a lot of distutils features/C code, I think most python softwares
should build fine.

cheers,

David


From david at ar.media.kyoto-u.ac.jp  Tue Feb  3 06:51:42 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 03 Feb 2009 14:51:42 +0900
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <20090202213018.GA5818@laurie.devork>
References: <1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
	<20090202213018.GA5818@laurie.devork>
Message-ID: <4987DB6E.10708@ar.media.kyoto-u.ac.jp>

Floris Bruynooghe wrote:
>
> I really like this idea.  It seems to solve most of the issues.  For
> some reason my brain kept thinking that categorising data by it's
> purpose is a really natural thing to do, but that seems not so.
>   

In this proposal, I don't understand why tagging data/doc/etc...  in a
separate file is easier than tagging it directly in setup.py ? Maybe I
am missing something important, but why requiring a different mechanism
than the existing one in setup, so that we end up with two mechanisms
(data_dir/data_files from setup, and tagging in a separate file) ?

cheers,

David

From david at ar.media.kyoto-u.ac.jp  Tue Feb  3 07:04:59 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 03 Feb 2009 15:04:59 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <87tz7d3iab.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>	<b654cd2e0902011329k322afab3p616f19ab622532ce@mail.gmail.com>
	<87tz7d3iab.fsf@benfinney.id.au>
Message-ID: <4987DE8B.6010800@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
> Instead, you're being asked only to determine which category a file
> belongs to; and those categories are (modulo some historical quirks,
> due to David's obvious strong preference for the C-language-focussed
> ?autoconf?) mostly about the *purpose* of a file ? precisely to allow
> that determination to be made without the same person needing to know
> about install locations.
>   

I've just found out that cabal ("distutils for haskell") supports most
of those options, BTW:

http://www.haskell.org/cabal/release/latest/doc/users-guide/builders.html#setup-configure-paths

And haskell is not really a C-language-focused language :) I don't know
anything about haskell, but it may be interesting to look at how they
are doing things,

David

From teemu.harju at gmail.com  Tue Feb  3 11:34:09 2009
From: teemu.harju at gmail.com (Teemu Harju)
Date: Tue, 3 Feb 2009 12:34:09 +0200
Subject: [Distutils] zc.buildout newbie question
Message-ID: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>

Hi all,

I've been looking into zc.buildout since it seems to be quite useful
application when developing and distributing Python applications. However,
I'm finding it difficult to figure out how it should actually be used. I
give you an example of what I'm trying to do and what my problem is. This is
probably really trivial but bear with me. ;-)

I'm developing a Twisted application and would like to use zc.buildout for
bootstrapping development environments and probably even deploying the
application to the production system.

Here's my very simple buildout.cfg:
---
[buildout]
parts = twisted

[twisted]
recipe = minitage.recipe:du
url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/Twisted-8.2.0.tar.bz2
---

This works fine and installs Twisted under parts/site-packages-2.5/ and
twistd, trial and other scripts under bin/. I'm doing all this inside
virtualenv, but I guess it shouldn't matter.

Now, my problem is that since the packages are installed in parts/ so they
are not visible to my python interpreter. What is the right way to do this?
Should I just add /parts/site-packages-2.5 to my PYTHONPATH or? Is there
some way to do that using buildout.cfg? I tried to find examples of this but
didn't find any.

Br,

- Teemu

-- 
Teemu Harju

email/jabber: teemu.harju at gmail.com
blog: http://blog.teemu.im
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090203/1347249d/attachment.htm>

From lists at zopyx.com  Tue Feb  3 11:43:43 2009
From: lists at zopyx.com (Andreas Jung)
Date: Tue, 03 Feb 2009 11:43:43 +0100
Subject: [Distutils] zc.buildout newbie question
In-Reply-To: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
References: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
Message-ID: <49881FDF.2050205@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Zope buildouts usually contain a part 'zopepy' generating
a Python interpreter under bin:

[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy


- -aj

On 03.02.2009 11:34 Uhr, Teemu Harju wrote:
> Hi all,
> 
> I've been looking into zc.buildout since it seems to be quite useful
> application when developing and distributing Python applications.
> However, I'm finding it difficult to figure out how it should actually
> be used. I give you an example of what I'm trying to do and what my
> problem is. This is probably really trivial but bear with me. ;-)
> 
> I'm developing a Twisted application and would like to use zc.buildout
> for bootstrapping development environments and probably even deploying
> the application to the production system.
> 
> Here's my very simple buildout.cfg:
> ---
> [buildout]
> parts = twisted
> 
> [twisted]
> recipe = minitage.recipe:du
> url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/Twisted-8.2.0.tar.bz2
> ---
> 
> This works fine and installs Twisted under parts/site-packages-2.5/ and
> twistd, trial and other scripts under bin/. I'm doing all this inside
> virtualenv, but I guess it shouldn't matter.
> 
> Now, my problem is that since the packages are installed in parts/ so
> they are not visible to my python interpreter. What is the right way to
> do this? Should I just add /parts/site-packages-2.5 to my PYTHONPATH or?
> Is there some way to do that using buildout.cfg? I tried to find
> examples of this but didn't find any.
> 
> Br,
> 
> - Teemu
> 
> -- 
> Teemu Harju
> 
> email/jabber: teemu.harju at gmail.com <mailto:teemu.harju at gmail.com>
> blog: http://blog.teemu.im
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


- -- 
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 T?bingen - Germany
Web: www.zopyx.com - Email: info at zopyx.com - Phone +49 - 7071 - 793376
Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
Gesch?ftsf?hrer/Gesellschafter: ZOPYX Limited, Birmingham, UK
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkmIH94ACgkQCJIWIbr9KYwZdQCg2jNDgHKk1L5qDgc6iXe9pGep
M1EAoLDyrCrofMedpQecm2RLUrsO1ccC
=Nbhh
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 330 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090203/2f77fad6/attachment-0001.vcf>

From kiorky at cryptelium.net  Tue Feb  3 14:28:16 2009
From: kiorky at cryptelium.net (kiorky)
Date: Tue, 03 Feb 2009 14:28:16 +0100
Subject: [Distutils] zc.buildout newbie question
In-Reply-To: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
References: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
Message-ID: <49884670.9080505@cryptelium.net>

Teemu Harju a ?crit :
> Hi all,
>
> I've been looking into zc.buildout since it seems to be quite useful
> application when developing and distributing Python applications. However,
> I'm finding it difficult to figure out how it should actually be used. I
> give you an example of what I'm trying to do and what my problem is. This is
> probably really trivial but bear with me. ;-)
>
> I'm developing a Twisted application and would like to use zc.buildout for
> bootstrapping development environments and probably even deploying the
> application to the production system.
>
> Here's my very simple buildout.cfg:
> ---
> [buildout]
> parts = twisted
>
> [twisted]
> recipe = minitage.recipe:du
> url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/Twisted-8.2.0.tar.bz2
> ---
>
> This works fine and installs Twisted under parts/site-packages-2.5/ and
> twistd, trial and other scripts under bin/. I'm doing all this inside
> virtualenv, but I guess it shouldn't matter.
>
> Now, my problem is that since the packages are installed in parts/ so they
> are not visible to my python interpreter. What is the right way to do this?
> Should I just add /parts/site-packages-2.5 to my PYTHONPATH or? Is there
> some way to do that using buildout.cfg? I tried to find examples of this but
> didn't find any.
>
> Br,
>
> - Teemu
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>   
Hi,
I am the maintener of minitage and you re using a quite old, and not
maintened recipe of this egg.
You may better to use minitage.recipe:eggs and minitage.recipe:scripts.


An example could be:
[buildout]
parts =
    twisted
    twisteds
[versions]
# The version which is put in the eggs cache.
Twisted = 8.2.0
[twisted]
recipe = minitage.recipe:egg
url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/Twisted-8.2.0.tar.bz2
[twisteds]
recipe = minitage.recipe:scripts
interpreter=myinterpreter
eggs=
    Twisted
    ${buildout:eggs}

Then, you ll have a nicely wrapped python interpreter in
bin/myinterpreter.-

More over, you can have a better look to minitage if you are not using
it in the whole already.

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF





From sdouche at gmail.com  Tue Feb  3 18:48:15 2009
From: sdouche at gmail.com (Sebastien Douche)
Date: Tue, 3 Feb 2009 18:48:15 +0100
Subject: [Distutils] Python package Management GUI - New Project on
	Sourceforge
In-Reply-To: <87iqnt1m1b.fsf@benfinney.id.au>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<402ded795e428be151d6e0d19d96f359@preisshare.net>
	<87iqnt1m1b.fsf@benfinney.id.au>
Message-ID: <5e1183fa0902030948k42fc3d8i93aa5acadc82f970@mail.gmail.com>

On Mon, Feb 2, 2009 at 05:48, Ben Finney <ben+python at benfinney.id.au> wrote:
>>  - fetches packages from http://pypi.python.org/pypi using
>>    their XML-RPC interface.
>
> Surely this is something that should be done by the package management
> library, making it available in one place for each front-end tool; not
> implemented in a specific front-end tool?

Right, for example if you use cache/mirror.


-- 
Sebastien Douche <sdouche at gmail.com>

From david.lyon at preisshare.net  Tue Feb  3 23:28:40 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 03 Feb 2009 17:28:40 -0500
Subject: [Distutils] Python package Management GUI - New Project on
	Sourceforge
In-Reply-To: <5e1183fa0902030948k42fc3d8i93aa5acadc82f970@mail.gmail.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<402ded795e428be151d6e0d19d96f359@preisshare.net>
	<87iqnt1m1b.fsf@benfinney.id.au>
	<5e1183fa0902030948k42fc3d8i93aa5acadc82f970@mail.gmail.com>
Message-ID: <71457c68eca5ae612141b282d49a098a@preisshare.net>



On Tue, 3 Feb 2009 18:48:15 +0100, Sebastien Douche <sdouche at gmail.com>
wrote:
> wrote:
>>>  - fetches packages from http://pypi.python.org/pypi using
>>>    their XML-RPC interface.
>>
>> Surely this is something that should be done by the package management
>> library, making it available in one place for each front-end tool; not
>> implemented in a specific front-end tool?
> 
> Right, for example if you use cache/mirror.

I'm not sure I understand what you mean?

As I work through the issues I have discovered that a number of the
installation libries support this functionality. That is pip and some
others. I don't think it is in setuptools yet. So I am putting provision
for that functionality in the gui from the start.

>From a developer perspective, what we need most is a list of packages that
we have installed (a manifest?) and then a way perphaps bundle all those
packages into one (file?).. and put them to our client machines in one
go...

At the moment, there doesn't seem to be any easy way of managing ones
installed packages. ie even finding help or documentation on ones packages
is quite difficult. That is what I am hoping to address.

That functionality is way down the track for me too....

I'm just trying to get the real simple stuff working first..... :-)

This is a screenshot so far...

http://sourceforge.net/projects/pythonpkgmgr/#item3rd-2


Regards

David 






From greg.landrum at gmail.com  Wed Feb  4 06:57:15 2009
From: greg.landrum at gmail.com (Greg Landrum)
Date: Wed, 4 Feb 2009 06:57:15 +0100
Subject: [Distutils] setuptools and shared library dependencies
Message-ID: <60825b0f0902032157y5db448a9x5d78a458a0764a5f@mail.gmail.com>

Dear all,

I'm in the process of shifting a project I run over to using
setuptools for distribution. Things have mostly worked quite well to
this point, so compliments to all involved. :-)

I'm just about finished, but I've run up against a problem with
extension modules that have shared library dependencies.

Here's a schematic of my directory structure:

BASE/
    bin/
        libDepA.so
    pydir/
         __init__.py
         modA.so
         foo.py

modA.so is an extension module that is dynamically linked against libDepA.so.

I've managed to get everything setup so that I can create an egg
reproducing the directory structure above, install that egg (not
zipped) with easy_install, and do things like
  >>> from pydir import foo
However, when I try to import modA:
  >>> from pydir import modA
I get the (expected) "ImportError: libDepA.so: cannot open shared
object file: No such file or directory"

If I were not using setuputils, I would make sure that I had BASE/bin
in my LD_LIBRARY_PATH and everything would work without problems.

If I explictly add the bin directory from the egg
(/usr/lib/python2.5/site-packages/BASE-0.1-py2.5-linux-i686.egg) to
LD_LIBRARY_PATH things also work, but requiring this level of
involvement from the user isn't particularly pleasing.

Is there a solution to this problem or is this a use case that doesn't
fit with setuptools?

thanks,
-greg

From teemu.harju at gmail.com  Wed Feb  4 11:29:03 2009
From: teemu.harju at gmail.com (Teemu Harju)
Date: Wed, 4 Feb 2009 12:29:03 +0200
Subject: [Distutils] zc.buildout newbie question
In-Reply-To: <49884670.9080505@cryptelium.net>
References: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
	<49884670.9080505@cryptelium.net>
Message-ID: <96274d630902040229s4936590aj3b73df3daed83b3@mail.gmail.com>

On Tue, Feb 3, 2009 at 3:28 PM, kiorky <kiorky at cryptelium.net> wrote:

> Hi,
> I am the maintener of minitage and you re using a quite old, and not
> maintened recipe of this egg.
> You may better to use minitage.recipe:eggs and minitage.recipe:scripts.
>
>
> An example could be:
> [buildout]
> parts =
>    twisted
>    twisteds
> [versions]
> # The version which is put in the eggs cache.
> Twisted = 8.2.0
> [twisted]
> recipe = minitage.recipe:egg
> url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/Twisted-8.2.0.tar.bz2
> [twisteds]
> recipe = minitage.recipe:scripts
> interpreter=myinterpreter
> eggs=
>    Twisted
>    ${buildout:eggs}
>
> Then, you ll have a nicely wrapped python interpreter in
> bin/myinterpreter.-
>
> More over, you can have a better look to minitage if you are not using
> it in the whole already.
>

Hi,

Thanks for the tip and for the great recipe. :) I'm liking the patching
capabilities of it also. Now my buildout is working.

- Teemu

-- 
Teemu Harju

email/jabber: teemu.harju at gmail.com
blog: http://blog.teemu.im
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090204/201f4854/attachment.htm>

From pje at telecommunity.com  Wed Feb  4 16:32:33 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 04 Feb 2009 10:32:33 -0500
Subject: [Distutils] setuptools and shared library dependencies
In-Reply-To: <60825b0f0902032157y5db448a9x5d78a458a0764a5f@mail.gmail.co
 m>
References: <60825b0f0902032157y5db448a9x5d78a458a0764a5f@mail.gmail.com>
Message-ID: <20090204153029.989EA3A4075@sparrow.telecommunity.com>

At 06:57 AM 2/4/2009 +0100, Greg Landrum wrote:
>If I were not using setuputils, I would make sure that I had BASE/bin
>in my LD_LIBRARY_PATH and everything would work without problems.
>
>If I explictly add the bin directory from the egg
>(/usr/lib/python2.5/site-packages/BASE-0.1-py2.5-linux-i686.egg) to
>LD_LIBRARY_PATH things also work, but requiring this level of
>involvement from the user isn't particularly pleasing.
>
>Is there a solution to this problem or is this a use case that doesn't
>fit with setuptools?

There's an experimental and unsupported solution, if you use 
setuptools to build the shared library, and you're on a compatible 
platform.  See the 'tests/shlib_test' subdirectory of the setuptools 
SVN trunk for info.  (The necessary code is in the 0.6 branch as 
well, but the tests are only in the trunk.)

(Also, if you use this approach, the .egg doesn't need to be 
installed unzipped.)


From lists at zopyx.com  Wed Feb  4 17:17:01 2009
From: lists at zopyx.com (Andreas Jung)
Date: Wed, 04 Feb 2009 17:17:01 +0100
Subject: [Distutils] [buildout] setuptools/buildout trying to use
 /root/.python-eggs after setuid()ing with Zope
Message-ID: <4989BF7D.3030606@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

I am not sure which component is in charge for the following issue:

 - standard Plone 3.1 installation installed with paster/Zopeskel
   with a 'zope' account without special privileges
 - in the instance is started from through /etc/init.d as root
   but immediately setuid to effective-user 'zope'

 - after setuid(), setuptools is trying to write to /root/.python-eggs
   during the startup-phase for (my own) module zopyx.textindexng3

- - it creates inside:

new:~/.python-eggs # ls -la
total 1
drwxrwxrwx  3 root root   144 Feb  4 13:17 .
drwxr-xr-x 24 root root  1200 Feb  4 17:10 ..
- -rw-r--r--  1 zope users    0 Feb  4 13:17 a
drwxr-xr-x  3 zope users   72 Feb  4 13:17
zopyx.textindexng3-4.0.1-py2.4-linux-x86_64.egg-tmp

Why is setuptools using /root/.python-eggs only for this particular
module and isn't it using something like /home/zope/.python-eggs
instead - especially having changed the UID already?

Andreas



- -- 
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 T?bingen - Germany
Web: www.zopyx.com - Email: info at zopyx.com - Phone +49 - 7071 - 793376
Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
Gesch?ftsf?hrer/Gesellschafter: ZOPYX Limited, Birmingham, UK
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkmJv30ACgkQCJIWIbr9KYwFVACbBM1JaanJvokkXLwcyMc6Dc4y
FCoAoIfG2qniHaLZbxS3OeGzh2fZkffe
=g4Ta
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090204/c7695783/attachment.vcf>

From greg.landrum at gmail.com  Wed Feb  4 17:30:01 2009
From: greg.landrum at gmail.com (Greg Landrum)
Date: Wed, 4 Feb 2009 17:30:01 +0100
Subject: [Distutils] setuptools and shared library dependencies
In-Reply-To: <20090204153029.989EA3A4075@sparrow.telecommunity.com>
References: <60825b0f0902032157y5db448a9x5d78a458a0764a5f@mail.gmail.com>
	<20090204153029.989EA3A4075@sparrow.telecommunity.com>
Message-ID: <60825b0f0902040830u3c4c8b1fr999d7f47b7e60566@mail.gmail.com>

On Wed, Feb 4, 2009 at 4:32 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:57 AM 2/4/2009 +0100, Greg Landrum wrote:
>>
>> If I were not using setuputils, I would make sure that I had BASE/bin
>> in my LD_LIBRARY_PATH and everything would work without problems.
>>
>> If I explictly add the bin directory from the egg
>> (/usr/lib/python2.5/site-packages/BASE-0.1-py2.5-linux-i686.egg) to
>> LD_LIBRARY_PATH things also work, but requiring this level of
>> involvement from the user isn't particularly pleasing.
>>
>> Is there a solution to this problem or is this a use case that doesn't
>> fit with setuptools?
>
> There's an experimental and unsupported solution, if you use setuptools to
> build the shared library, and you're on a compatible platform.  See the
> 'tests/shlib_test' subdirectory of the setuptools SVN trunk for info.  (The
> necessary code is in the 0.6 branch as well, but the tests are only in the
> trunk.)

I'm afraid I'm not using setuptools to build. Does this mean I'm stuck?

-greg

From jim at zope.com  Wed Feb  4 18:46:02 2009
From: jim at zope.com (Jim Fulton)
Date: Wed, 4 Feb 2009 12:46:02 -0500
Subject: [Distutils] [buildout] setuptools/buildout trying to use
	/root/.python-eggs after setuid()ing with Zope
In-Reply-To: <4989BF7D.3030606@zopyx.com>
References: <4989BF7D.3030606@zopyx.com>
Message-ID: <BD52645E-4905-4703-B430-31A5609E607A@zope.com>


On Feb 4, 2009, at 11:17 AM, Andreas Jung wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi there,
>
> I am not sure which component is in charge for the following issue:
>
> - standard Plone 3.1 installation installed with paster/Zopeskel
>   with a 'zope' account without special privileges
> - in the instance is started from through /etc/init.d as root
>   but immediately setuid to effective-user 'zope'
>
> - after setuid(), setuptools is trying to write to /root/.python-eggs
>   during the startup-phase for (my own) module zopyx.textindexng3
>
> - - it creates inside:
>
> new:~/.python-eggs # ls -la
> total 1
> drwxrwxrwx  3 root root   144 Feb  4 13:17 .
> drwxr-xr-x 24 root root  1200 Feb  4 17:10 ..
> - -rw-r--r--  1 zope users    0 Feb  4 13:17 a
> drwxr-xr-x  3 zope users   72 Feb  4 13:17
> zopyx.textindexng3-4.0.1-py2.4-linux-x86_64.egg-tmp
>
> Why is setuptools using /root/.python-eggs only for this particular
> module and isn't it using something like /home/zope/.python-eggs
> instead - especially having changed the UID already?


I'll answer the problem, not the question. (I don't think setuptools  
should ever write to a user's home directory, whatever the user.)

The easiest way to solve this problem is to use unzipped eggs, which  
also import faster in my experience. Buildout now has a global option  
(unzip = true) that causes all eggs it installs to be unzipped.

Jim

--
Jim Fulton
Zope Corporation



From cz at gocept.com  Thu Feb  5 09:28:42 2009
From: cz at gocept.com (Christian Zagrodnick)
Date: Thu, 5 Feb 2009 09:28:42 +0100
Subject: [Distutils] zc.buildout 'extends' options
References: <4971C758.9070802@gmail.com>
Message-ID: <gme7vq$fbk$1@ger.gmane.org>

On 2009-01-17 12:56:08 +0100, Martin Aspeli <optilude+lists at gmail.com> said:

> Hi Jim & co,
> 
> Plone 3.2 currently distributes its packages as eggs, and people use an
> 'extends' option with a remote URL to get a known good set for each
> version, e.g.:
> 
>   [buildout]
>   extends = http://dist.plone.org/release/3.2/versions.cfg

[....]

> 
> Is this desirable?

Yes, definitely.


BTW: I also have test failures on trunk/py2.5.


-- 
Christian Zagrodnick ? cz at gocept.com
gocept gmbh & co. kg ? forsterstra?e 29 ? 06112 halle (saale) ? germany
http://gocept.com ? tel +49 345 1229889 4 ? fax +49 345 1229889 1
Zope and Plone consulting and development



From fadhley.salim at uk.calyon.com  Thu Feb  5 19:43:26 2009
From: fadhley.salim at uk.calyon.com (Fadhley Salim)
Date: Thu, 5 Feb 2009 18:43:26 -0000
Subject: [Distutils] occasional zipimport.ZipImportError: bad local file
	header
References: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
	<49884670.9080505@cryptelium.net>
Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483F3B@MXCU10MX1.MSX.CIB>

I'm working on an automatic testing framework which installs eggs
downloaded directly from the web-server. From time to time I get a very
long stacktrace leading to a ZipImportError (see the pastebin link).
Leading up to the error all I do is download the egg into the system's
%TEMP% folder. I'm installing the egg like this:

easy_install.main( ['-zma', '-f' , config.INSTALL_TOOLS_EGG_VALIDHOSTS,
filepath, ]

There is apparantly nothing wrong with the file - I can open it in
Winzip. If I re-run the process it seems to work fine. The file in
question appears to be identical. The error is rare, it seems to occur
in about 1 out of 100 easy_install operations. Any suggestions what
might be causing this? 

Very long stacktrace on pastebin:
http://python.pastebin.com/m378949f3

I am using setuptools==0.6c9 on Python 2.4 running on Windows XP
-------------- next part --------------
This email does not create a legal relationship between any member of the 
Cr?dit Agricole group and the recipient or constitute investment advice. 
The content of this email should not be copied or disclosed (in whole or 
part) to any other person. It may contain information which is 
confidential, privileged or otherwise protected from disclosure. If you 
are not the intended recipient, you should notify us and delete it from 
your system. Emails may be monitored, are not secure and may be amended, 
destroyed or contain viruses and in communicating with us such conditions 
are accepted. Any content which does not relate to business matters is not 
endorsed by us.

Calyon is authorised by the Comit? des Etablissements de Cr?dit et des
Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire
in France and subject to limited regulation by the Financial Services Authority.
Details about the extent of our regulation by the Financial Services Authority
are available from us on request. Calyon is incorporated in France with limited
liability and registered in England & Wales. Registration number: FC008194.
Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA.


From ziade.tarek at gmail.com  Sun Feb  8 11:05:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 8 Feb 2009 11:05:41 +0100
Subject: [Distutils] Maintainer and Maintainer email metadata
Message-ID: <94bdd2610902080205w63dcd009v6d9af00d63af47ca@mail.gmail.com>

Hello,

Right now Distutils will let you add the name of the maintainer and
the name of the author as two distinct metadata in your package.

When creating the PKG-INFO, Distutils includes an "Author" line.

To build it, it will look at first for at the maintainer metadata and
use it if provided, then only at the author metadata.

This is not a correct behavior;

Two points:

1/ Distutils should not use the maintainer metadata to create the
*Author* line in the PKG-INFO
2/ We could add  the extra "Maintainer"  field in PKG-INFO,

I am all for 1/,   but I am not sure for 2/.  I never use the
Maintainer field my self.

Any opinions ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From regebro at gmail.com  Sun Feb  8 11:31:51 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 8 Feb 2009 11:31:51 +0100
Subject: [Distutils] Maintainer and Maintainer email metadata
In-Reply-To: <94bdd2610902080205w63dcd009v6d9af00d63af47ca@mail.gmail.com>
References: <94bdd2610902080205w63dcd009v6d9af00d63af47ca@mail.gmail.com>
Message-ID: <319e029f0902080231g23953464ib0d4fe1a39724500@mail.gmail.com>

On Sun, Feb 8, 2009 at 11:05, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> Right now Distutils will let you add the name of the maintainer and
> the name of the author as two distinct metadata in your package.
>
> When creating the PKG-INFO, Distutils includes an "Author" line.
>
> To build it, it will look at first for at the maintainer metadata and
> use it if provided, then only at the author metadata.
>
> This is not a correct behavior;
>
> Two points:
>
> 1/ Distutils should not use the maintainer metadata to create the
> *Author* line in the PKG-INFO
> 2/ We could add  the extra "Maintainer"  field in PKG-INFO,
>
> I am all for 1/,   but I am not sure for 2/.  I never use the
> Maintainer field my self.
>
> Any opinions ?

Who is the maintainer is more important than who is the author. +1 on
adding a Maintainer field in PKG_INFO.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From marius at pov.lt  Tue Feb 10 10:09:31 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 10 Feb 2009 11:09:31 +0200
Subject: [Distutils] [buildout] setuptools/buildout trying to use
	/root/.python-eggs after setuid()ing with Zope
In-Reply-To: <4989BF7D.3030606@zopyx.com>
References: <4989BF7D.3030606@zopyx.com>
Message-ID: <20090210090931.GA665@fridge.pov.lt>

On Wed, Feb 04, 2009 at 05:17:01PM +0100, Andreas Jung wrote:
> Hi there,
> 
> I am not sure which component is in charge for the following issue:
> 
>  - standard Plone 3.1 installation installed with paster/Zopeskel
>    with a 'zope' account without special privileges
>  - in the instance is started from through /etc/init.d as root
>    but immediately setuid to effective-user 'zope'
> 
>  - after setuid(), setuptools is trying to write to /root/.python-eggs
>    during the startup-phase for (my own) module zopyx.textindexng3
> 
> - it creates inside:
> 
> new:~/.python-eggs # ls -la
> total 1
> drwxrwxrwx  3 root root   144 Feb  4 13:17 .
> drwxr-xr-x 24 root root  1200 Feb  4 17:10 ..
> -rw-r--r--  1 zope users    0 Feb  4 13:17 a
> drwxr-xr-x  3 zope users   72 Feb  4 13:17
> zopyx.textindexng3-4.0.1-py2.4-linux-x86_64.egg-tmp
> 
> Why is setuptools using /root/.python-eggs only for this particular
> module and isn't it using something like /home/zope/.python-eggs
> instead - especially having changed the UID already?

It doesn't look at UID, it looks at $HOME.  Actually, it looks at
$PYTHON_EGG_CACHE first, so you can set that.

I had the same problem with trac and matplotlib (which doesn't have a
sane separate environment variable), and in the end had to resort to an
ugly hack in my /etc/init.d/script:

  sudo -u $NONPRIVILEGED_USER env HOME=/path/writable/to/that/user \
    start-stop-daemon --start --exec /path/to/executable -- $options

Marius Gedminas
-- 
HOST SYSTEM NOT RESPONDING, PROBABLY DOWN. DO YOU WANT TO WAIT? (Y/N)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090210/504fdd30/attachment.pgp>

From pje at telecommunity.com  Wed Feb 11 05:11:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 10 Feb 2009 23:11:10 -0500
Subject: [Distutils] occasional zipimport.ZipImportError: bad local file
 header
In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483F3B@MXCU10MX1.MSX.CIB
 >
References: <96274d630902030234w12e9cc5xb22d2bb2fbdf7757@mail.gmail.com>
	<49884670.9080505@cryptelium.net>
	<7F347D91614EBC48AA7540A2A76D3BB204483F3B@MXCU10MX1.MSX.CIB>
Message-ID: <20090211040904.156B93A4075@sparrow.telecommunity.com>

At 06:43 PM 2/5/2009 +0000, Fadhley Salim wrote:
>I'm working on an automatic testing framework which installs eggs
>downloaded directly from the web-server. From time to time I get a very
>long stacktrace leading to a ZipImportError (see the pastebin link).
>Leading up to the error all I do is download the egg into the system's
>%TEMP% folder. I'm installing the egg like this:
>
>easy_install.main( ['-zma', '-f' , config.INSTALL_TOOLS_EGG_VALIDHOSTS,
>filepath, ]
>
>There is apparantly nothing wrong with the file - I can open it in
>Winzip. If I re-run the process it seems to work fine. The file in
>question appears to be identical. The error is rare, it seems to occur
>in about 1 out of 100 easy_install operations. Any suggestions what
>might be causing this?
>
>Very long stacktrace on pastebin:
>http://python.pastebin.com/m378949f3
>
>I am using setuptools==0.6c9 on Python 2.4 running on Windows XP

This might be due to:

http://bugs.python.org/issue856103

Specifically, zipimport's caching doesn't notice that it's not 
working on the same zipfile any more.  easy_install is supposed to 
have some code in it to clear out the zipimport cache, but there is 
some possibility that it could have two versions of the path in there 
on a case-insensitive filesystem (e.g. Windows), and only one of them 
is getting cleared.  Dunno if that's the case or not, but it might be 
something to look into.  You could always stick a debugging print in 
the uncache_zipdir function and see if there's any correlation 
between what it's doing and when you're getting the error.


From wright at esrf.fr  Wed Feb 11 09:33:48 2009
From: wright at esrf.fr (Jon Wright)
Date: Wed, 11 Feb 2009 09:33:48 +0100
Subject: [Distutils] How to use buildout with a scripts directory?
Message-ID: <49928D6C.3050209@esrf.fr>

Dear distutils-sig,

[Apologies if this is a newbie question]

I am trying to use buildout and am having trouble to figure out how to
get it to install the scripts that come with a package. They are
present inside the egg which is installed by buildout, but they don't
end up in the bin directory. After running buildout my bin directory
is empty, but I can find:

.\eggs\imaged11-1.2.3-py2.5-win32.egg\EGG-INFO\scripts\peaksearch.py

In the package I have a directory (ImageD11) with the library stuff in
it and a directory named "scripts" with the scripts. When I use
distutils or setuptools it does what I expected:

setup.py:
...
setup(name='ImageD11',
       ...,
       packages = ["ImageD11"],
       package_dir = {"ImageD11":"ImageD11"},
      scripts = [ "scripts/peaksearch.py",
                     "scripts/bgmaker.py",
                     ...]
      )

c:\testImageD11distutils\> python setup.py install
...
Installing peaksearch.py script to c:\python25\Scripts
Installing bgmaker.py script to c:\python25\Scripts

Now I am wondering how to persuade "buildout" to do the same thing,
but place those scripts into it's own little area.

In buildout.cfg I have:
==================
[buildout]
parts = ImageD11

[ImageD11]
recipe = zc.recipe.egg:scripts
scripts = scripts/peaksearch.py
eggs = ImageD11
version = 1.2.3

It seems to acknowledge that scripts/peaksearch.py is requested, but
doesn't seem to do anything about it. My scripts mostly just map
command line arguments to function calls, something like this:

if __name__=="__main__":
     import ImageD11.something, sys
     ImageD11.something.do_thing( sys.argv[1], sys.argv[2] )

I've read the documentation at http://pypi.python.org/pypi/zc.buildout
several times but I still can't figure out how to make these scripts
part of the build. There seems to be a lot of talk about entry_points,
but I'm blocked on those:
  * what is the entry point for an if __name__=="__main__": idiom?
  * as these things were in ./scripts/, they aren't in the module, so 
how to I reference them anyway?

Thanks in advance for any help!

Jon


From jim at zope.com  Wed Feb 11 15:49:05 2009
From: jim at zope.com (Jim Fulton)
Date: Wed, 11 Feb 2009 09:49:05 -0500
Subject: [Distutils] How to use buildout with a scripts directory?
In-Reply-To: <49928D6C.3050209@esrf.fr>
References: <49928D6C.3050209@esrf.fr>
Message-ID: <64C7D438-5B35-4D4B-B18E-B37CACEE8F72@zope.com>

buildout only supports scripts defined as entry points.  I'm skeptical  
that it can support non-entry-point scripts, as it wouldn't, without  
some serious hackery, be able to control the paths used by the scripts.

Jim

On Feb 11, 2009, at 3:33 AM, Jon Wright wrote:

> Dear distutils-sig,
>
> [Apologies if this is a newbie question]
>
> I am trying to use buildout and am having trouble to figure out how to
> get it to install the scripts that come with a package. They are
> present inside the egg which is installed by buildout, but they don't
> end up in the bin directory. After running buildout my bin directory
> is empty, but I can find:
>
> .\eggs\imaged11-1.2.3-py2.5-win32.egg\EGG-INFO\scripts\peaksearch.py
>
> In the package I have a directory (ImageD11) with the library stuff in
> it and a directory named "scripts" with the scripts. When I use
> distutils or setuptools it does what I expected:
>
> setup.py:
> ...
> setup(name='ImageD11',
>      ...,
>      packages = ["ImageD11"],
>      package_dir = {"ImageD11":"ImageD11"},
>     scripts = [ "scripts/peaksearch.py",
>                    "scripts/bgmaker.py",
>                    ...]
>     )
>
> c:\testImageD11distutils\> python setup.py install
> ...
> Installing peaksearch.py script to c:\python25\Scripts
> Installing bgmaker.py script to c:\python25\Scripts
>
> Now I am wondering how to persuade "buildout" to do the same thing,
> but place those scripts into it's own little area.
>
> In buildout.cfg I have:
> ==================
> [buildout]
> parts = ImageD11
>
> [ImageD11]
> recipe = zc.recipe.egg:scripts
> scripts = scripts/peaksearch.py
> eggs = ImageD11
> version = 1.2.3
>
> It seems to acknowledge that scripts/peaksearch.py is requested, but
> doesn't seem to do anything about it. My scripts mostly just map
> command line arguments to function calls, something like this:
>
> if __name__=="__main__":
>    import ImageD11.something, sys
>    ImageD11.something.do_thing( sys.argv[1], sys.argv[2] )
>
> I've read the documentation at http://pypi.python.org/pypi/zc.buildout
> several times but I still can't figure out how to make these scripts
> part of the build. There seems to be a lot of talk about entry_points,
> but I'm blocked on those:
> * what is the entry point for an if __name__=="__main__": idiom?
> * as these things were in ./scripts/, they aren't in the module, so  
> how to I reference them anyway?
>
> Thanks in advance for any help!
>
> Jon
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Jim Fulton
Zope Corporation



From patrick.gerken at googlemail.com  Wed Feb 11 20:06:47 2009
From: patrick.gerken at googlemail.com (Patrick Gerken)
Date: Wed, 11 Feb 2009 20:06:47 +0100
Subject: [Distutils] Wrong depedencies in install_lib command?
Message-ID: <24d2b8960902111106r690df6c3y30234122d3223755@mail.gmail.com>

Hello,

i tried to install reportlab via easy_install, which always failed while
compiling some c-modules.
Installing it via setup.py install worked flawlessly

After a while, I found the difference.
setup.py install calls the command _build_, which calls _build_clib_ and
then _build_ext_

easy_install calls _bdist_egg_, which calls _install_lib_, which calls
_build_ext_ WITHOUT calling _build_clib_.
The module docstring of the command build_clib mentions, that these
libraries can be a dependency to build_ext, so I patched install_lib.py to
call build_clib too, and now I can easy_install the newest reportlab!

If anybody wants to reproduce the behaviour, I added my patch, which I did
against the current trunk of distutils/command/install_lib.py
I made a relative patch so to patch the file. you have to go into this
directory and call

patch install_lib.py dependency.patch

Does this qualify as a bug in the bugtracker?

Best regards,

         Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090211/ae2ff966/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dependency.patch
Type: application/octet-stream
Size: 471 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090211/ae2ff966/attachment.obj>

From pje at telecommunity.com  Wed Feb 11 20:17:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 11 Feb 2009 14:17:41 -0500
Subject: [Distutils] Wrong depedencies in install_lib command?
In-Reply-To: <24d2b8960902111106r690df6c3y30234122d3223755@mail.gmail.co
 m>
References: <24d2b8960902111106r690df6c3y30234122d3223755@mail.gmail.com>
Message-ID: <20090211191535.E795B3A40A0@sparrow.telecommunity.com>

At 08:06 PM 2/11/2009 +0100, Patrick Gerken wrote:
>Hello,
>
>i tried to install reportlab via easy_install, which always failed 
>while compiling some c-modules.
>Installing it via setup.py install worked flawlessly
>
>After a while, I found the difference.
>setup.py install calls the command _build_, which calls _build_clib_ 
>and then _build_ext_
>
>easy_install calls _bdist_egg_, which calls _install_lib_, which 
>calls _build_ext_ WITHOUT calling _build_clib_.
>The module docstring of the command build_clib mentions, that these 
>libraries can be a dependency to build_ext, so I patched 
>install_lib.py to call build_clib too, and now I can easy_install 
>the newest reportlab!
>
>If anybody wants to reproduce the behaviour, I added my patch, which 
>I did against the current trunk of distutils/command/install_lib.py
>I made a relative patch so to patch the file. you have to go into 
>this directory and call
>
>patch install_lib.py dependency.patch
>
>Does this qualify as a bug in the bugtracker?

Yes, definitely.  It'd be nice to have a setuptools patch to 
workaround the problem as well.


From wright at esrf.fr  Thu Feb 12 10:13:11 2009
From: wright at esrf.fr (Jon Wright)
Date: Thu, 12 Feb 2009 10:13:11 +0100
Subject: [Distutils] How to use buildout with a scripts directory?
In-Reply-To: <64C7D438-5B35-4D4B-B18E-B37CACEE8F72@zope.com>
References: <49928D6C.3050209@esrf.fr>
	<64C7D438-5B35-4D4B-B18E-B37CACEE8F72@zope.com>
Message-ID: <4993E827.5060109@esrf.fr>

Thanks for your clarifying this for me - I will convert the scripts to 
use entry points instead.

Jon

Jim Fulton wrote:
> buildout only supports scripts defined as entry points.  I'm skeptical 
> that it can support non-entry-point scripts, as it wouldn't, without 
> some serious hackery, be able to control the paths used by the scripts.
> 
> Jim
> 
> On Feb 11, 2009, at 3:33 AM, Jon Wright wrote:
> 
>> Dear distutils-sig,
>>
>> [Apologies if this is a newbie question]
>>
>> I am trying to use buildout and am having trouble to figure out how to
>> get it to install the scripts that come with a package. They are
>> present inside the egg which is installed by buildout, but they don't
>> end up in the bin directory. After running buildout my bin directory
>> is empty, but I can find:
>>
>> .\eggs\imaged11-1.2.3-py2.5-win32.egg\EGG-INFO\scripts\peaksearch.py
>>
>> In the package I have a directory (ImageD11) with the library stuff in
>> it and a directory named "scripts" with the scripts. When I use
>> distutils or setuptools it does what I expected:
>>
>> setup.py:
>> ...
>> setup(name='ImageD11',
>>      ...,
>>      packages = ["ImageD11"],
>>      package_dir = {"ImageD11":"ImageD11"},
>>     scripts = [ "scripts/peaksearch.py",
>>                    "scripts/bgmaker.py",
>>                    ...]
>>     )
>>
>> c:\testImageD11distutils\> python setup.py install
>> ...
>> Installing peaksearch.py script to c:\python25\Scripts
>> Installing bgmaker.py script to c:\python25\Scripts
>>
>> Now I am wondering how to persuade "buildout" to do the same thing,
>> but place those scripts into it's own little area.
>>
>> In buildout.cfg I have:
>> ==================
>> [buildout]
>> parts = ImageD11
>>
>> [ImageD11]
>> recipe = zc.recipe.egg:scripts
>> scripts = scripts/peaksearch.py
>> eggs = ImageD11
>> version = 1.2.3
>>
>> It seems to acknowledge that scripts/peaksearch.py is requested, but
>> doesn't seem to do anything about it. My scripts mostly just map
>> command line arguments to function calls, something like this:
>>
>> if __name__=="__main__":
>>    import ImageD11.something, sys
>>    ImageD11.something.do_thing( sys.argv[1], sys.argv[2] )
>>
>> I've read the documentation at http://pypi.python.org/pypi/zc.buildout
>> several times but I still can't figure out how to make these scripts
>> part of the build. There seems to be a lot of talk about entry_points,
>> but I'm blocked on those:
>> * what is the entry point for an if __name__=="__main__": idiom?
>> * as these things were in ./scripts/, they aren't in the module, so 
>> how to I reference them anyway?
>>
>> Thanks in advance for any help!
>>
>> Jon
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> -- 
> Jim Fulton
> Zope Corporation
> 
> 


From patrick.gerken at googlemail.com  Fri Feb 13 10:59:07 2009
From: patrick.gerken at googlemail.com (Patrick Gerken)
Date: Fri, 13 Feb 2009 10:59:07 +0100
Subject: [Distutils] Wrong depedencies in install_lib command?
In-Reply-To: <20090211191535.E795B3A40A0@sparrow.telecommunity.com>
References: <24d2b8960902111106r690df6c3y30234122d3223755@mail.gmail.com>
	<20090211191535.E795B3A40A0@sparrow.telecommunity.com>
Message-ID: <24d2b8960902130159u51197f56l4202f18a661fdf13@mail.gmail.com>

On Wed, Feb 11, 2009 at 20:17, P.J. Eby <pje at telecommunity.com> wrote:

> At 08:06 PM 2/11/2009 +0100, Patrick Gerken wrote:
>
>> Hello,
>>
>> i tried to install reportlab via easy_install, which always failed while
>> compiling some c-modules.
>> Installing it via setup.py install worked flawlessly
>>
>> After a while, I found the difference.
>> setup.py install calls the command _build_, which calls _build_clib_ and
>> then _build_ext_
>>
>> easy_install calls _bdist_egg_, which calls _install_lib_, which calls
>> _build_ext_ WITHOUT calling _build_clib_.
>> The module docstring of the command build_clib mentions, that these
>> libraries can be a dependency to build_ext, so I patched install_lib.py to
>> call build_clib too, and now I can easy_install the newest reportlab!
>>
>> If anybody wants to reproduce the behaviour, I added my patch, which I did
>> against the current trunk of distutils/command/install_lib.py
>> I made a relative patch so to patch the file. you have to go into this
>> directory and call
>>
>> patch install_lib.py dependency.patch
>>
>> Does this qualify as a bug in the bugtracker?
>>
>
> Yes, definitely.  It'd be nice to have a setuptools patch to workaround the
> problem as well.


I feel a bit uneasy about this kind of stuff, because of potential side
effects I am not aware of. Would it be fine to call the command _build_
after the command _egg_info_ in _bdist_egg_ ? That would call the other
commands in the right order.

I added a ticket, but I am still not finished writing the test case. I will
add one, hopefully during the weekend
http://bugs.python.org/issue5243

Best regards,

       Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090213/01bc8af0/attachment.htm>

From setuptools at bugs.python.org  Sat Feb 14 02:10:17 2009
From: setuptools at bugs.python.org (Philip Jenvey)
Date: Sat, 14 Feb 2009 01:10:17 +0000
Subject: [Distutils] [issue59] [PATCH] sandbox.py replaces builtin type file
	with builtin function open
In-Reply-To: <1234573815.97.0.144580398838.issue59@psf.upfronthosting.co.za>
Message-ID: <1234573815.97.0.144580398838.issue59@psf.upfronthosting.co.za>


New submission from Philip Jenvey <pjenvey at underboss.org>:

builtin open changed from an alias to the file type to a function in Python 2.5. 
sandbox.py assumes file is open when replacing/restoring the two with sandboxed 
versions

Attached is a patch that removes that hardcoding

Prioritized as critical, this is evil!

----------
assignedto: pje
files: builtin_file-r66750.diff
messages: 236
nosy: pje, pjenvey
priority: critical
status: unread
title: [PATCH] sandbox.py replaces builtin type file with builtin function open
Added file: http://bugs.python.org/setuptools/file36/builtin_file-r66750.diff

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue59>
_______________________________________________
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: builtin_file-r66750.diff
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090214/5a32031a/attachment.txt>

From ziade.tarek at gmail.com  Sat Feb 14 11:28:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 14 Feb 2009 11:28:07 +0100
Subject: [Distutils] Extending default file list
Message-ID: <94bdd2610902140228o62cc28fcna0be18f2e2121a49@mail.gmail.com>

Hi,

for http://bugs.python.org/issue2279

I would like to extend the default file list (sdist.add_defaults) so
it adds files pointed by package_data and data_files

Any objection/opinion ?

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sat Feb 14 13:26:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 14 Feb 2009 13:26:40 +0100
Subject: [Distutils] console scripts in distutils
Message-ID: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>

Hello

I really like the way setuptools wraps scripts depending on the platform.

http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation

I am not talking about the entry_point feature, but the way it creates
executables.

What would people think about making that feature go into distutils
itself to enhance the current script behavior?

http://bugs.python.org/issue870479

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From robert.kern at gmail.com  Sat Feb 14 23:23:33 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 14 Feb 2009 16:23:33 -0600
Subject: [Distutils] console scripts in distutils
In-Reply-To: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>
References: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>
Message-ID: <gn7g95$ms2$1@ger.gmane.org>

On 2009-02-14 06:26, Tarek Ziad? wrote:
> Hello
>
> I really like the way setuptools wraps scripts depending on the platform.
>
> http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation
>
> I am not talking about the entry_point feature, but the way it creates
> executables.
>
> What would people think about making that feature go into distutils
> itself to enhance the current script behavior?
>
> http://bugs.python.org/issue870479

I have a couple of issues with this feature as implemented in setuptools. I 
think they should be addressed if the feature makes its way into the stdlib:

1. The pkg_resources import can make startup time quite large. For small 
command-line utilities like my grin tool and DivMod's PyFlakes, this can make 
them unusable. If you bring the feature into the stdlib, please pay attention to 
the overhead of locating the real script however you end up doing it.

2. In the generated script, please use "if __name__ == '__main__':" to block out 
the executable bits. Currently, multiprocessing does not work on Windows when 
the application is started by a console_script. Because Windows does not have a 
true fork, multiprocessing will start up clean Python subprocesses that import 
the __main__ of the parent process.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ziade.tarek at gmail.com  Sun Feb 15 10:39:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Feb 2009 10:39:48 +0100
Subject: [Distutils] Uninstall command
Message-ID: <94bdd2610902150139w1d8039c0g925b073dd71216ce@mail.gmail.com>

Hi,

I started to write a detailed description of the uninstall feature
based on the previous threads we had, and there are two points I'd
like to
discuss:

1. should the uninstall command be part of the Python interpreter itself ?

    for instance, be callable like this :

               $ python -r MyPackage

   or :
               $ python --remove-package MyPackage


2. If we add an uninstall command, it makes sense to me to add an
install command as well, at
    the same level.

    maybe :

               $ python -a MyPackage

    or :
               $ python --add-package MyPackage



Regards
Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sun Feb 15 10:51:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Feb 2009 10:51:38 +0100
Subject: [Distutils] console scripts in distutils
In-Reply-To: <gn7g95$ms2$1@ger.gmane.org>
References: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>
	<gn7g95$ms2$1@ger.gmane.org>
Message-ID: <94bdd2610902150151o34b551f7gea4a11834325eb83@mail.gmail.com>

On Sat, Feb 14, 2009 at 11:23 PM, Robert Kern <robert.kern at gmail.com> wrote:
> I have a couple of issues with this feature as implemented in setuptools. I
> think they should be addressed if the feature makes its way into the stdlib:
>
> 1. The pkg_resources import can make startup time quite large. For small
> command-line utilities like my grin tool and DivMod's PyFlakes, this can
> make them unusable. If you bring the feature into the stdlib, please pay
> attention to the overhead of locating the real script however you end up
> doing it.
>

Right, it getting very slow when you have a lot of sitedirs and .pth files,

I think an API in pkgutil should handle this part and focus on performances,

There's a patch I have started for this, that takes back pkg_resources
import principles
to look for a packages to return their metadata.

http://bugs.python.org/issue4908

> 2. In the generated script, please use "if __name__ == '__main__':" to block
> out the executable bits. Currently, multiprocessing does not work on Windows
> when the application is started by a console_script. Because Windows does
> not have a true fork, multiprocessing will start up clean Python
> subprocesses that import the __main__ of the parent process.


OK, I'll point this thread in the ticket

Tarek

From ziade.tarek at gmail.com  Sun Feb 15 11:29:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Feb 2009 11:29:55 +0100
Subject: [Distutils] Maintainer and Maintainer email metadata
In-Reply-To: <319e029f0902080231g23953464ib0d4fe1a39724500@mail.gmail.com>
References: <94bdd2610902080205w63dcd009v6d9af00d63af47ca@mail.gmail.com>
	<319e029f0902080231g23953464ib0d4fe1a39724500@mail.gmail.com>
Message-ID: <94bdd2610902150229v251b89afq6d75c0775cca0042@mail.gmail.com>

I am going to start a new PEP defining a new, more sane EGG-INFO
directory format,
and clarifying the need to make metadata that we can put in setup.py match with
the ones in PKG-INFO file.


On Sun, Feb 8, 2009 at 11:31 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Sun, Feb 8, 2009 at 11:05, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> Hello,
>>
>> Right now Distutils will let you add the name of the maintainer and
>> the name of the author as two distinct metadata in your package.
>>
>> When creating the PKG-INFO, Distutils includes an "Author" line.
>>
>> To build it, it will look at first for at the maintainer metadata and
>> use it if provided, then only at the author metadata.
>>
>> This is not a correct behavior;
>>
>> Two points:
>>
>> 1/ Distutils should not use the maintainer metadata to create the
>> *Author* line in the PKG-INFO
>> 2/ We could add  the extra "Maintainer"  field in PKG-INFO,
>>
>> I am all for 1/,   but I am not sure for 2/.  I never use the
>> Maintainer field my self.
>>
>> Any opinions ?
>
> Who is the maintainer is more important than who is the author. +1 on
> adding a Maintainer field in PKG_INFO.
>
> --
> Lennart Regebro: Zope and Plone consulting.
> http://www.colliberty.com/
> +33 661 58 14 64
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From floris.bruynooghe at gmail.com  Sun Feb 15 12:53:12 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sun, 15 Feb 2009 11:53:12 +0000
Subject: [Distutils] Uninstall command
In-Reply-To: <94bdd2610902150139w1d8039c0g925b073dd71216ce@mail.gmail.com>
References: <94bdd2610902150139w1d8039c0g925b073dd71216ce@mail.gmail.com>
Message-ID: <20090215115312.GA8131@laurie.devork>

On Sun, Feb 15, 2009 at 10:39:48AM +0100, Tarek Ziad? wrote:
> 1. should the uninstall command be part of the Python interpreter
> itself ?

Don't think I like this.  Seems a very big jump from how this normally
works.  What I mean is that it feels like it's builtin interperter
functionality, while really it's a module/script doing distutils-like
stuff.  I'd feel more comfortable with something like:

    $ python -m some_tool --remove MyPackage


>     for instance, be callable like this :
> 
>                $ python -r MyPackage

Definitely -1 on the short option.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From p.f.moore at gmail.com  Sun Feb 15 15:07:48 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 15 Feb 2009 14:07:48 +0000
Subject: [Distutils] console scripts in distutils
In-Reply-To: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>
References: <94bdd2610902140426y7fb1e0b4o7de2e78de4443e@mail.gmail.com>
Message-ID: <79990c6b0902150607u3ab1bb8ay7c483cc26b2f6db3@mail.gmail.com>

2009/2/14 Tarek Ziad? <ziade.tarek at gmail.com>:
> I really like the way setuptools wraps scripts depending on the platform.
>
> http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation
>
> I am not talking about the entry_point feature, but the way it creates
> executables.
>
> What would people think about making that feature go into distutils
> itself to enhance the current script behavior?

I mentioned this on the ticket, but I'm adding it here for comments:

If this is to go into the stdlib, can it be implemented in such a way
that creating a bdist_wininst installer for a pure-python package with
wrapped scripts does *not* make the package version-dependent?

Paul.

From greno at verizon.net  Sun Feb 15 23:56:24 2009
From: greno at verizon.net (Gerry Reno)
Date: Sun, 15 Feb 2009 17:56:24 -0500
Subject: [Distutils] bdist_rpm:  spec file: %config
Message-ID: <49989D98.7070803@verizon.net>

I've been looking at the spec file that 'bdist_rpm' generates and I do 
not see a %config or %config(noreplace) section.  Is there a way to 
generate a %config section and specify a list of files?  This is 
important so that config files do not get overwritten when using 'rpm -U'.


From ziade.tarek at gmail.com  Mon Feb 16 00:04:14 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Feb 2009 00:04:14 +0100
Subject: [Distutils] bdist_rpm: spec file: %config
In-Reply-To: <49989D98.7070803@verizon.net>
References: <49989D98.7070803@verizon.net>
Message-ID: <94bdd2610902151504n3862cfb0h706a7dcb31c51ba9@mail.gmail.com>

On Sun, Feb 15, 2009 at 11:56 PM, Gerry Reno <greno at verizon.net> wrote:
> I've been looking at the spec file that 'bdist_rpm' generates and I do not
> see a %config or %config(noreplace) section.  Is there a way to generate a
> %config section and specify a list of files?  This is important so that
> config files do not get overwritten when using 'rpm -U'.

Looking at the code, I don't see such option,

please fill a feature request in http://bugs.python.org/ selecting
'Distutils' as a component,

(with a sample of such a spec file if possible)

Thanks

>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From greno at verizon.net  Mon Feb 16 03:59:13 2009
From: greno at verizon.net (Gerry Reno)
Date: Sun, 15 Feb 2009 21:59:13 -0500
Subject: [Distutils] bdist_rpm: spec file: %config
In-Reply-To: <94bdd2610902151504n3862cfb0h706a7dcb31c51ba9@mail.gmail.com>
References: <49989D98.7070803@verizon.net>
	<94bdd2610902151504n3862cfb0h706a7dcb31c51ba9@mail.gmail.com>
Message-ID: <4998D681.60905@verizon.net>

Tarek Ziad? wrote:
> On Sun, Feb 15, 2009 at 11:56 PM, Gerry Reno <greno at verizon.net> wrote:
>   
>> I've been looking at the spec file that 'bdist_rpm' generates and I do not
>> see a %config or %config(noreplace) section.  Is there a way to generate a
>> %config section and specify a list of files?  This is important so that
>> config files do not get overwritten when using 'rpm -U'.
>>     
>
> Looking at the code, I don't see such option,
>
> please fill a feature request in http://bugs.python.org/ selecting
> 'Distutils' as a component,
>
> (with a sample of such a spec file if possible)
>
> Thanks
>
>   
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>>     
>
>
>
>   
Ok, I found a way to use %config like this:

In setup.cfg define a line:
install_script=install.txt

In the file 'install.txt' I put this:

---------------------------------------------------
CONFIGFILES="\
%config(noreplace) /etc/myconfigfile.cfg"

echo "$CONFIGFILES" | cat INSTALLED_FILES - > INSTALLED_FILES.new
mv INSTALLED_FILES.new INSTALLED_FILES
---------------------------------------------------

Make sure your config file is added to the filelist fed to 'setup()' by 
setup.py
And this works.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090215/45466ac4/attachment.htm>

From chris at simplistix.co.uk  Mon Feb 16 14:38:49 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 16 Feb 2009 13:38:49 +0000
Subject: [Distutils] [buildout] how do I force an egg to rebuild?
Message-ID: <49996C69.8090902@simplistix.co.uk>

Hi All,

As a result of the Debian 5.0 release, psycopg2 needs to be recompiled 
to use the new shared library:

     import psycopg2 as psycopg
   File "build/bdist.linux-i686/egg/psycopg2/__init__.py", line 60, in 
<module>
   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 7, in 
<module>
   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 6, in 
__bootstrap__
ImportError: libpq.so.4: cannot open shared object file: No such file or 
directory


How do I tell buildout to recompile this egg?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From lists at zopyx.com  Mon Feb 16 15:03:44 2009
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 16 Feb 2009 15:03:44 +0100
Subject: [Distutils] [buildout] how do I force an egg to rebuild?
In-Reply-To: <49996C69.8090902@simplistix.co.uk>
References: <49996C69.8090902@simplistix.co.uk>
Message-ID: <49997240.3090904@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 16.02.2009 14:38 Uhr, Chris Withers wrote:
> Hi All,
> 
> As a result of the Debian 5.0 release, psycopg2 needs to be recompiled
> to use the new shared library:
> 
>     import psycopg2 as psycopg
>   File "build/bdist.linux-i686/egg/psycopg2/__init__.py", line 60, in
> <module>
>   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 7, in
> <module>
>   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 6, in
> __bootstrap__
> ImportError: libpq.so.4: cannot open shared object file: No such file or
> directory
> 
> 
> How do I tell buildout to recompile this egg?
> 
> 

By removing the old one first and reinstalling it?

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkmZckAACgkQCJIWIbr9KYz3ZACeKjpG1vVgEHDOfPoKMUYPxotM
s+UAn0YqRvhzwyOrHR6n0v2lmGjR7kA6
=NTyW
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090216/11361a35/attachment.vcf>

From chris at simplistix.co.uk  Mon Feb 16 15:06:29 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 16 Feb 2009 14:06:29 +0000
Subject: [Distutils] [buildout] how do I force an egg to rebuild?
In-Reply-To: <49997240.3090904@zopyx.com>
References: <49996C69.8090902@simplistix.co.uk> <49997240.3090904@zopyx.com>
Message-ID: <499972E5.6090200@simplistix.co.uk>

Andreas Jung wrote:
>>     import psycopg2 as psycopg
>>   File "build/bdist.linux-i686/egg/psycopg2/__init__.py", line 60, in
>> <module>
>>   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 7, in
>> <module>
>>   File "build/bdist.linux-i686/egg/psycopg2/_psycopg.py", line 6, in
>> __bootstrap__
>> ImportError: libpq.so.4: cannot open shared object file: No such file or
>> directory
>>
>>
>> How do I tell buildout to recompile this egg?
> 
> By removing the old one first and reinstalling it?

How do I do that? I'd guess at:

rm -rf /home/chris/buildout-eggs/psycopg2-2.0.8-py2.*

...but then the buildout's .installed.cfg won't know about that and so 
won't try to install anything.

Is there a "proper" way to do this, as the above all seems very hackish 
and unreliable...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Tue Feb 17 11:25:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Feb 2009 11:25:38 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
Message-ID: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>

Hello

I am starting to work on a script to generate a .deb package out of a buildout.

Anyone has done anything in this area already ? Or is interested in
working on it with me ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From strawman at astraw.com  Tue Feb 17 12:36:39 2009
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 17 Feb 2009 03:36:39 -0800
Subject: [Distutils] [ANN] stdeb 0.2.3 released
Message-ID: <499AA147.8060308@astraw.com>

Hot on the heels of stdeb 0.2.2 comes 0.2.3. More fantastic features for
this Python-package-to-debian-package automator, you ask? Only one, a
--ignore-install-requires option added by Brett (last name unknown).
More importantly, however, are a couple bugfixes from Brett and Zooko
O'Whielacronx for the new --autofind-depends option.

Get it while it, and your new Lenny installation, are hot. (It also
works for Ubuntu.)

The homepage: http://github.com/astraw/stdeb
Download: http://pypi.python.org/pypi/stdeb/0.2.3

-Andrew

From jim at zope.com  Tue Feb 17 13:08:20 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 17 Feb 2009 07:08:20 -0500
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
Message-ID: <BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>

We routinely use zc.sourcerelease to build RPMs.

Jim

On Feb 17, 2009, at 5:25 AM, Tarek Ziad? wrote:

> Hello
>
> I am starting to work on a script to generate a .deb package out of  
> a buildout.
>
> Anyone has done anything in this area already ? Or is interested in
> working on it with me ?
>
> Regards
> Tarek
>
> -- 
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Tue Feb 17 13:26:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Feb 2009 13:26:09 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
Message-ID: <94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>

Ok, I will check this approach,

Do you deliver Zope within the buildout or do you have a separate RPM for Zope,
then another RPM for each instance ?



2009/2/17 Jim Fulton <jim at zope.com>:
> We routinely use zc.sourcerelease to build RPMs.
>
> Jim
>
> On Feb 17, 2009, at 5:25 AM, Tarek Ziad? wrote:
>
>> Hello
>>
>> I am starting to work on a script to generate a .deb package out of a buildout.
>>
>> Anyone has done anything in this area already ? Or is interested in
>> working on it with me ?
>>
>> Regards
>> Tarek
>>
>> --
>> Tarek Ziad? | Association AfPy | www.afpy.org
>> Blog FR | http://programmation-python.org
>> Blog EN | http://tarekziade.wordpress.com/
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>
> --
> Jim Fulton
> Zope Corporation
>
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pjenvey at underboss.org  Tue Feb 17 20:49:57 2009
From: pjenvey at underboss.org (Philip Jenvey)
Date: Tue, 17 Feb 2009 11:49:57 -0800
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
Message-ID: <9B071CB6-EFC9-4624-B632-DB90DE250D5F@underboss.org>


On Feb 17, 2009, at 2:25 AM, Tarek Ziad? wrote:

> Hello
>
> I am starting to work on a script to generate a .deb package out of  
> a buildout.
>
> Anyone has done anything in this area already ? Or is interested in
> working on it with me ?


FYI Jeff Dairiki attempted a bdist_deb a few years ago:

http://bugs.python.org/issue1054967

--
Philip Jenvey


From zookog at gmail.com  Tue Feb 17 23:13:10 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Tue, 17 Feb 2009 15:13:10 -0700
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
Message-ID: <cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>

I use stdeb to build .deb's from Python source distributions.  I'm not
entirely sure what buildout does, but after letting this screencast
play in the background while I work:
http://rhodesmill.org/brandon/buildout , I think that I'm
accomplishing the same goal using setuptools "develop" command with a
"--prefix=" option.

I just tested the new stdeb 0.2.3 at the request of Andrew Straw, and
it was able to produce a .deb from the allmydata-tahoe source tree.  I
intend to configure some buildbots to automatically run stdeb to
produce .deb's of various of my projects: zfec, pycryptopp, pyutil,
etc.

Regards,

Zooko

From ziade.tarek at gmail.com  Tue Feb 17 23:23:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Feb 2009 23:23:04 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <9B071CB6-EFC9-4624-B632-DB90DE250D5F@underboss.org>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<9B071CB6-EFC9-4624-B632-DB90DE250D5F@underboss.org>
Message-ID: <94bdd2610902171423y238b7978u16697e2c20e73f8f@mail.gmail.com>

2009/2/17 Philip Jenvey <pjenvey at underboss.org>:
>
> On Feb 17, 2009, at 2:25 AM, Tarek Ziad? wrote:
>
>> Hello
>>
>> I am starting to work on a script to generate a .deb package out of a buildout.
>>
>> Anyone has done anything in this area already ? Or is interested in
>> working on it with me ?
>
>
> FYI Jeff Dairiki attempted a bdist_deb a few years ago:
>
> http://bugs.python.org/issue1054967

For zc.buildout package, which is more like a collection of packages,
I don't think it will fit,

but this bdist_deb command could really fit into Distutils, besides bdist_rpm,

I am going to re-activate this ticket,



>
> --
> Philip Jenvey
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Tue Feb 17 23:27:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 17 Feb 2009 23:27:20 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
Message-ID: <94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>

2009/2/17 Zooko O'Whielacronx <zookog at gmail.com>:
> I use stdeb to build .deb's from Python source distributions.  I'm not
> entirely sure what buildout does, but after letting this screencast
> play in the background while I work:
> http://rhodesmill.org/brandon/buildout , I think that I'm
> accomplishing the same goal using setuptools "develop" command with a
> "--prefix=" option.
>
> I just tested the new stdeb 0.2.3 at the request of Andrew Straw, and
> it was able to produce a .deb from the allmydata-tahoe source tree.  I
> intend to configure some buildbots to automatically run stdeb to
> produce .deb's of various of my projects: zfec, pycryptopp, pyutil,
> etc.
>

Same remark as bdist_deb : it looks nice to create .deb files for
single packages.

I am starting to look at Geoffrey T. Dairiki patch in Distutils, at
http://bugs.python.org/issue1054967

Maybe the stdeb team could work on this too ? I'll sen a mail to stdeb
maintainer

I can probably include such a command in Distutils for 2.7 if I get
help from Debian specialists

Regards
Tarek

From strawman at astraw.com  Wed Feb 18 00:20:50 2009
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 17 Feb 2009 15:20:50 -0800
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
	<94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
Message-ID: <499B4652.4000806@astraw.com>

Tarek Ziad? wrote:
> 2009/2/17 Zooko O'Whielacronx <zookog at gmail.com>:
>> I use stdeb to build .deb's from Python source distributions.  I'm not
>> entirely sure what buildout does, but after letting this screencast
>> play in the background while I work:
>> http://rhodesmill.org/brandon/buildout , I think that I'm
>> accomplishing the same goal using setuptools "develop" command with a
>> "--prefix=" option.
>>
>> I just tested the new stdeb 0.2.3 at the request of Andrew Straw, and
>> it was able to produce a .deb from the allmydata-tahoe source tree.  I
>> intend to configure some buildbots to automatically run stdeb to
>> produce .deb's of various of my projects: zfec, pycryptopp, pyutil,
>> etc.
>>
> 
> Same remark as bdist_deb : it looks nice to create .deb files for
> single packages.
> 
> I am starting to look at Geoffrey T. Dairiki patch in Distutils, at
> http://bugs.python.org/issue1054967
> 
> Maybe the stdeb team could work on this too ? I'll sen a mail to stdeb
> maintainer
> 

There is no need to email me separately. I usually lurk here...

If you're trying to do package management on Debian, I'd suggest using
the Debian system rather than trying to invent your own. (I am reading
between the lines here by noting that you are not talking about building
debian source packages, but only .debs. Please correct me if that
interpretation is wrong.)

I personally don't see the point in creating .deb packages without
actually generating a .dsc first -- you're just going to avoid Debian
machinery that helps make sure your .debs are OK. Furthermore, you have
some chance that your .dsc packages will work across debian/ubuntu
versions, whereas that chance is much reduced if you're using pure .deb
packages. The "benefit" of a straight .deb builder is that it could be
incredibly dumb and just build raw archives that get unpacked. I imagine
that would bypass Debian policy by unpacking everything in
/usr/lib/python2.5/site-packages. (Nowadays, the python-support
machinery in Debian unpacks files to /usr/share/pyshared and them
symlinks them across acceptable Python versions' site-package
directories). Finally, you'll miss out on all the script installation
and so on.

So, to me, the interesting discussion is not about auto-generation of
.debs. It's about auto-generation of .dscs. Those can trivially be
turned into .debs, anyway.

> I can probably include such a command in Distutils for 2.7 if I get
> help from Debian specialists

In case the above arguments persue you to reconsider something like
bdist_deb in favor of something like sdist_dsc, may I mention that this
is already a distutils command installed by stdeb?

However, I don't think stdeb is anywhere near ready for inclusion in the
stdlib. But I'd welcome help!

-Andrew

From ziade.tarek at gmail.com  Wed Feb 18 00:48:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 18 Feb 2009 00:48:59 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <499B4652.4000806@astraw.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
	<94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
	<499B4652.4000806@astraw.com>
Message-ID: <94bdd2610902171548r2d7706e7p238361bed9359692@mail.gmail.com>

2009/2/18 Andrew Straw <strawman at astraw.com>:
>
> There is no need to email me separately. I usually lurk here...

ok :)

>
> If you're trying to do package management on Debian, I'd suggest using
> the Debian system rather than trying to invent your own. (I am reading
> between the lines here by noting that you are not talking about building
> debian source packages, but only .debs. Please correct me if that
> interpretation is wrong.)

yes indeed,

But note that being able to build .deb packages from another system
than debian could
be a great feature when doable.

>
> I personally don't see the point in creating .deb packages without
> actually generating a .dsc first -- you're just going to avoid Debian
> machinery that helps make sure your .debs are OK. Furthermore, you have
> some chance that your .dsc packages will work across debian/ubuntu
> versions, whereas that chance is much reduced if you're using pure .deb
> packages. The "benefit" of a straight .deb builder is that it could be
> incredibly dumb and just build raw archives that get unpacked. I imagine
> that would bypass Debian policy by unpacking everything in
> /usr/lib/python2.5/site-packages. (Nowadays, the python-support
> machinery in Debian unpacks files to /usr/share/pyshared and them
> symlinks them across acceptable Python versions' site-package
> directories). Finally, you'll miss out on all the script installation
> and so on.
>
> So, to me, the interesting discussion is not about auto-generation of
> .debs. It's about auto-generation of .dscs. Those can trivially be
> turned into .debs, anyway.

Ok. I am clueless here. I need to read some documentation on my side,
But if this is comparable to the RPM spec files, it sounds like
a good approach.

What about two commands then ?

- sdist_deb (which is a sdist call + the .dsc file generation)
- bdist_deb (which is a sdist_deb call + the creation of the .deb)

>
>> I can probably include such a command in Distutils for 2.7 if I get
>> help from Debian specialists
>
> In case the above arguments persue you to reconsider something like
> bdist_deb in favor of something like sdist_dsc, may I mention that this
> is already a distutils command installed by stdeb?
>
> However, I don't think stdeb is anywhere near ready for inclusion in the
> stdlib. But I'd welcome help!

Well, looking at the sdist_dsc code, it is based on setuptools, so I
doubt it could
be integrated easily.

That said, I don't think the integration of a new command in Distutils itself,
is a huge amount of work, as long as it does one single thing.

If we could work on a simple isolated command
that builds the .dsc, then on another command that creates the .deb
out of it, it could
be the right approach imho.

What do you think ?

Regards
Tarek

From zooko at zooko.com  Wed Feb 18 00:49:48 2009
From: zooko at zooko.com (zooko)
Date: Tue, 17 Feb 2009 16:49:48 -0700
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
	<94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
Message-ID: <A92B85B0-164F-49C8-9671-11A88AC0A864@zooko.com>

On Feb 17, 2009, at 15:27 PM, Tarek Ziad? wrote:

> Same remark as bdist_deb : it looks nice to create .deb files for  
> single packages.

The way that I currently use it is that I run stdeb on each package  
which is required, and make them all available in an apt repository.   
This is certainly the way that Debian/Ubuntu users would expect --  
they would object if I produced a single .deb package containing  
multiple separate Python packages.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig


From strawman at astraw.com  Wed Feb 18 01:46:17 2009
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 17 Feb 2009 16:46:17 -0800
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <94bdd2610902171548r2d7706e7p238361bed9359692@mail.gmail.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>	
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>	
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>	
	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>	
	<94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>	
	<499B4652.4000806@astraw.com>
	<94bdd2610902171548r2d7706e7p238361bed9359692@mail.gmail.com>
Message-ID: <499B5A59.4060401@astraw.com>

Tarek Ziad? wrote:
> But note that being able to build .deb packages from another system
> than debian could
> be a great feature when doable.
>   
Yes, that's true. To produce nice .debs, though you'd have to
re-implement a lot of the Debian packaging tools. Probably it's easier
just to run a chroot and install them directly. (I guess this suggestion
would leave out the Windows users who want to generate .debs.)
>> So, to me, the interesting discussion is not about auto-generation of
>> .debs. It's about auto-generation of .dscs. Those can trivially be
>> turned into .debs, anyway.
>>     
>
> Ok. I am clueless here. I need to read some documentation on my side,
> But if this is comparable to the RPM spec files, it sounds like
> a good approach.
>   
I know very little about spec files. I understand that they can specify
conditional compilation based on the build system. So from one spec
file, RHEL 4 might execute different code and have different
dependencies than Fedora 10. Debian source packages don't do this -- you
have a source package uploaded to the Debian/Ubuntu repository specific
for e.g. Lenny or 8.10. The source packages *should* work if all their
dependencies are met and transferred to a different system, but you're
moving beyond the maintainer's responsibility to care. But, the upshot
is that one .dsc file (which is usually cryptographically signed--always
for the official repos--and definitely has hash checksums of the
original upstream tarball and the debian diffs) specifies one build
behavior for one set of dependencies and is typically targeted for one
Debian/Ubuntu distribution, but often works on more.

I'm more-or-less ignorant on all other aspects of .rpm generation. They
really annoyed me with their (lack of) dependency resolution back in the
late 90s, but I understand things have improved since then. :)
> What about two commands then ?
>
> - sdist_deb (which is a sdist call + the .dsc file generation)
>   
I guess you mean sdist_dsc?
> - bdist_deb (which is a sdist_deb call + the creation of the .deb)
>   
That command would be pretty easy, I think
('os.system("dpkg-buildpackage -rfakeroot -uc -b")').

It seems reasonable to do.

I personally prefer to keep the .dsc, .orig.tar.gz, and .diff.gz files
in case I want to build a binary for a new version of Debian/Ubuntu
(altogether the debian source package, which until now I've been
abbreviating as .dsc, but that's not quite true). But you knew that
already...
>   
>>> I can probably include such a command in Distutils for 2.7 if I get
>>> help from Debian specialists
>>>       
>> In case the above arguments persue you to reconsider something like
>> bdist_deb in favor of something like sdist_dsc, may I mention that this
>> is already a distutils command installed by stdeb?
>>
>> However, I don't think stdeb is anywhere near ready for inclusion in the
>> stdlib. But I'd welcome help!
>>     
>
> Well, looking at the sdist_dsc code, it is based on setuptools, so I
> doubt it could
> be integrated easily.
>   
True. I don't know how much is critical, and how much that is critical
is already in distutils.

> That said, I don't think the integration of a new command in Distutils itself,
> is a huge amount of work, as long as it does one single thing.
>   
> If we could work on a simple isolated command
> that builds the .dsc, then on another command that creates the .deb
> out of it, it could
> be the right approach imho.
>
> What do you think ?
>   
Well, there's not really a whole lot to stdeb other than what is needed
for the sdist_dsc command. There are two things that I think are
requirements before going forward with a proposal to include in distutils:

1) I'm under no illusions about the stdeb code -- it is pretty darn
ugly. I wouldn't want to be in the room if it was shown to the
python-dev team with a proposal to include in the stdlib in its current
state! :) I think a rewrite or major refactoring of stdeb would stand a
much better chance of getting in to the stdlib, and would be more
maintainable. Unfortunately, I don't have time for that...

2) Sample auto-generated source packages, particularly the debian/rules
files, should be critiqued by real Debian developers. Sending them to
the debian-python list, for example, would be a reasonable place to start.

-Andrew

From ziade.tarek at gmail.com  Wed Feb 18 01:46:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 18 Feb 2009 01:46:34 +0100
Subject: [Distutils] [zc.buildout] buildout to .deb package
In-Reply-To: <A92B85B0-164F-49C8-9671-11A88AC0A864@zooko.com>
References: <94bdd2610902170225u58c9ea66l2e19663dc289a04f@mail.gmail.com>
	<BBE58026-AB01-4F21-AE09-AE3228D22AE4@zope.com>
	<94bdd2610902170426x7c0bc93eh6f5258d2bb01d5b3@mail.gmail.com>
	<cd6401a0902171413s303ef766g3cddc10e7c8a10d9@mail.gmail.com>
	<94bdd2610902171427n7ca7436fs6a060734f597dc48@mail.gmail.com>
	<A92B85B0-164F-49C8-9671-11A88AC0A864@zooko.com>
Message-ID: <94bdd2610902171646p27adddb8k5d9313cdf58ea504@mail.gmail.com>

2009/2/18 zooko <zooko at zooko.com>:
> On Feb 17, 2009, at 15:27 PM, Tarek Ziad? wrote:
>
>> Same remark as bdist_deb : it looks nice to create .deb files for single packages.
>
> The way that I currently use it is that I run stdeb on each package which is required, and make them all available in an apt repository. > This is certainly the way that Debian/Ubuntu users would expect -- they would object if I produced a single .deb package containing
> multiple separate Python packages.

I think this might generate another long thread ... :)

Zope (my use case with zc.buildout) runs with hundreds of packages. In
fact, they all share a few namespaces (zope.*, plone.*)
so Zope can be considered as a set of a few "big" packages. When I
deploy a customer application, I add some specific
packages with the usual set. The set of packages needed to run my
customer app is built by zc.buildout,
which grabs them at pypi or elswhere, and gather them under the same directory.

zc.buildout, besides Python packages, provides some features to
generate console scripts and everything around
the packages to make an application run. So there's some extra work in
there that is not located in any python package.
(well it's an application)  The scripts generated by buildout hack
sys.path so they use the packages gathered by zc.buildout.

This mecanism is really useful for us Zope/Plone developers. For
example, some packages that are delivered in Zope 2 are now
delivered as single python packages (like zope.component,
zope.interface and so on). But the problem is :
if you have a Zope 2 running on a system, it contains a zope.* code
tree that is incompatible with the new components,
and having them installed in Python itself will break everything.

So basically, one Zope instance is now a directory containing
everything needed to run.

Making a .deb out of it for me means installing the different parts
the buildout is composed of in the right
places in the system tree, but also keep the collection of packages in
a separated directory, specific to this instance
and I would call this directory : my zope application.

Regards
Tarek

From mauriceling at gmail.com  Wed Feb 18 14:29:27 2009
From: mauriceling at gmail.com (Maurice Ling)
Date: Wed, 18 Feb 2009 21:29:27 +0800
Subject: [Distutils] Problems using easy_install to download my module
Message-ID: <499C0D37.8070303@acm.org>

Hi,

I've just registered 'copad' module with CheeseShop and tried to use 
easy_install to test but unsuccessful... witht he following error...

ML-iB:/sw/lib/python2.5/site-packages mauriceling$ sudo easy_install copads
Searching for copads
Reading http://pypi.python.org/simple/copads/
Reading http://www.sourceforge.net/projects/copads
No local packages or download links found for copads
error: Could not find suitable distribution for Requirement.parse('copads')

Here is my setup.py

from ez_setup import use_setuptools
use_setuptools()

from setuptools import setup, find_packages

setup(name='copads',
     version='0.1',
     description='Collection of Python Algorithms and Data Structures',
     long_description='Collection of Python Algorithms and Data 
Structures',
     author='Maurice HT Ling',
     author_email='mauriceling at acm.org',
     url='http://www.sourceforge.net/projects/copads',
     license = 'GNU General Public License version 2',
     platform = 'OS independent',
     package_dir = {'copads' : 'src',
                    'copads.test' : 'test'},
     packages = ['copads', 'copads.test'],
     classifiers=['Development Status :: 3 - Alpha',
                  'Intended Audience :: Developers',
                  'License :: OSI Approved :: GNU General Public License 
(GPL)',
                  'Operating System :: OS Independent',
                  'Programming Language :: Python',
                  'Topic :: Scientific/Engineering :: Artificial 
Intelligence',
                  'Topic :: Scientific/Engineering :: Mathematics',
                  'Topic :: Software Development :: Libraries :: Python 
Modules'
                  ],
    )


Any suggestions?

Thanks
ML

-- 
Maurice LING, BSc(Hons)(Biochem), BSc(Comp), FIFA, MACM 
Lecturer, Chemical and Life Sciences, Singapore Polytechnic
Co-Editor-in-Chief, The Python Papers Anthology
Firebird Foundation Committee Member
Secretary, University of Melbourne Alumni Association (Singapore) 
mobile: +6596669233, +6568707927
resume: http://maurice.vodien.com/maurice_resume.pdf
www: http://maurice.vodien.com


From mauriceling at gmail.com  Wed Feb 18 15:06:17 2009
From: mauriceling at gmail.com (Maurice Ling)
Date: Wed, 18 Feb 2009 22:06:17 +0800
Subject: [Distutils] Problems using easy_install to download my module
Message-ID: <499C15D9.7020306@acm.org>

Hi,

I've just registered 'copad' module with CheeseShop and tried to use 
easy_install to test but unsuccessful... witht he following error...

ML-iB:/sw/lib/python2.5/site-packages mauriceling$ sudo easy_install copads
Searching for copads
Reading http://pypi.python.org/simple/copads/
Reading http://www.sourceforge.net/projects/copads
No local packages or download links found for copads
error: Could not find suitable distribution for Requirement.parse('copads')

Here is my setup.py

from ez_setup import use_setuptools
use_setuptools()

from setuptools import setup, find_packages

setup(name='copads',
    version='0.1',
    description='Collection of Python Algorithms and Data Structures',
    long_description='Collection of Python Algorithms and Data Structures',
    author='Maurice HT Ling',
    author_email='mauriceling at acm.org',
    url='http://www.sourceforge.net/projects/copads',
    license = 'GNU General Public License version 2',
    platform = 'OS independent',
    package_dir = {'copads' : 'src',
                   'copads.test' : 'test'},
    packages = ['copads', 'copads.test'],
    classifiers=['Development Status :: 3 - Alpha',
                 'Intended Audience :: Developers',
                 'License :: OSI Approved :: GNU General Public License 
(GPL)',
                 'Operating System :: OS Independent',
                 'Programming Language :: Python',
                 'Topic :: Scientific/Engineering :: Artificial 
Intelligence',
                 'Topic :: Scientific/Engineering :: Mathematics',
                 'Topic :: Software Development :: Libraries :: Python 
Modules'
                 ],
   )


Any suggestions?

Thanks
ML

-- 
Maurice LING, BSc(Hons)(Biochem), BSc(Comp), FIFA, MACM 
Lecturer, Chemical and Life Sciences, Singapore Polytechnic
Co-Editor-in-Chief, The Python Papers Anthology
Firebird Foundation Committee Member
Secretary, University of Melbourne Alumni Association (Singapore) 
mobile: +6596669233, +6568707927
resume: http://maurice.vodien.com/maurice_resume.pdf
www: http://maurice.vodien.com


From pje at telecommunity.com  Wed Feb 18 16:34:17 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 18 Feb 2009 10:34:17 -0500
Subject: [Distutils] Problems using easy_install to download my  module
In-Reply-To: <499C15D9.7020306@acm.org>
References: <499C15D9.7020306@acm.org>
Message-ID: <20090218153209.429303A4074@sparrow.telecommunity.com>

Seems to work fine for me.  Perhaps you hadn't uploaded the source at 
that point?

If you want to use SourceForge's download system, you need to include 
a link to the files-listing page as your downloads URL, as 
easy_install doesn't follow arbitrary links, it just picks up 
*direct* download links on the pages specified as the URL and 
download URL of your package in setup.py.

At 10:06 PM 2/18/2009 +0800, Maurice Ling wrote:
>Hi,
>
>I've just registered 'copad' module with CheeseShop and tried to use 
>easy_install to test but unsuccessful... witht he following error...
>
>ML-iB:/sw/lib/python2.5/site-packages mauriceling$ sudo easy_install copads
>Searching for copads
>Reading http://pypi.python.org/simple/copads/
>Reading http://www.sourceforge.net/projects/copads
>No local packages or download links found for copads
>error: Could not find suitable distribution for Requirement.parse('copads')
>
>Here is my setup.py
>
>from ez_setup import use_setuptools
>use_setuptools()
>
>from setuptools import setup, find_packages
>
>setup(name='copads',
>    version='0.1',
>    description='Collection of Python Algorithms and Data Structures',
>    long_description='Collection of Python Algorithms and Data Structures',
>    author='Maurice HT Ling',
>    author_email='mauriceling at acm.org',
>    url='http://www.sourceforge.net/projects/copads',
>    license = 'GNU General Public License version 2',
>    platform = 'OS independent',
>    package_dir = {'copads' : 'src',
>                   'copads.test' : 'test'},
>    packages = ['copads', 'copads.test'],
>    classifiers=['Development Status :: 3 - Alpha',
>                 'Intended Audience :: Developers',
>                 'License :: OSI Approved :: GNU General Public 
> License (GPL)',
>                 'Operating System :: OS Independent',
>                 'Programming Language :: Python',
>                 'Topic :: Scientific/Engineering :: Artificial Intelligence',
>                 'Topic :: Scientific/Engineering :: Mathematics',
>                 'Topic :: Software Development :: Libraries :: 
> Python Modules'
>                 ],
>   )
>
>
>Any suggestions?
>
>Thanks
>ML
>
>--
>Maurice LING, BSc(Hons)(Biochem), BSc(Comp), FIFA, MACM Lecturer, 
>Chemical and Life Sciences, Singapore Polytechnic
>Co-Editor-in-Chief, The Python Papers Anthology
>Firebird Foundation Committee Member
>Secretary, University of Melbourne Alumni Association (Singapore) 
>mobile: +6596669233, +6568707927
>resume: http://maurice.vodien.com/maurice_resume.pdf
>www: http://maurice.vodien.com
>
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From pje at telecommunity.com  Thu Feb 19 16:31:30 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 19 Feb 2009 10:31:30 -0500
Subject: [Distutils] why does pkg_resources call imp.load_module?
In-Reply-To: <98ee68e30902190606g274b3c79l316a5e23356adaf@mail.gmail.com
 >
References: <98ee68e30902190606g274b3c79l316a5e23356adaf@mail.gmail.com>
Message-ID: <20090219152921.AFE453A4109@sparrow.telecommunity.com>

At 02:06 PM 2/19/2009 +0000, Karsten Petersen wrote:
>Hi,
>
>I'm trying to make Google App Engine work smoothly with setuptools 
>and I'm struggling to understand one specific piece of what 
>pkg_resources does.  Would you mind giving me a hint?
>
>The question is basically:  Why does _handle_ns() in pkg_resources 
>import other modules that implement the same namespace.  It's not 
>using the module handle that is returned.  I looked for side effects 
>but the only ones I found was that exceptions are raised if the 
>other modules are broken and that the modules __file__ var is 
>changed to point to the last found module that implements the 
>namespace in question.
>
>What am I missing?

In the unlikely event that one of the __init__.py's contains anything 
other than namespace setup code, loading the module ensures that it 
will be run.  This was done for compatibility with older packages 
that use distutils hacks to create a namespace package.  (i.e., 
shipping one package with the __init__.py, and the rest without 
it).  Such packages need to have their __init__.py code run.

It probably won't hurt anything too badly to leave it out, since it's 
strictly for compatibility with a configuration that's deprecated 
anyway.  (At least, it's deprecated for setuptools.)


From marius at pov.lt  Thu Feb 19 18:53:44 2009
From: marius at pov.lt (Marius Gedminas)
Date: Thu, 19 Feb 2009 19:53:44 +0200
Subject: [Distutils] Problems using easy_install to download my  module
In-Reply-To: <20090218153209.429303A4074@sparrow.telecommunity.com>
References: <499C15D9.7020306@acm.org>
	<20090218153209.429303A4074@sparrow.telecommunity.com>
Message-ID: <20090219175343.GA15577@fridge.pov.lt>

On Wed, Feb 18, 2009 at 10:34:17AM -0500, P.J. Eby wrote:
> Seems to work fine for me.  Perhaps you hadn't uploaded the source at 
> that point?
> 
> If you want to use SourceForge's download system,

/me cringes from the memory

IMHO using 'python setup.py sdist register upload' for releasing &
uploading Python packages is most convenient both for the author, and
for the users.

Marius Gedminas
-- 
The moral of the story is that with a contrived example, you can prove
anything. Oops. No, that's not what I meant to say.
		-- Joel Spolski
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090219/997b52ef/attachment.pgp>

From chch at s-plus.in  Fri Feb 20 04:30:01 2009
From: chch at s-plus.in (Andrey Chichak)
Date: Fri, 20 Feb 2009 09:30:01 +0600
Subject: [Distutils] setuptools/command/sdist.py small fix
Message-ID: <499E23B9.304@s-plus.in>

Good day!

small fix for setuptools/command/sdist.py:

*** sdist.py    Thu Feb 19 19:34:31 2009
--- sdist.py.ORIG    Thu Feb 19 19:33:21 2009
***************
*** 2,4 ****
  from distutils.util import convert_path
- from distutils import log
  import os, re, sys, pkg_resources
--- 2,3 ----

-- 
Andrey Chichak
JID: chch at ru-pro.com
ICQ: 13154894


From sgwoodjr at gmail.com  Sat Feb 21 23:06:51 2009
From: sgwoodjr at gmail.com (Simon Wood)
Date: Sat, 21 Feb 2009 17:06:51 -0500
Subject: [Distutils] Subclassing the compiler
Message-ID: <e3c7d2ba0902211406x462fe125s79d7b891bca11eb@mail.gmail.com>

Hi,

The API reference for distutils tells how to get an instance of a new
compiler sub-class with the following command:

mycompiler = distutils.ccompiler.new_compiler(*plat=None*, *compiler=None*,
*verbose=0*, *dry_run=0*, *force=0*)

The API also specifies the commands to modify specific parameters within the
returned compiler sub-class. However the API says nothing about how to
actually "use" the new sub-class. In other words, how does one go about
getting the following to work (or something similar)?

python setup.py build

where the build steps now use 'mycompiler'

Thanks,

-Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090221/c66a1cbb/attachment.htm>

From ziade.tarek at gmail.com  Sat Feb 21 23:37:46 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 21 Feb 2009 23:37:46 +0100
Subject: [Distutils] Subclassing the compiler
In-Reply-To: <e3c7d2ba0902211406x462fe125s79d7b891bca11eb@mail.gmail.com>
References: <e3c7d2ba0902211406x462fe125s79d7b891bca11eb@mail.gmail.com>
Message-ID: <94bdd2610902211437i65836183x1f31cdf8675ef78@mail.gmail.com>

Hi,

You need to add your compiler into the ccompiler.compiler_class dict,
then specify the compiler type in the metadata in setup.py, with the
compiler option.

Regards
Tarek

On Sat, Feb 21, 2009 at 11:06 PM, Simon Wood <sgwoodjr at gmail.com> wrote:
> Hi,
>
> The API reference for distutils tells how to get an instance of a new
> compiler sub-class with the following command:
>
> mycompiler = distutils.ccompiler.new_compiler(plat=None, compiler=None,
> verbose=0, dry_run=0, force=0)
>
> The API also specifies the commands to modify specific parameters within the
> returned compiler sub-class. However the API says nothing about how to
> actually "use" the new sub-class. In other words, how does one go about
> getting the following to work (or something similar)?
>
> python setup.py build
>
> where the build steps now use 'mycompiler'
>
> Thanks,
>
> -Simon
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From sgwoodjr at gmail.com  Sat Feb 21 23:57:59 2009
From: sgwoodjr at gmail.com (Simon Wood)
Date: Sat, 21 Feb 2009 17:57:59 -0500
Subject: [Distutils] Subclassing the compiler
In-Reply-To: <94bdd2610902211437i65836183x1f31cdf8675ef78@mail.gmail.com>
References: <e3c7d2ba0902211406x462fe125s79d7b891bca11eb@mail.gmail.com>
	<94bdd2610902211437i65836183x1f31cdf8675ef78@mail.gmail.com>
Message-ID: <e3c7d2ba0902211457q53fb44a5p5d555a916344f0ab@mail.gmail.com>

Excellent!!! that did the trick.

On Sat, Feb 21, 2009 at 5:37 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> You need to add your compiler into the ccompiler.compiler_class dict,
> then specify the compiler type in the metadata in setup.py, with the
> compiler option.
>
> Regards
> Tarek
>
> On Sat, Feb 21, 2009 at 11:06 PM, Simon Wood <sgwoodjr at gmail.com> wrote:
> > Hi,
> >
> > The API reference for distutils tells how to get an instance of a new
> > compiler sub-class with the following command:
> >
> > mycompiler = distutils.ccompiler.new_compiler(plat=None, compiler=None,
> > verbose=0, dry_run=0, force=0)
> >
> > The API also specifies the commands to modify specific parameters within
> the
> > returned compiler sub-class. However the API says nothing about how to
> > actually "use" the new sub-class. In other words, how does one go about
> > getting the following to work (or something similar)?
> >
> > python setup.py build
> >
> > where the build steps now use 'mycompiler'
> >
> > Thanks,
> >
> > -Simon
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > http://mail.python.org/mailman/listinfo/distutils-sig
> >
> >
>
>
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090221/db3fca7a/attachment.htm>

From ziade.tarek at gmail.com  Sun Feb 22 03:37:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 22 Feb 2009 03:37:50 +0100
Subject: [Distutils] PEP 376 for Distutils
Message-ID: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>

Hello,

I have started a PEP for Distutils, I would like to work out for Pycon.

http://svn.python.org/projects/peps/trunk/pep-0376.txt

It proposes some changes on the egg-info format, and various other enhancements,

It's an early draft and any help/feedback is welcome !

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Sun Feb 22 04:30:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 21 Feb 2009 22:30:26 -0500
Subject: [Distutils] Distutils 2.6.2 changes will break setuptools
In-Reply-To: <94bdd2610902211455t5d599535la996c8b1485e6e3@mail.gmail.com
 >
References: <94bdd2610902211455t5d599535la996c8b1485e6e3@mail.gmail.com>
Message-ID: <20090222032816.B57983A4074@sparrow.telecommunity.com>

At 11:55 PM 2/21/2009 +0100, Tarek Ziad? wrote:
>Hi Phillip,
>
>Some changes in distutils/sdist are breaking some commands in
>setuptools' egg_info because of a getattr that make a recursive error,
>
>I think it could be a great thing to fix it as soon as possible before
>Python 2.6.1 is out,
>
>I am available if you need more info on this

At a bare minimum, the Python SVN revision numbers and the resulting 
traceback would be helpful.  A bug filed to the setuptools tracker 
would probably also be a good idea, so that I'm not the only person 
who can look at it or provide a fix...  either for setuptools or the 
stdlib, as appropriate.


From ziade.tarek at gmail.com  Sun Feb 22 04:50:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 22 Feb 2009 04:50:09 +0100
Subject: [Distutils] Distutils 2.6.2 changes will break setuptools
In-Reply-To: <20090222032816.B57983A4074@sparrow.telecommunity.com>
References: <94bdd2610902211455t5d599535la996c8b1485e6e3@mail.gmail.com>
	<20090222032816.B57983A4074@sparrow.telecommunity.com>
Message-ID: <94bdd2610902211950s7a327199y20366a5689bb686c@mail.gmail.com>

On Sun, Feb 22, 2009 at 4:30 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 11:55 PM 2/21/2009 +0100, Tarek Ziad? wrote:
>>
>> Hi Phillip,
>>
>> Some changes in distutils/sdist are breaking some commands in
>> setuptools' egg_info because of a getattr that make a recursive error,
>>
>> I think it could be a great thing to fix it as soon as possible before
>> Python 2.6.1 is out,

Typo : before the first version of 2.7 is out

Notice that standalone releases of Distutils are planned soon, including a
development snapshots, so it should be easier to test the latest changes.

>>
>> I am available if you need more info on this
>
> At a bare minimum, the Python SVN revision numbers and the resulting
> traceback would be helpful.  A bug filed to the setuptools tracker would
> probably also be a good idea, so that I'm not the only person who can look
> at it or provide a fix...  either for setuptools or the stdlib, as
> appropriate.
>

Sure,

it happens because setuptools build_py implementation has a __getattr__ that
computes data_file::

  def __getattr__(self,attr):
        if attr=='data_files':  # lazily compute data files
            self.data_files = files = self._get_data_files(); return files
        return _build_py.__getattr__(self,attr)

But Distutils sdist command loops over build_py.data_files to add them
in the MANIFEST file,
because this attribute should be computed by finalize_command.

And this is done when sdist.add_default is called, so it loops
recursively and dies

see "for pkg, src_dir, build_dir, filenames in build_py.data_files: in"

http://svn.python.org/view/python/trunk/Lib/distutils/command/sdist.py?r1=68951&r2=69692

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Sun Feb 22 05:13:46 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 21 Feb 2009 23:13:46 -0500
Subject: [Distutils] Distutils 2.6.2 changes will break setuptools
In-Reply-To: <94bdd2610902211950s7a327199y20366a5689bb686c@mail.gmail.co
 m>
References: <94bdd2610902211455t5d599535la996c8b1485e6e3@mail.gmail.com>
	<20090222032816.B57983A4074@sparrow.telecommunity.com>
	<94bdd2610902211950s7a327199y20366a5689bb686c@mail.gmail.com>
Message-ID: <20090222041137.A29723A4074@sparrow.telecommunity.com>

At 04:50 AM 2/22/2009 +0100, Tarek Ziad? wrote:
>On Sun, Feb 22, 2009 at 4:30 AM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 11:55 PM 2/21/2009 +0100, Tarek Ziad? wrote:
> >>
> >> Hi Phillip,
> >>
> >> Some changes in distutils/sdist are breaking some commands in
> >> setuptools' egg_info because of a getattr that make a recursive error,
> >>
> >> I think it could be a great thing to fix it as soon as possible before
> >> Python 2.6.1 is out,
>
>Typo : before the first version of 2.7 is out
>
>Notice that standalone releases of Distutils are planned soon, including a
>development snapshots, so it should be easier to test the latest changes.
>
> >>
> >> I am available if you need more info on this
> >
> > At a bare minimum, the Python SVN revision numbers and the resulting
> > traceback would be helpful.  A bug filed to the setuptools tracker would
> > probably also be a good idea, so that I'm not the only person who can look
> > at it or provide a fix...  either for setuptools or the stdlib, as
> > appropriate.
> >
>
>Sure,
>
>it happens because setuptools build_py implementation has a __getattr__ that
>computes data_file::
>
>   def __getattr__(self,attr):
>         if attr=='data_files':  # lazily compute data files
>             self.data_files = files = self._get_data_files(); return files
>         return _build_py.__getattr__(self,attr)
>
>But Distutils sdist command loops over build_py.data_files to add them
>in the MANIFEST file,
>because this attribute should be computed by finalize_command.
>
>And this is done when sdist.add_default is called, so it loops
>recursively and dies
>
>see "for pkg, src_dir, build_dir, filenames in build_py.data_files: in"
>
>http://svn.python.org/view/python/trunk/Lib/distutils/command/sdist.py?r1=68951&r2=69692

I'm not sure this problem is fixable on the setuptools side -- the 
calculation was made lazy specifically to *avoid* this problem!  That 
is, I intentionally prevented Python 2.4+ distutils from calculating 
the data files during option finalization, because the data files 
can't be correctly known until *after* the manifest is built.

I have no idea what to do about this -- on either setuptools or 
distutils.  For setuptools, file list calculation is inherently 
circular - an existing manifest is always reused and pruned, in order 
to support situations where a revctrl-based file list is then used in 
a scenario without revision control.  (That is, the MANIFEST included 
with a shipped sdist is used as a backup for identifying files that 
might originally have been found via a revctrl finder.)


From ziade.tarek at gmail.com  Sun Feb 22 14:35:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 22 Feb 2009 14:35:24 +0100
Subject: [Distutils] Distutils 2.6.2 changes will break setuptools
In-Reply-To: <20090222041137.A29723A4074@sparrow.telecommunity.com>
References: <94bdd2610902211455t5d599535la996c8b1485e6e3@mail.gmail.com>
	<20090222032816.B57983A4074@sparrow.telecommunity.com>
	<94bdd2610902211950s7a327199y20366a5689bb686c@mail.gmail.com>
	<20090222041137.A29723A4074@sparrow.telecommunity.com>
Message-ID: <94bdd2610902220535m3235a9e8s29f6307bf56e6b9a@mail.gmail.com>

On Sun, Feb 22, 2009 at 5:13 AM, P.J. Eby <pje at telecommunity.com> wrote:
>
> I have no idea what to do about this -- on either setuptools or distutils.
>  For setuptools, file list calculation is inherently circular - an existing
> manifest is always reused and pruned, in order to support situations where a
> revctrl-based file list is then used in a scenario without revision control.
>  (That is, the MANIFEST included with a shipped sdist is used as a backup
> for identifying files that might originally have been found via a revctrl
> finder.)

For the revctrl part, can't setuptools build_py become an isolated command,
run after the manifest is currently written by sdist ? some kind of "build_vcs"
that would revise the file list in a second phase ?


>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sun Feb 22 22:21:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 22 Feb 2009 22:21:47 +0100
Subject: [Distutils] bdist_rpm new option
Message-ID: <94bdd2610902221321l15d9d82t2cd9df658cf8400a@mail.gmail.com>

Hello

We have worked on a fix for bdist_rpm, to avoid the error that occurs
under Fedora or RedHat when the optimize flag is not
activated.

http://bugs.python.org/issue1533164

I am about to introduce a new option for this command, called
"force-optimize" (and activated by default)

This new option makes the bdist_rpm command drive the install command
with the "optimize" flag activated by default (this fixes the problem)

The rationale to introduce a new option is to make sure people don't
get confused between the optimize flag that is forced when building a
rpm package,
and the "regular" optimize option used by the install command.

This will prevent undesired side-effects. For instance, writing a
setup.cfg with "optimize" to build rpms, might interfer when someone
tries to
tries to install the same package under Jython,

Any opinion ? Otherwise I'll commit it this way

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From greno at verizon.net  Mon Feb 23 14:23:24 2009
From: greno at verizon.net (Gerry Reno)
Date: Mon, 23 Feb 2009 08:23:24 -0500
Subject: [Distutils] bdist_rpm new option
In-Reply-To: <94bdd2610902221321l15d9d82t2cd9df658cf8400a@mail.gmail.com>
References: <94bdd2610902221321l15d9d82t2cd9df658cf8400a@mail.gmail.com>
Message-ID: <49A2A34C.2000206@verizon.net>

Tarek Ziad? wrote:
> Hello
>
> We have worked on a fix for bdist_rpm, to avoid the error that occurs
> under Fedora or RedHat when the optimize flag is not
> activated.
>
> http://bugs.python.org/issue1533164
>
> I am about to introduce a new option for this command, called
> "force-optimize" (and activated by default)
>
>   
Hopefully this will eliminate the current need to create separate 
install-scripts with lines like this which we are having to do right now:
python setup.py install --optimize 1 --root=$RPM_BUILD_ROOT 
--record=INSTALLED_FILES

Regards,
Gerry


From ziade.tarek at gmail.com  Mon Feb 23 14:25:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Feb 2009 14:25:19 +0100
Subject: [Distutils] bdist_rpm new option
In-Reply-To: <49A2A34C.2000206@verizon.net>
References: <94bdd2610902221321l15d9d82t2cd9df658cf8400a@mail.gmail.com>
	<49A2A34C.2000206@verizon.net>
Message-ID: <94bdd2610902230525s78acbeddt3b55aafff38b0b2e@mail.gmail.com>

On Mon, Feb 23, 2009 at 2:23 PM, Gerry Reno <greno at verizon.net> wrote:
> Tarek Ziad? wrote:
>>
>> Hello
>>
>> We have worked on a fix for bdist_rpm, to avoid the error that occurs
>> under Fedora or RedHat when the optimize flag is not
>> activated.
>>
>> http://bugs.python.org/issue1533164
>>
>> I am about to introduce a new option for this command, called
>> "force-optimize" (and activated by default)
>>
>>
>
> Hopefully this will eliminate the current need to create separate
> install-scripts with lines like this which we are having to do right now:
> python setup.py install --optimize 1 --root=$RPM_BUILD_ROOT
> --record=INSTALLED_FILES

Yes that's the fix indeed. bdist_rpm is now doing :

python setup.py install -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES

>
> Regards,
> Gerry
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From setuptools at bugs.python.org  Mon Feb 23 20:47:51 2009
From: setuptools at bugs.python.org (do3cc)
Date: Mon, 23 Feb 2009 19:47:51 +0000
Subject: [Distutils] [issue60] Workaround for a distutils bug
In-Reply-To: <1235418471.75.0.804306528307.issue60@psf.upfronthosting.co.za>
Message-ID: <1235418471.75.0.804306528307.issue60@psf.upfronthosting.co.za>


New submission from do3cc <do3ccqrv at googlemail.com>:

There is a missing dependency in the install_lib command:
http://bugs.python.org/issue5243

As a result, some eggs can not be installed. Reportlab for example

attached is a possible workaround. apply with patch -p0 < bdist_egg.diff

----------
files: bdist_egg.diff
messages: 242
nosy: do3cc
priority: wish
status: unread
title: Workaround for a distutils bug
Added file: http://bugs.python.org/setuptools/file40/bdist_egg.diff

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue60>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bdist_egg.diff
Type: application/octet-stream
Size: 745 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090223/fe6608d0/attachment.obj>

From setuptools at bugs.python.org  Mon Feb 23 22:28:38 2009
From: setuptools at bugs.python.org (Miroslav Madecki)
Date: Mon, 23 Feb 2009 21:28:38 +0000
Subject: [Distutils] [issue61] bdist_egg command doesn't stop after
	byte-compile errors
In-Reply-To: <1235424517.99.0.11846749373.issue61@psf.upfronthosting.co.za>
Message-ID: <1235424517.99.0.11846749373.issue61@psf.upfronthosting.co.za>


New submission from Miroslav Madecki <miro at a-bb.net>:

I'm using setuptools 06c9 on CentOs 5.X i386/x86_64.

When using bdist_egg command one would expect setuptools to stop
once byte-compile encounters an error
(e.g. python source file with syntax errors) and would also return
exit value other than 0 indicating a failure.

Unfortunately neither is the case so if you use setuptools within makefiles
it won't stop make in the case of errors.

I'm surprised that nobody else reported this issue.

Example is attached below.

-Miro


[miro at t42 monty]$ python25 ipdrfc-setup.py bdist_egg
.....
byte-compiling build/bdist.linux-i686/egg/abn/ipdrfc/cfg.py to cfg.pyc
  File "build/bdist.linux-i686/egg/abn/ipdrfc/cfg.py", line 184
    defset_exporter(ipAddr, cfg, RexGroup=None, RexListenPort=None, LogLevel=None):
                                                                                  ^
SyntaxError: invalid syntax

byte-compiling build/bdist.linux-i686/egg/abn/ipdrfc/reader.py to reader.pyc
byte-compiling build/bdist.linux-i686/egg/abn/ipdrfc/sshconsole.py to sshconsole.pyc
byte-compiling build/bdist.linux-i686/egg/abn/ipdrfc/backup.py to backup.pyc
creating build/bdist.linux-i686/egg/EGG-INFO
copying src/abn_ipdrfc.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO
copying src/abn_ipdrfc.egg-info/SOURCES.txt -> build/bdist.linux-i686/egg/EGG-INFO
copying src/abn_ipdrfc.egg-info/dependency_links.txt ->
build/bdist.linux-i686/egg/EGG-INFO
copying src/abn_ipdrfc.egg-info/namespace_packages.txt ->
build/bdist.linux-i686/egg/EGG-INFO
copying src/abn_ipdrfc.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/abn_ipdrfc-3.2.0-py2.5.egg' and adding
'build/bdist.linux-i686/egg' to it
removing 'build/bdist.linux-i686/egg' (and everything under it)
[miro at t42 monty]$ echo $?
0

----------
messages: 243
nosy: miro
priority: urgent
status: unread
title: bdist_egg command doesn't stop after byte-compile errors

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue61>
_______________________________________________

From floris.bruynooghe at gmail.com  Tue Feb 24 00:13:10 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 23 Feb 2009 23:13:10 +0000
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
Message-ID: <20090223231310.GA5711@laurie.devork>

Hello Tarek

On Sun, Feb 22, 2009 at 03:37:50AM +0100, Tarek Ziad? wrote:
> I have started a PEP for Distutils, I would like to work out for Pycon.
> 
> http://svn.python.org/projects/peps/trunk/pep-0376.txt

Looks quite nice so far IMHO.  Here some early feedback...


"""
Back to our `zlib` example, we will have::

    - zlib
    - zlib-2.5.2-py2.6.egg-info/
        PKG-INFO
        MANIFEST
        RECORD
"""

What is the rationale for the `-2.5.2-py2.6' part in the name of the
.egg-info directory?  I'm not saying I'm against it, rather that I'd
like to see a rationale for why it is there and why it is different
from the current distutils bit (doesn't add the python version, only
package version).

Related to this how is the behaviour of pkgutil.get_egg_info() defined
in respect to this?  It mentions that all `pkg_name*.egg-info' files
are looked for and None is returned if none is found.  But no word on
what happens if more then one is found.  (Should that be an error?
Guess it depends on the anser to the previous paragraph.)


"""
Let's use it over our `zlib` example::

    >>> from pkgutil import get_egg_info, get_metadata,
        get_egg_info_file
    >>> get_egg_info('zlib')
    '/opt/local/lib/python2.6/site-packages/zlib-2.5.2-py2.6.egg-info'
    >>> metadata = get_metadata('zlib')
    >>> metadata.version
    '2.5.2'
    >>> from distutils.dist import EGG_INFO_FILES
    >>> get_metadata('zlib', EGG_INFO_FILES.manifest).read()
    some
    ...
    files
"""

I think there's a typo in the last call here.  Should that not be
`get_egg_info_file('zlib', EGG_INFO_FILES.manifest).read()'?


Finally about the install and uninstall script.  Is this going to
attempt to address the namespace packages problem or will this be left
for a later PEP?  AIUI if it doesn't (which seems reasonable,
distutils never could do this) but setuptools does install such a
package then distutils will happily try to remove it, using the
.egg-info/RECORD file.  But this might then mess up any uninstall
attempt from setuptools since a shared __init__.py might have been
removed.


Hope this was helpful in some way.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Tue Feb 24 01:50:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 24 Feb 2009 01:50:51 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090223231310.GA5711@laurie.devork>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
Message-ID: <94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>

On Tue, Feb 24, 2009 at 12:13 AM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> Hello Tarek
>
> On Sun, Feb 22, 2009 at 03:37:50AM +0100, Tarek Ziad? wrote:
>> I have started a PEP for Distutils, I would like to work out for Pycon.
>>
>> http://svn.python.org/projects/peps/trunk/pep-0376.txt
>
> Looks quite nice so far IMHO. ?Here some early feedback...
>
>
> """
> Back to our `zlib` example, we will have::
>
> ? ?- zlib
> ? ?- zlib-2.5.2-py2.6.egg-info/
> ? ? ? ?PKG-INFO
> ? ? ? ?MANIFEST
> ? ? ? ?RECORD
> """
>
> What is the rationale for the `-2.5.2-py2.6' part in the name of the
> .egg-info directory? ?I'm not saying I'm against it, rather that I'd
> like to see a rationale for why it is there and why it is different
> from the current distutils bit (doesn't add the python version, only
> package version).

If you look at install_egg_info, it will add the Python version

http://svn.python.org/projects/python/trunk/Lib/distutils/command/install_egg_info.py

I am not sure either this should be kept. I don't see the rationale
either, since
sys.version is known at runtime, it seems superfluous.

Maybe it should be deprecated.

I having the same problem with the version : since it is already
located in PKG-INFO,
there's no need to have it in the folder name;

So maybe the final version could be:

    - zlib
    - zlib.egg-info/
        PKG-INFO
        MANIFEST
        RECORD

>
> Related to this how is the behaviour of pkgutil.get_egg_info() defined
> in respect to this? ?It mentions that all `pkg_name*.egg-info' files
> are looked for and None is returned if none is found. ?But no word on
> what happens if more then one is found. ?(Should that be an error?
> Guess it depends on the anser to the previous paragraph.)

In Distutils, there should be one egg-info per package. (While
setuptools multi-package
options allows to activate/deactivate various versions of the same package).

That said, I don't think this PEP should address multiple versions issues.

I'll add some details about this.

>
>
> """
> Let's use it over our `zlib` example::
>
> ? ?>>> from pkgutil import get_egg_info, get_metadata,
> ? ? ? ?get_egg_info_file
> ? ?>>> get_egg_info('zlib')
> ? ?'/opt/local/lib/python2.6/site-packages/zlib-2.5.2-py2.6.egg-info'
> ? ?>>> metadata = get_metadata('zlib')
> ? ?>>> metadata.version
> ? ?'2.5.2'
> ? ?>>> from distutils.dist import EGG_INFO_FILES
> ? ?>>> get_metadata('zlib', EGG_INFO_FILES.manifest).read()
> ? ?some
> ? ?...
> ? ?files
> """
>
> I think there's a typo in the last call here. ?Should that not be
> `get_egg_info_file('zlib', EGG_INFO_FILES.manifest).read()'?
>

right,

>
> Finally about the install and uninstall script. ?Is this going to
> attempt to address the namespace packages problem or will this be left
> for a later PEP? AIUI if it doesn't (which seems reasonable,
> distutils never could do this) but setuptools does install such a
> package then distutils will happily try to remove it, using the
> .egg-info/RECORD file. ?But this might then mess up any uninstall
> attempt from setuptools since a shared __init__.py might have been
> removed.

Right.  I think we can try to address namespace packages in another
PEP, because as far as I
can see, the namespace package boilerplate in setuptools is using
pkgutil.extend_path which
fixes __path__ variables on the fly. So I can't think of a case where
a static __init__.py file
will be shared, thus removed.


>
>
> Hope this was helpful in some way.

Thanks for the feedback Floris, I'll update the PEP accordingly

>
> Regards
> Floris
>
> --
> Debian GNU/Linux -- The Power of Freedom
> www.debian.org | www.gnu.org | www.kernel.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Tue Feb 24 03:53:17 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Feb 2009 21:53:17 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com
 >
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
Message-ID: <20090224025132.5D3003A409E@sparrow.telecommunity.com>

At 01:50 AM 2/24/2009 +0100, Tarek Ziad? wrote:
>I having the same problem with the version : since it is already
>located in PKG-INFO,
>there's no need to have it in the folder name;

It's there so pkg_resources doesn't need to read the file in order to 
locate an available version of the package, just the listdir() that 
it was doing anyway.  In other words, it's a performance 
optimization, and a pretty major one when you're building an 
Environment object to look for available package versions.  Cuts down 
on a LOT of file opens in the case where you have a ton of 
.egg-info's in site-packages, like on a Linux system with Python 2.5.

Now, it is supported to leave that information out, but that's really 
intended only for working with development packages/checkouts, where 
you don't have so many of them on sys.path at any one time that the 
extra open is a big deal.


>In Distutils, there should be one egg-info per package. (While
>setuptools multi-package
>options allows to activate/deactivate various versions of the same package).
>
>That said, I don't think this PEP should address multiple versions issues.

Note that sys.path could have more than one version of the same 
package on it, in two different directories.  (Granted, one would 
take precedence over the other, but that should still be explicit.)


>PEP, because as far as I
>can see, the namespace package boilerplate in setuptools is using
>pkgutil.extend_path which
>fixes __path__ variables on the fly. So I can't think of a case where
>a static __init__.py file
>will be shared, thus removed.

When a distutils package does it.  I'm not positive, but if 'pip' 
supports namespace packages without using .pth files, then it has to 
use a shared __init__ also.

And in the long run, easy_install will do this too.

So, the uninstallation code should simply not remove file(s) that are 
referenced by more than one manifest in the target directory -- a 
relatively simple, future-proof safeguard, that doesn't require any 
specific knowledge of "namespace packages" per se.


From him at online.de  Tue Feb 24 09:39:18 2009
From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=)
Date: Tue, 24 Feb 2009 09:39:18 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
Message-ID: <49A3B236.4050001@online.de>

Tarek Ziad? wrote:
> If you look at install_egg_info, it will add the Python version
>
> http://svn.python.org/projects/python/trunk/Lib/distutils/command/install_egg_info.py
>
> I am not sure either this should be kept. I don't see the rationale
> either, since
> sys.version is known at runtime, it seems superfluous.
>
> Maybe it should be deprecated.
>
> I having the same problem with the version : since it is already
> located in PKG-INFO,
> there's no need to have it in the folder name;
>
> So maybe the final version could be:
>
>     - zlib
>     - zlib.egg-info/
>         PKG-INFO
>         MANIFEST
>         RECORD
>   
And while we are at it:

could the egg-info directory be put somewhere else (as a 
configuration/command line option)?

It's meta information about the packages and not python code. And when 
doing a simple 'ls'
on the site-packages folders I woul like to see the packages only and 
not 2 entries per
package.

An other option could be to put the egg-info dir into the package 
itself, e.g.

zlib/
    __init__.py
    egg-info/
        PKG-INFO
        MANIFEST
        RECORD
   ...

P.J. Eby wrote about the encoding of package and python version inside 
the egg-info directory name:
> It's there so pkg_resources doesn't need to read the file in order to 
> locate an available version of the package, just the listdir() that it 
> was doing anyway.  In other words, it's a performance optimization, 
> and a pretty major one when you're building an Environment object to 
> look for available package versions.  Cuts down on a LOT of file opens 
> in the case where you have a ton of .egg-info's in site-packages, like 
> on a Linux system with Python 2.5.

If the location of the egg-info dir and the encoding of the python 
version and package
version are only there for performance optimization reasons I'd suggest 
to really decouple
optimization from file naming and having a caching directory that can 
compute an
optimized representation once when an out-of-date situation is detected 
(e.g. when new
packages are installed) making the optimization even faster, e.g. by 
having a file
in suitable format instead of calling os.listdir() and iterating over 
the result.






From ziade.tarek at gmail.com  Tue Feb 24 13:33:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 24 Feb 2009 13:33:41 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <49A3B236.4050001@online.de>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
Message-ID: <94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>

Philip wrote:
> When a distutils package does it.  I'm not positive, but if 'pip' supports namespace packages without using .pth files, then it has to use a shared __init__ also.
>
> And in the long run, easy_install will do this too.
>
> So, the uninstallation code should simply not remove file(s) that are referenced by more than one manifest in the target directory -- a relatively simple, future-proof safeguard, that doesn't require any specific knowledge of "namespace packages" per se.
>

Sounds good. Although, it requires scanning the files again which is
not optimal, but the last point of this mail might be an idea to
adress this.

2009/2/24 Joachim K?nig <him at online.de>:
> An other option could be to put the egg-info dir into the package itself, e.g.
>
> zlib/
> ? __init__.py
> ? egg-info/
> ? ? ? PKG-INFO
> ? ? ? MANIFEST
> ? ? ? RECORD
> ?...

This would require setuptools and pip to change the way they look for
the packages,
but if the functions to work with this are located in Python, they
will be able to use the
same bits which could be great.


> If the location of the egg-info dir and the encoding of the python version and package
> version are only there for performance optimization reasons I'd suggest to really decouple
> optimization from file naming and having a caching directory that can compute an
> optimized representation once when an out-of-date situation is detected (e.g. when new
> packages are installed) making the optimization even faster, e.g. by having a file
> in suitable format instead of calling os.listdir() and iterating over the result.
>

Indeed. Having an index file would make things a whole lot simpler.
I am wondering then if this is not an evolution of the .pth files.

Although I find that having as many .pth file as we want is not robust.
It make things slow down when you have too many of them.

So, I'd be in favor of a new, unique, optimized, index file,
with a set of functions located in pkgutil to read and write in it

This index file could also index the namespace information,
in order to speed up the work needed to uninstall a package that
shares a namespace.

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Tue Feb 24 16:05:43 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 10:05:43 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <49A3B236.4050001@online.de>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
Message-ID: <20090224150333.54FDE3A4075@sparrow.telecommunity.com>

At 09:39 AM 2/24/2009 +0100, Joachim K?nig wrote:
>could the egg-info directory be put somewhere else (as a 
>configuration/command line option)?

No, since it's used to identify the installed location of the code 
that goes with it, ala PEP 262.  In other words, sys.path is its own 
installation database.


>If the location of the egg-info dir and the encoding of the python 
>version and package
>version are only there for performance optimization reasons I'd 
>suggest to really decouple
>optimization from file naming and having a caching directory that 
>can compute an
>optimized representation once when an out-of-date situation is 
>detected (e.g. when new
>packages are installed) making the optimization even faster, e.g. by 
>having a file
>in suitable format instead of calling os.listdir() and iterating 
>over the result.

That would introduce the possibility of the cache being out of 
date.  See PEP 262.

(By the way, the Python version info isn't needed for this 
optimization; only the package version.)


From pje at telecommunity.com  Tue Feb 24 16:20:35 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 10:20:35 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.co
 m>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
Message-ID: <20090224151824.A79383A4075@sparrow.telecommunity.com>

At 01:33 PM 2/24/2009 +0100, Tarek Ziad? wrote:
>Philip wrote:
> > So, the uninstallation code should simply not remove file(s) that 
> are referenced by more than one manifest in the target directory -- 
> a relatively simple, future-proof safeguard, that doesn't require 
> any specific knowledge of "namespace packages" per se.
> >
>
>Sounds good. Although, it requires scanning the files again which is
>not optimal, but the last point of this mail might be an idea to
>adress this.

Uninstallation isn't exactly a performance-critical activity.  It's 
trivial to read targetdir/*/RECORD and build up a dictionary of 
file->package and package->file links; this is Python, after all.  ;-)


>2009/2/24 Joachim K?nig <him at online.de>:
> > An other option could be to put the egg-info dir into the package 
> itself, e.g.
> >
> > zlib/
> >   __init__.py
> >   egg-info/
> >       PKG-INFO
> >       MANIFEST
> >       RECORD
> >  ...
>
>This would require setuptools and pip to change the way they look for
>the packages,

More precisely, it would require pkg_resources to become ridiculously 
slow, by massively increasing the number of directories that need to 
be read at runtime to determine what packages are present.


>Indeed. Having an index file would make things a whole lot simpler.

For *whom*?  Certainly not for system packaging tools (rpm, deb, et al).

A design goal should be to allow system packaging tools to install a 
static file footprint: i.e., independent files with predefined 
content, and no post-processing steps.  You can't do that with a 
shared file, which is why setuptools uses a .pth hack to install 
namespace packages when building packages for rpm et al.


>I am wondering then if this is not an evolution of the .pth files.
>
>Although I find that having as many .pth file as we want is not robust.
>It make things slow down when you have too many of them.
>
>So, I'd be in favor of a new, unique, optimized, index file,
>with a set of functions located in pkgutil to read and write in it
>
>This index file could also index the namespace information,
>in order to speed up the work needed to uninstall a package that
>shares a namespace.

So, .pth files are bad...  let's make more?  In fact, let's make a 
new thing that'll have its own, new bugs?  So that we can have 
uninstalls that take only 1/10th of a second instead of 1/2 a second?

The standard to beat in this area, I believe, is PEP 262.  At the 
very least, if you're making any major changes in direction from that 
PEP, the rationale for those differences should be documented.  (And 
consolidated index files are explicitly rejected by that PEP, with 
good reason.)


From him at online.de  Tue Feb 24 16:29:21 2009
From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=)
Date: Tue, 24 Feb 2009 16:29:21 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224150333.54FDE3A4075@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<20090224150333.54FDE3A4075@sparrow.telecommunity.com>
Message-ID: <49A41251.2060902@online.de>

P.J. Eby schrieb:
> At 09:39 AM 2/24/2009 +0100, Joachim K?nig wrote:
>> could the egg-info directory be put somewhere else (as a 
>> configuration/command line option)?
>
> No, since it's used to identify the installed location of the code 
> that goes with it, ala PEP 262.  In other words, sys.path is its own 
> installation database.
But PEP262 suggests <prefix>/lib/python<version>/install-db/ for the 
location and the
open issues there state (among other things):

> PJE suggests the installation database "be potentially present on
>     every directory in sys.path, 
So PEP262 implies to me that we either would have one
"install-db/" directory on "<prefix>/lib/python<version>/" or one on 
each directory
of sys.path, e.g. also one on "site-packages". This would really solve my
problem, having only one entry more per directory of sys.path instead
of having one directory per packages at the locations I generally look 
for the code.


From ronaldoussoren at mac.com  Tue Feb 24 16:39:24 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 24 Feb 2009 16:39:24 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
Message-ID: <D82C4BE2-9948-4486-960D-5C3DE49AEA65@mac.com>


On 24 Feb, 2009, at 13:33, Tarek Ziad? wrote:
>
>
> 2009/2/24 Joachim K?nig <him at online.de>:
>> An other option could be to put the egg-info dir into the package  
>> itself, e.g.
>>
>> zlib/
>>   __init__.py
>>   egg-info/
>>       PKG-INFO
>>       MANIFEST
>>       RECORD
>>  ...
>
> This would require setuptools and pip to change the way they look for
> the packages,

Not only that, it also requires that the name of the distribution is  
the same as the name of (one of) its python packages. I have several  
projects where this correspondence is not present.

>
> but if the functions to work with this are located in Python, they
> will be able to use the
> same bits which could be great.
>
>
>> If the location of the egg-info dir and the encoding of the python  
>> version and package
>> version are only there for performance optimization reasons I'd  
>> suggest to really decouple
>> optimization from file naming and having a caching directory that  
>> can compute an
>> optimized representation once when an out-of-date situation is  
>> detected (e.g. when new
>> packages are installed) making the optimization even faster, e.g.  
>> by having a file
>> in suitable format instead of calling os.listdir() and iterating  
>> over the result.
>>
>
> Indeed. Having an index file would make things a whole lot simpler.

I don't see how this would make thing easier. An index file introduces  
another concept and requires care to ensure that it doesn't get out of  
date.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090224/4619c213/attachment-0001.bin>

From ronaldoussoren at mac.com  Tue Feb 24 16:45:39 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 24 Feb 2009 16:45:39 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224151824.A79383A4075@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
Message-ID: <EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>


On 24 Feb, 2009, at 16:20, P.J. Eby wrote:
>
>> Indeed. Having an index file would make things a whole lot simpler.
>
> For *whom*?  Certainly not for system packaging tools (rpm, deb, et  
> al).
>
> A design goal should be to allow system packaging tools to install a  
> static file footprint: i.e., independent files with predefined  
> content, and no post-processing steps.  You can't do that with a  
> shared file, which is why setuptools uses a .pth hack to install  
> namespace packages when building packages for rpm et al.

What about another interoperability hook for system packages: specify  
a file that a (system) package manager can include into the egg-info  
directory (or egg-file) to tell setuptools/pip that this egg is  
managed by the system and hence shouldn't be removed by setuptools/pip.

Which such a file the user of python package tool could be warned if  
he tries to uninstall an egg that's owned by the system, instead of  
invoking the wrath of a sysadmin by removing such files.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090224/87685ba3/attachment.bin>

From floris.bruynooghe at gmail.com  Tue Feb 24 19:21:32 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 24 Feb 2009 18:21:32 +0000
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224025132.5D3003A409E@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<20090224025132.5D3003A409E@sparrow.telecommunity.com>
Message-ID: <20090224182132.GA4404@laurie.devork>

On Mon, Feb 23, 2009 at 09:53:17PM -0500, P.J. Eby wrote:
> At 01:50 AM 2/24/2009 +0100, Tarek Ziad? wrote:
>> PEP, because as far as I
>> can see, the namespace package boilerplate in setuptools is using
>> pkgutil.extend_path which
>> fixes __path__ variables on the fly. So I can't think of a case where
>> a static __init__.py file
>> will be shared, thus removed.
>
> When a distutils package does it.  I'm not positive, but if 'pip'  
> supports namespace packages without using .pth files, then it has to use 
> a shared __init__ also.
>
> And in the long run, easy_install will do this too.
>
> So, the uninstallation code should simply not remove file(s) that are  
> referenced by more than one manifest in the target directory -- a  
> relatively simple, future-proof safeguard, that doesn't require any  
> specific knowledge of "namespace packages" per se.

But it does require that the uninstall reads all *.egg-info/RECORD
files in a site directory when uninstalling.  That's slow, but I guess
that's fair enough.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Tue Feb 24 19:30:31 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 24 Feb 2009 18:30:31 +0000
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <D82C4BE2-9948-4486-960D-5C3DE49AEA65@mac.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<D82C4BE2-9948-4486-960D-5C3DE49AEA65@mac.com>
Message-ID: <20090224183031.GB4404@laurie.devork>

On Tue, Feb 24, 2009 at 04:39:24PM +0100, Ronald Oussoren wrote:
>
> On 24 Feb, 2009, at 13:33, Tarek Ziad? wrote:
>>
>>
>> 2009/2/24 Joachim K?nig <him at online.de>:
>>> An other option could be to put the egg-info dir into the package  
>>> itself, e.g.
>>>
>>> zlib/
>>>   __init__.py
>>>   egg-info/
>>>       PKG-INFO
>>>       MANIFEST
>>>       RECORD
>>>  ...
>>
>> This would require setuptools and pip to change the way they look for
>> the packages,
>
> Not only that, it also requires that the name of the distribution is the 
> same as the name of (one of) its python packages. I have several  
> projects where this correspondence is not present.

It also requires that a distribution installs a package.  What about
single-module packages?


>> Indeed. Having an index file would make things a whole lot simpler.
>
> I don't see how this would make thing easier. An index file introduces  
> another concept and requires care to ensure that it doesn't get out of  
> date.

An index file should probably a be private optimisation of a tool and
not be required or standard.  See also P. Eby's arguments about this.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From strawman at astraw.com  Tue Feb 24 20:48:13 2009
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 24 Feb 2009 11:48:13 -0800
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>	<20090223231310.GA5711@laurie.devork>	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>	<49A3B236.4050001@online.de>	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
Message-ID: <49A44EFD.50201@astraw.com>

Ronald Oussoren wrote:
> 
> On 24 Feb, 2009, at 16:20, P.J. Eby wrote:
>>
>>> Indeed. Having an index file would make things a whole lot simpler.
>>
>> For *whom*?  Certainly not for system packaging tools (rpm, deb, et al).
>>
>> A design goal should be to allow system packaging tools to install a
>> static file footprint: i.e., independent files with predefined
>> content, and no post-processing steps.  You can't do that with a
>> shared file, which is why setuptools uses a .pth hack to install
>> namespace packages when building packages for rpm et al.
> 
> What about another interoperability hook for system packages: specify a
> file that a (system) package manager can include into the egg-info
> directory (or egg-file) to tell setuptools/pip that this egg is managed
> by the system and hence shouldn't be removed by setuptools/pip.
> 
> Which such a file the user of python package tool could be warned if he
> tries to uninstall an egg that's owned by the system, instead of
> invoking the wrath of a sysadmin by removing such files.

But that is already implemented via file/dir permissions. By your
reasoning, we should also have something which warns users not to
install to the system directory. These ideas are a duplication of
functionality -- this functionality is implemented by the disabling
write permissions of non-sysadmins into system directories.

Or do you propose users put some stuff into their system directories not
managed by their package managers and other stuff managed by the package
managers?

From pje at telecommunity.com  Tue Feb 24 20:57:29 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 14:57:29 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
Message-ID: <20090224195518.175D03A4108@sparrow.telecommunity.com>

At 04:45 PM 2/24/2009 +0100, Ronald Oussoren wrote:
>What about another interoperability hook for system packages: specify
>a file that a (system) package manager can include into the egg-info
>directory (or egg-file) to tell setuptools/pip that this egg is
>managed by the system and hence shouldn't be removed by setuptools/pip.
>
>Which such a file the user of python package tool could be warned if
>he tries to uninstall an egg that's owned by the system, instead of
>invoking the wrath of a sysadmin by removing such files.

The absence of a RECORD file could suffice for that, I think.  Any 
file that isn't positively listed in *exactly* one RECORD file 
shouldn't be removed.

I'm not super-set on that position, just noting that the absence of a 
listing is sufficient to detect something that "shouldn't" (as 
opposed to "mustn't") be deleted.  The only contradicting use case is 
the deletion of packages that were installed via an older distutils version.


From pje at telecommunity.com  Tue Feb 24 20:58:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 14:58:37 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224182132.GA4404@laurie.devork>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<20090224025132.5D3003A409E@sparrow.telecommunity.com>
	<20090224182132.GA4404@laurie.devork>
Message-ID: <20090224195625.36A3D3A4108@sparrow.telecommunity.com>

At 06:21 PM 2/24/2009 +0000, Floris Bruynooghe wrote:
>On Mon, Feb 23, 2009 at 09:53:17PM -0500, P.J. Eby wrote:
> > So, the uninstallation code should simply not remove file(s) that are
> > referenced by more than one manifest in the target directory -- a
> > relatively simple, future-proof safeguard, that doesn't require any
> > specific knowledge of "namespace packages" per se.
>
>But it does require that the uninstall reads all *.egg-info/RECORD
>files in a site directory when uninstalling.  That's slow, but I guess
>that's fair enough.

You're going to need to look at all the egg-info dirs anyway, at 
least if you're planning to verify whether another installed package 
depends on the package you're removing.


From ziade.tarek at gmail.com  Tue Feb 24 22:10:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 24 Feb 2009 22:10:48 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224150333.54FDE3A4075@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<20090224150333.54FDE3A4075@sparrow.telecommunity.com>
Message-ID: <94bdd2610902241310j2e78d2den3918d851bba88230@mail.gmail.com>

2009/2/24 P.J. Eby <pje at telecommunity.com>:
> At 09:39 AM 2/24/2009 +0100, Joachim K?nig wrote:
>>
>> could the egg-info directory be put somewhere else (as a configuration/command line option)?
>
> No, since it's used to identify the installed location of the code that goes with it, ala PEP 262. ?In other words, sys.path is its own installation database.
>
>
>> If the location of the egg-info dir and the encoding of the python version and package
>> version are only there for performance optimization reasons I'd suggest to really decouple
>> optimization from file naming and having a caching directory that can compute an
>> optimized representation once when an out-of-date situation is detected (e.g. when new
>> packages are installed) making the optimization even faster, e.g. by having a file
>> in suitable format instead of calling os.listdir() and iterating over the result.
>
> That would introduce the possibility of the cache being out of date. ?See PEP 262.
>
> (By the way, the Python version info isn't needed for this optimization; only the package version.)

ok so far, from the whole dicussion it seems that everyone agrees that
the Python version is superfluous
in the .egg-info files, so I'll update the PEP for this point.

I'll also start to write more details about uninstallation

From pje at telecommunity.com  Tue Feb 24 22:26:57 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 16:26:57 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090224195518.175D03A4108@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
	<20090224195518.175D03A4108@sparrow.telecommunity.com>
Message-ID: <20090224212446.8CAF33A4108@sparrow.telecommunity.com>

At 02:57 PM 2/24/2009 -0500, P.J. Eby wrote:
>At 04:45 PM 2/24/2009 +0100, Ronald Oussoren wrote:
>>What about another interoperability hook for system packages: specify
>>a file that a (system) package manager can include into the egg-info
>>directory (or egg-file) to tell setuptools/pip that this egg is
>>managed by the system and hence shouldn't be removed by setuptools/pip.
>>
>>Which such a file the user of python package tool could be warned if
>>he tries to uninstall an egg that's owned by the system, instead of
>>invoking the wrath of a sysadmin by removing such files.
>
>The absence of a RECORD file could suffice for that, I think.  Any 
>file that isn't positively listed in *exactly* one RECORD file 
>shouldn't be removed.
>
>I'm not super-set on that position, just noting that the absence of 
>a listing is sufficient to detect something that "shouldn't" (as 
>opposed to "mustn't") be deleted.  The only contradicting use case 
>is the deletion of packages that were installed via an older distutils version.

Hm.  This is also relevant for *install* operations... which should 
not overwrite (without warning) a file that's not listed in any 
RECORD files, even if it has the exact same content.  (As that would 
mean a later uninstall would silently delete it.)


From zooko at zooko.com  Wed Feb 25 04:42:39 2009
From: zooko at zooko.com (zooko)
Date: Tue, 24 Feb 2009 20:42:39 -0700
Subject: [Distutils] permissions and GNU stow (was: PEP 376 for Distutils)
In-Reply-To: <49A44EFD.50201@astraw.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>	<20090223231310.GA5711@laurie.devork>	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>	<49A3B236.4050001@online.de>	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
	<49A44EFD.50201@astraw.com>
Message-ID: <0ACFC8F4-987C-44F9-9E91-197324CFCAEF@zooko.com>

On Feb 24, 2009, at 12:48 PM, Andrew Straw wrote:

> But that is already implemented via file/dir permissions.

That's what's beautiful about GNU stow.  Look:

sudo mkdir /usr/local/stow/grozz
sudo chown `whoami` /usr/local/stow/grozz
python ./setup.py install --prefix=/usr/local/stow/grozz # LOOK NO SUDO
cd /usr/local/stow
sudo stow grozz

The important point is that the install process doesn't have  
permission to write into the system, but GNU stow does.  GNU stow is  
more trusted to behave well than the install scripts of the grozz  
package are, and it is extremely simple and fail-safe -- all it does  
is make symlinks from /usr/local/x/y -> /usr/local/stow/grozz/x/y .

Note that GNU stow can therefore completely and correctly *uninstall*  
everything that it installed, by examining all of /usr/local looking  
for symlinks into /usr/local/stow/grozz and removing them.   
(Therefore it doesn't need a "RECORD" file -- the filesystem itself  
contains the exact record.)

I hope that this new distutils work will make it possible to use GNU  
stow to manage Python packages (by making it so that installation  
consists only of *adding* files to the system, not requiring  
*editing* files such as .pth files).

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From him at online.de  Wed Feb 25 09:16:22 2009
From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=)
Date: Wed, 25 Feb 2009 09:16:22 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <94bdd2610902241310j2e78d2den3918d851bba88230@mail.gmail.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>	
	<20090223231310.GA5711@laurie.devork>	
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>	
	<49A3B236.4050001@online.de>	
	<20090224150333.54FDE3A4075@sparrow.telecommunity.com>
	<94bdd2610902241310j2e78d2den3918d851bba88230@mail.gmail.com>
Message-ID: <49A4FE56.5080805@online.de>

Tarek Ziad? wrote:
> ok so far, from the whole dicussion it seems that everyone agrees that
> the Python version is superfluous
> in the .egg-info files, so I'll update the PEP for this point.
>
> I'll also start to write more details about uninstallation
>   
As P.J.Eby pointed out the importance of PEP 262, could all the egg-info 
directories be also
put into a subdirectory called "install-db" as described in that PEP. As 
proposed by P.J.Eby
in PEP262 under "open issues", there could/should be one of these for 
each directory
where packages are installed.

So to repeat the initial example for zlib, it should look like this:

zlib/
install_db/
     zlib-2.5.2.egg-info/
        PKG-INFO
        MANIFEST
        RECORD

The name of the "install_db" directory would not be that important, e.g. 
it could e.g. be called "egg-info" and
the ".egg-info" extension from the zlib egg-info directory name could be 
stripped, making the parsing of the listdir() call
even faster 8-)



From ziade.tarek at gmail.com  Wed Feb 25 12:48:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 25 Feb 2009 12:48:48 +0100
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <49A4FE56.5080805@online.de>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<20090224150333.54FDE3A4075@sparrow.telecommunity.com>
	<94bdd2610902241310j2e78d2den3918d851bba88230@mail.gmail.com>
	<49A4FE56.5080805@online.de>
Message-ID: <94bdd2610902250348g57eecb07t77281100d56f086@mail.gmail.com>

2009/2/25 Joachim K?nig <him at online.de>:
> Tarek Ziad? wrote:
>>
>> ok so far, from the whole dicussion it seems that everyone agrees that
>> the Python version is superfluous
>> in the .egg-info files, so I'll update the PEP for this point.
>>
>> I'll also start to write more details about uninstallation
>>
>
> As P.J.Eby pointed out the importance of PEP 262, could all the egg-info directories be also
> put into a subdirectory called "install-db" as described in that PEP. As proposed by P.J.Eby
> in PEP262 under "open issues", there could/should be one of these for each directory
> where packages are installed.
>
> So to repeat the initial example for zlib, it should look like this:
>
> zlib/
> install_db/
> ? ?zlib-2.5.2.egg-info/
> ? ? ? PKG-INFO
> ? ? ? MANIFEST
> ? ? ? RECORD
>
> The name of the "install_db" directory would not be that important, e.g. it could e.g. be called "egg-info" and
> the ".egg-info" extension from the zlib egg-info directory name could be stripped, making the parsing of the listdir() call
> even faster 8-)

Let's call it "install-db" then, like suggested in PEP 262 : this will
ensure that it cannot be a package/module name
because of the "-".

I doubt anyone is calling a module or a package install_db out there
but it's safer that way

Tarek

From pje at telecommunity.com  Wed Feb 25 15:17:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Feb 2009 09:17:52 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <49A4FE56.5080805@online.de>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<20090224150333.54FDE3A4075@sparrow.telecommunity.com>
	<94bdd2610902241310j2e78d2den3918d851bba88230@mail.gmail.com>
	<49A4FE56.5080805@online.de>
Message-ID: <20090225141541.E676E3A4109@sparrow.telecommunity.com>

At 09:16 AM 2/25/2009 +0100, Joachim K?nig wrote:
>Tarek Ziad? wrote:
>>ok so far, from the whole dicussion it seems that everyone agrees that
>>the Python version is superfluous
>>in the .egg-info files, so I'll update the PEP for this point.
>>
>>I'll also start to write more details about uninstallation
>>
>As P.J.Eby pointed out the importance of PEP 262,

I pointed out the importance its *requirements* and *criteria*  -- 
not its implementation.  There's a big difference.


>  could all the egg-info directories be also
>put into a subdirectory called "install-db" as described in that 
>PEP. As proposed by P.J.Eby
>in PEP262 under "open issues", there could/should be one of these 
>for each directory
>where packages are installed.
>
>So to repeat the initial example for zlib, it should look like this:
>
>zlib/
>install_db/
>     zlib-2.5.2.egg-info/
>        PKG-INFO
>        MANIFEST
>        RECORD
>
>The name of the "install_db" directory would not be that important, 
>e.g. it could e.g. be called "egg-info" and
>the ".egg-info" extension from the zlib egg-info directory name 
>could be stripped, making the parsing of the listdir() call
>even faster 8-)

I'm -1 on this, as it breaks backward compatibility with setuptools.


From setuptools at bugs.python.org  Wed Feb 25 15:58:17 2009
From: setuptools at bugs.python.org (Arve Knudsen)
Date: Wed, 25 Feb 2009 14:58:17 +0000
Subject: [Distutils] [issue62] pkg_resources.WorkingSet.add_entry
	inconsistency
In-Reply-To: <1235573897.57.0.3145920264.issue62@psf.upfronthosting.co.za>
Message-ID: <1235573897.57.0.3145920264.issue62@psf.upfronthosting.co.za>


New submission from Arve Knudsen <arve.knudsen at gmail.com>:

According to WorkingSet's documentation, it should call find_distributions with
False for the only-parameter. In reality, it passes True, however. The
documented behaviour is what I was after, but maybe it could be parameterized
also in this method?

----------
keyword: pkg_resources
messages: 245
nosy: aknuds-1
priority: bug
status: unread
title: pkg_resources.WorkingSet.add_entry inconsistency

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue62>
_______________________________________________

From pje at telecommunity.com  Wed Feb 25 18:43:00 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Feb 2009 12:43:00 -0500
Subject: [Distutils] [issue62] pkg_resources.WorkingSet.add_entry
	inconsistency
In-Reply-To: <1235573897.57.0.3145920264.issue62@psf.upfronthosting.co.z
 a>
References: <1235573897.57.0.3145920264.issue62@psf.upfronthosting.co.za>
	<1235573897.57.0.3145920264.issue62@psf.upfronthosting.co.za>
Message-ID: <20090225174049.884CB3A4075@sparrow.telecommunity.com>

At 02:58 PM 2/25/2009 +0000, Arve Knudsen wrote:
>According to WorkingSet's documentation, it should call 
>find_distributions with
>False for the only-parameter.

Where in the documentation is this?  It should be fixed.


>In reality, it passes True, however.

This is correct.


>The documented behaviour is what I was after,

Then you probably want an Environment, not a WorkingSet.


From floris.bruynooghe at gmail.com  Wed Feb 25 23:03:10 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Wed, 25 Feb 2009 22:03:10 +0000
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <49A44EFD.50201@astraw.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
	<49A44EFD.50201@astraw.com>
Message-ID: <20090225220310.GA4665@laurie.devork>

On Tue, Feb 24, 2009 at 11:48:13AM -0800, Andrew Straw wrote:
> Ronald Oussoren wrote:
> > What about another interoperability hook for system packages: specify a
> > file that a (system) package manager can include into the egg-info
> > directory (or egg-file) to tell setuptools/pip that this egg is managed
> > by the system and hence shouldn't be removed by setuptools/pip.
> > 
> > Which such a file the user of python package tool could be warned if he
> > tries to uninstall an egg that's owned by the system, instead of
> > invoking the wrath of a sysadmin by removing such files.
> 
> But that is already implemented via file/dir permissions.

Not all systems are managed by experienced sysadmins and on many
single user systems `sudo' is all to easy to invoke.

> By your
> reasoning, we should also have something which warns users not to
> install to the system directory. These ideas are a duplication of
> functionality -- this functionality is implemented by the disabling
> write permissions of non-sysadmins into system directories.

As pointed out by PJE simply removing the RECORD file should do the
trick.

> Or do you propose users put some stuff into their system directories not
> managed by their package managers and other stuff managed by the package
> managers?

It's interesting to point out what seems to be planned for Debian:
http://lists.debian.org/debian-devel/2009/02/msg00431.html

Quoting just the relevant part:

"""
Local installation path
-----------------------

When installing Python modules using distutils, the resulting files
end up in the same location wether they are installed by a Debian
package, or by a local user or administrator, unless the installation
path is overwritten on the command line.  Compare this with most
software based on autoconf, where an explicit prefix has to be
provided for the packaging, while the default install installs into
/usr/local.  For new Python versions packaged in Debian this will
change so that an installation into /usr (not /usr/local) requires an
extra option to distutils install command (--install-layout=deb).  To
avoid breaking the packaging of existing code the distutils install
command for 2.4 and 2.5 will just accept this option and ignore it.
For the majority of packages we won't see changes in the packaging,
provided that the python packaging helpers can find the files in the
right location and move it to the expected target path.

A second issue raised by developers was the clash of modules and
extensions installed by a local python installation (with default
prefix /usr/local) with the modules provided by Debian packages
(/usr/local/lib/pythonX.Y/site-packages shared by the patched "system"
python and the locally installed python.  To avoid this clash the
directory `site-packages' should be renamed to `dist-packages' in
both locations:

 - /usr/lib/pythonX.Y/dist-packages (installation location for code
   packaged for Debian)
 - /usr/local/lib/pythonX.Y/dist-packages (installation location
   for locally installed code using distutils install without
   options).

The path /usr/lib/pythonX.Y/site-packages is not found on sys.path
anymore.

About the name: Discussed this with Barry Warsaw and Martin v. Loewis,
and we came to the conclusion that using the same directory name for
both locations would be the most consistent way.

This change should make the request to conditionalize the inclusion of
/usr/local/lib/pythonX.Y/site-packages into sys.path obsolete.

If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages
pointing to dist-packages.
"""

This should address the concern of sysadmins making mistakes when
adding and removing python distributions (packages/modules).  In other
words, maybe it's not python's problem but the OS distribution's
problem.

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Wed Feb 25 23:16:50 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Feb 2009 17:16:50 -0500
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090225220310.GA4665@laurie.devork>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
	<49A44EFD.50201@astraw.com> <20090225220310.GA4665@laurie.devork>
Message-ID: <20090225221440.59D323A4075@sparrow.telecommunity.com>

At 10:03 PM 2/25/2009 +0000, Floris Bruynooghe wrote:
>It's interesting to point out what seems to be planned for Debian:
>http://lists.debian.org/debian-devel/2009/02/msg00431.html
>
>Quoting just the relevant part:
>
>"""
>Local installation path
>-----------------------
>
>When installing Python modules using distutils, the resulting files
>end up in the same location wether they are installed by a Debian
>package, or by a local user or administrator, unless the installation
>path is overwritten on the command line.  Compare this with most
>software based on autoconf, where an explicit prefix has to be
>provided for the packaging, while the default install installs into
>/usr/local.  For new Python versions packaged in Debian this will
>change so that an installation into /usr (not /usr/local) requires an
>extra option to distutils install command (--install-layout=deb).  To
>avoid breaking the packaging of existing code the distutils install
>command for 2.4 and 2.5 will just accept this option and ignore it.

I wonder why they don't just use the sitewide distutils.cfg file, 
which would let them configure user-installed packages to go to 
somewhere else, e.g.:

[install]
prefix = /usr/local

(And of course the build tools could specify different options.)


From floris.bruynooghe at gmail.com  Thu Feb 26 12:33:28 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 26 Feb 2009 11:33:28 +0000
Subject: [Distutils] PEP 376 for Distutils
In-Reply-To: <20090225221440.59D323A4075@sparrow.telecommunity.com>
References: <94bdd2610902211837w4a584e15q26217052418d0593@mail.gmail.com>
	<20090223231310.GA5711@laurie.devork>
	<94bdd2610902231650k4fcace1w59742c7fd0ed314c@mail.gmail.com>
	<49A3B236.4050001@online.de>
	<94bdd2610902240433o1efcf344x37fd8d36f412708e@mail.gmail.com>
	<20090224151824.A79383A4075@sparrow.telecommunity.com>
	<EB0AE92F-32E6-466B-9C5D-2668293BFADC@mac.com>
	<49A44EFD.50201@astraw.com> <20090225220310.GA4665@laurie.devork>
	<20090225221440.59D323A4075@sparrow.telecommunity.com>
Message-ID: <e09e42760902260333t4835e114yaab86f8b6941f52f@mail.gmail.com>

2009/2/25 P.J. Eby <pje at telecommunity.com>:
> At 10:03 PM 2/25/2009 +0000, Floris Bruynooghe wrote:
>>
>> It's interesting to point out what seems to be planned for Debian:
>> http://lists.debian.org/debian-devel/2009/02/msg00431.html
>>
>> Quoting just the relevant part:
>>
>> """
>> Local installation path
>> -----------------------
>>
>> When installing Python modules using distutils, the resulting files
>> end up in the same location wether they are installed by a Debian
>> package, or by a local user or administrator, unless the installation
>> path is overwritten on the command line. ?Compare this with most
>> software based on autoconf, where an explicit prefix has to be
>> provided for the packaging, while the default install installs into
>> /usr/local. ?For new Python versions packaged in Debian this will
>> change so that an installation into /usr (not /usr/local) requires an
>> extra option to distutils install command (--install-layout=deb). ?To
>> avoid breaking the packaging of existing code the distutils install
>> command for 2.4 and 2.5 will just accept this option and ignore it.
>
> I wonder why they don't just use the sitewide distutils.cfg file, which
> would let them configure user-installed packages to go to somewhere else,
> e.g.:
>
> [install]
> prefix = /usr/local
>
> (And of course the build tools could specify different options.)

There might be an issue with this while building python itself.  I
remember having issues failing to build python because I had

[install]
home=/home/flub

in my .pydistutils.cfg.  When python is building and installing it's
standard library it uses distutils and having that in my configuration
file messed everything up and the builds failed (this was both when
building the upstream tarball as well as rebuilding the debian
package).  This is all IIRC however, I'd have to try this again to
remind me of the exact issues (thinking about it now this is probably
because python's makefile doesn't invoke setup.py with enough options
to be sure to overwrite the configuration file).

So you'd have to build-conflict against python, which would be
annoying at best, but you also need python to build python these days
(again IIRC, might be possible to get round this) so you're stuck.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From techtonik at gmail.com  Thu Feb 26 19:19:00 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 26 Feb 2009 20:19:00 +0200
Subject: [Distutils] Optional dependencies
Message-ID: <d34314100902261019g67d2d207ie9f76c4441d13493@mail.gmail.com>

Greetings,

I see that "setup.py register" does not understand "extras_require"

>setup.py register
\Python25\lib\distutils\dist.py:263: UserWarning: Unknown distribution
option: 'extras_require'
  warnings.warn(msg)

Is there any other way to provide optional dependencies and give user
an ability to know about them and make a choice?

-- 
--anatoly t.

From ziade.tarek at gmail.com  Thu Feb 26 22:49:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Feb 2009 22:49:04 +0100
Subject: [Distutils] Optional dependencies
In-Reply-To: <d34314100902261019g67d2d207ie9f76c4441d13493@mail.gmail.com>
References: <d34314100902261019g67d2d207ie9f76c4441d13493@mail.gmail.com>
Message-ID: <94bdd2610902261349qca168b6ub2eb7be9153c4e42@mail.gmail.com>

On Thu, Feb 26, 2009 at 7:19 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> Greetings,
>

Hi,

> I see that "setup.py register" does not understand "extras_require"
>
>>setup.py register
> \Python25\lib\distutils\dist.py:263: UserWarning: Unknown distribution
> option: 'extras_require'
> ?warnings.warn(msg)

This is not a distutils option, but a keyword added in setuptools

>
> Is there any other way to provide optional dependencies and give user
> an ability to know about them and make a choice?
>

Not on Distutils / PyPI side yet,

regards
Tarek

> --
> --anatoly t.
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Fri Feb 27 14:22:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 27 Feb 2009 14:22:29 +0100
Subject: [Distutils] Distutils standalone release
Message-ID: <94bdd2610902270522v2ee0dea1ibd0e1a90ba90ec8a@mail.gmail.com>

Hello,

the current Distutils trunk in Python is now in theory compatible with
Python 2.3, as I fixed all the broken tests.

My goal is to make a standalone release anyone can use, by replacing
their Python's distutils with the newest
one. This is important to speed up distutils improvements.

There's one more task I need to address in order to make a standalone
release of Distutils that will
be usable under Python 2.3->2.6 : fix the problem described in this thread:

http://mail.python.org/pipermail/distutils-sig/2009-February/011016.html

otherwise all packages based on setuptools will be broken.

I don't have a solution for that problem yet, and I am not sure if
this should be adressed in setuptools,
or in distutils, or on both sides.

Although, I have seen some other problems along the way, and the test
coverage is not sufficient
to see everything.

Any help on this will be appreciated to fix all the problem and
improve the test coverage.
If someone wants to give a hand, it's quite simple.

1/ install the standalone Distutils version located here:

$ svn co http://svn.python.org/projects/distutils/trunk/ distutils
$ cd distutils
$ python setup.py install

2/ remove the original distutils package located in Python lib

3/ run an installation (or anything) of a setuptools-based package

4/ report all the bugs here  :)


Thanks
Tarek


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/