From david.lyon at preisshare.net  Fri May  1 00:29:25 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 18:29:25 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090430175857.B5B093A4070@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<49F9A677.6090809@ar.media.kyoto-u.ac.jp>
	<b1bad02cee7d01b32556d080a4664e6e@preisshare.net>
	<20090430175857.B5B093A4070@sparrow.telecommunity.com>
Message-ID: <68ade51f557b5965cdc97e1b42b960a8@preisshare.net>



On Thu, 30 Apr 2009 14:01:31 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> A distribution found on PyPI such as "FooBar-2.79" is the 2.79
> release of the *project* "FooBar"...  this tells you absolutely
> nothing about the names of packages or modules contained in that project.

Ok - but that's hardly my fault...

> The simplicity that you are describing *never actually
> existed*, even before setuptools and eggs came on the scene.

Ok - Sure.

So that's why what I am working on is something to make things work *more*
simply in the future...


Best Regards

David



From david.lyon at preisshare.net  Fri May  1 00:57:50 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 30 Apr 2009 18:57:50 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300829n1221dcc4yebad5f3ea9abfd26@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
	<c7c43dc617a0bd08c241a4a297edc51e@preisshare.net>
	<94bdd2610904300829n1221dcc4yebad5f3ea9abfd26@mail.gmail.com>
Message-ID: <49ce1e61115d61a1d7568601edc666c4@preisshare.net>

Tarek,

On Thu, 30 Apr 2009 17:29:40 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> And then, as we said earlier, it will be impossible to uninstall packages
> that were previously uninstalled with some other techniques since we
won't have
> the info required.

Let's just accept this as your final position..

I'm led to believe that pip and enstaller both claim deinstallation of
packages.

It shows that people out there are doing - or at least claiming -
deinstallation.

Finally, if you know that easy_install has certain failings, then it is
wrong to
suggest that because I use it, that I am somehow responsible for it's
internal
faults.

Anyway - Have a nice day

David




From david at ar.media.kyoto-u.ac.jp  Fri May  1 06:41:11 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 01 May 2009 13:41:11 +0900
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <49ce1e61115d61a1d7568601edc666c4@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>	<c7c43dc617a0bd08c241a4a297edc51e@preisshare.net>	<94bdd2610904300829n1221dcc4yebad5f3ea9abfd26@mail.gmail.com>
	<49ce1e61115d61a1d7568601edc666c4@preisshare.net>
Message-ID: <49FA7D67.90008@ar.media.kyoto-u.ac.jp>

David Lyon wrote:
>
> Let's just accept this as your final position..
>
> I'm led to believe that pip and enstaller both claim deinstallation of
> packages.
>   

Yes - packages they have installed *themselves*. Nobody has ever claimed
that uninstallation is impossible - uninstallation of something which
was installed outside your knowledge is.

cheers,

David

From david.lyon at preisshare.net  Fri May  1 14:50:17 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 01 May 2009 08:50:17 -0400
Subject: [Distutils] Installed Packages..
In-Reply-To: <94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
Message-ID: <a519008479f4afc8f9b96b8c2d210fa9@preisshare.net>



Hi?Tarek,
On?Thu,?30?Apr?2009?16:48:32?+0200,?Tarek?Ziad???wrote:
>>?--code-------
>>
>>?import?pkg_resources
>>
>>?ws?=?pkg_resources.WorkingSet()
>>
>>?for?i?in?ws:
>>????s?=?str(i)
>>????print?s
>>
>>?--end?code---
..
>>>?No,?you?just?have?a?list?of?relative?paths?to?installed?package?there
>>>?that's?it.
It?seems?you?are?right...
But?...?when?I?check?more?closely?..?I?see?that?the?above?code?doesn't?
really?display?all?the?installed?packages?in?the?site-lib?directory.
There?are?some?directories?in?there?that?don't?get?included?in?the
results.?I?would?love?to?know?why.
So?I?am?realising?that?the?above?code?doesn't?give?the?right?answers.
I?thought?if?programmers?used?distutils?and?so?forth?everything?would
work?"properly".
or?is?this?just?an?issue?with?pkg_resources??
Are?these?failures?of?the?python?subsystem??
I?would?love?to?know...
David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090501/ac4d36a0/attachment.htm>

From brandon at rhodesmill.org  Fri May  1 16:20:07 2009
From: brandon at rhodesmill.org (Brandon Craig Rhodes)
Date: Fri, 01 May 2009 10:20:07 -0400
Subject: [Distutils] One Package Per Project
Message-ID: <87y6tg52i0.fsf@rhodesmill.org>

I think that, going forward, Python packaging tools (not installation
tools; they should remain as they are, for backwards compatibility)
should move to supporting only One Package Per Project.  And, each
project should have the same name as the package inside.  In the future,
people should have to download an old copy of distutils deliberately if
they want to build projects with several packages inside; we should stop
releasing tools that support or encourage it.

 1. It is easier on developers who want to "import escher" to know that
    they can simply list "escher" as a dependency instead of having to
    guess whether it's "Escher" or "EscherProject" or whether it's part
    of a larger "lithographers" project or whether, heaven forbid, the
    author decided to redundantly call the project "pyescher".

 2. This practice would make PyPI's name make actual sense.  It actually
    claims to be (you can check the site!) the "Python *Package* Index"
    whereas in fact it's currently nothing of the sort!  It's really an
    index of "projects" that might have zero, one, or several packages
    inside of them.  We should move all projects towards the good
    behavior of the ones that already name themselves after the single
    package that they contain.

 3. I think the whole idea of putting several packages in a project was
    useful back when dependencies didn't exist.  It made sense, in
    ancient days, for "ZODB" to include "transaction" because there was
    no other way to make sure they got installed together.  But now that
    dependencies are possible, there is no longer a need for multiple-
    package project that outweights the costs involved.

 4. The current scheme makes it impossible to choose a "safe" package
    name when creating and registering a new package.  Just because
    there's no "escher" *project* when you look at PyPI doesn't mean
    that some project doesn't have an "escher" package hidden inside.
    You could choose a package name, distribute your product, and only
    find out later that your users cannot install both your product and
    another product simultaneously because the other product was, in
    fact, already using that package name but without your knowing it.

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon

From ronaldoussoren at mac.com  Fri May  1 16:43:31 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 01 May 2009 16:43:31 +0200
Subject: [Distutils] One Package Per Project
Message-ID: <43733322507200595342122554726168648415-Webmail@me.com>

I'm -1 on that, for three reasons.

1. I have a number of packages where the PyPI name is not the name of the toplevel package where the two names aren't the same on purpose. An example of this is "pyobjc-framework-WebKit", containing the python package "WebKit". The two don't have the same name because "import WebKit" is more natural during imports, while "pyobjc-framework-WebKit" is clearer in the PyPI listing (you don't have to wonder if this is some cross-platform web toolkit, it's obviously related to PyObjC).

2. For basicly the same reason a number of my PyPI packages have a number of toplevel Python packages. That's again because that makes sense for these packages, and it's furthermore needed for backward compatibility.

3. There actually a good reason for using "pysomelib" instead of "somelib" as the PyPI name for the python bindings for the C library "somelib". Naming the python bindings the same as the base project is confusion, while at the same time "import pysomelib" looks lame in Python code.

That said, I agree that there should be a good reason for not using the python package/module name as the PyPI name, and I'm +1 on adding advice to the distutils documentation to keep the two the same.

It's also not clear to me what your proposal would mean for namespace packages. Would packages like "zope.interface" be allowed with your proposal?

Ronald
 
On Friday, May 01, 2009, at 04:20PM, "Brandon Craig Rhodes" <brandon at rhodesmill.org> wrote:
>I think that, going forward, Python packaging tools (not installation
>tools; they should remain as they are, for backwards compatibility)
>should move to supporting only One Package Per Project.  And, each
>project should have the same name as the package inside.  In the future,
>people should have to download an old copy of distutils deliberately if
>they want to build projects with several packages inside; we should stop
>releasing tools that support or encourage it.
>
> 1. It is easier on developers who want to "import escher" to know that
>    they can simply list "escher" as a dependency instead of having to
>    guess whether it's "Escher" or "EscherProject" or whether it's part
>    of a larger "lithographers" project or whether, heaven forbid, the
>    author decided to redundantly call the project "pyescher".
>
> 2. This practice would make PyPI's name make actual sense.  It actually
>    claims to be (you can check the site!) the "Python *Package* Index"
>    whereas in fact it's currently nothing of the sort!  It's really an
>    index of "projects" that might have zero, one, or several packages
>    inside of them.  We should move all projects towards the good
>    behavior of the ones that already name themselves after the single
>    package that they contain.
>
> 3. I think the whole idea of putting several packages in a project was
>    useful back when dependencies didn't exist.  It made sense, in
>    ancient days, for "ZODB" to include "transaction" because there was
>    no other way to make sure they got installed together.  But now that
>    dependencies are possible, there is no longer a need for multiple-
>    package project that outweights the costs involved.
>
> 4. The current scheme makes it impossible to choose a "safe" package
>    name when creating and registering a new package.  Just because
>    there's no "escher" *project* when you look at PyPI doesn't mean
>    that some project doesn't have an "escher" package hidden inside.
>    You could choose a package name, distribute your product, and only
>    find out later that your users cannot install both your product and
>    another product simultaneously because the other product was, in
>    fact, already using that package name but without your knowing it.
>
>-- 
>Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig
>
>

From tseaver at palladion.com  Fri May  1 16:49:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 01 May 2009 10:49:43 -0400
Subject: [Distutils] One Package Per Project
In-Reply-To: <87y6tg52i0.fsf@rhodesmill.org>
References: <87y6tg52i0.fsf@rhodesmill.org>
Message-ID: <gtf26j$o99$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brandon Craig Rhodes wrote:
> I think that, going forward, Python packaging tools (not installation
> tools; they should remain as they are, for backwards compatibility)
> should move to supporting only One Package Per Project.  And, each
> project should have the same name as the package inside.  In the future,
> people should have to download an old copy of distutils deliberately if
> they want to build projects with several packages inside; we should stop
> releasing tools that support or encourage it.
> 
>  1. It is easier on developers who want to "import escher" to know that
>     they can simply list "escher" as a dependency instead of having to
>     guess whether it's "Escher" or "EscherProject" or whether it's part
>     of a larger "lithographers" project or whether, heaven forbid, the
>     author decided to redundantly call the project "pyescher".
> 
>  2. This practice would make PyPI's name make actual sense.  It actually
>     claims to be (you can check the site!) the "Python *Package* Index"
>     whereas in fact it's currently nothing of the sort!  It's really an
>     index of "projects" that might have zero, one, or several packages
>     inside of them.  We should move all projects towards the good
>     behavior of the ones that already name themselves after the single
>     package that they contain.
> 
>  3. I think the whole idea of putting several packages in a project was
>     useful back when dependencies didn't exist.  It made sense, in
>     ancient days, for "ZODB" to include "transaction" because there was
>     no other way to make sure they got installed together.  But now that
>     dependencies are possible, there is no longer a need for multiple-
>     package project that outweights the costs involved.
> 
>  4. The current scheme makes it impossible to choose a "safe" package
>     name when creating and registering a new package.  Just because
>     there's no "escher" *project* when you look at PyPI doesn't mean
>     that some project doesn't have an "escher" package hidden inside.
>     You could choose a package name, distribute your product, and only
>     find out later that your users cannot install both your pr-duct and
>     another product simultaneously because the other product was, in
>     fact, already using that package name but without your knowing it.

- -1. Infeasible, AFAICT, even if I agreed with the goals:

- - While trying out new bits of software, the developer is already going
  to know the PyPI name:  that is how she found the package in the first
  place (most likely).

- - There are scores of "projects" extant whose names don't match the
  single, top-level package they contain, and will be for the
  foreseeable future.

- - There are a number of "projects" which don't ship any top-level
  package at all, but contain one or more modules.

- - "Bundling" packages together might actually happen more often in
  the future (e.g., to ease installation onto the GAE, or to solve
  other deployment problems (e.g., needing to install without network
  access).

- - Namespace packages were invented to reduce the problem you raise in #4
  to competition for the top-level name only.  Because those names are
  usually about "identity" of the releasing organization, rather than
  functionality of the software being released, it has, in fact, worked.

Current status which affects some of your concerns:

- - PEP 314 added a 'Provides' field to PKG-INFO; that field is supposed
  to name modules and packages included.  That PEP also added 'Requires'
  and 'Obsoletes' fields, which are mostly unused.  This information is
  available on a per-package basis from PyPI at the moment, but see
  below.

- - PEP 345 will add 'Provides-Dist', 'Requires-Dist', and
  'Obsoletes-Dist' fields to PKG-INFO, where the values are going to be
  "project" names and versions.

- - We expect to be able to build index files which can be downloaded
  (e.g., from PyPI or another index) which will make it easier to find
  and resolve dependency graphs a priori,.  This file will be similar to
  the index files which yum and apt download from indexes, and could
  support queries like those done by 'apt-cache query' or 'yum search',
  which should make it easier to find the projects which 'Provide' a
  given package name, as well


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ+wwG+gerLs4ltQ4RArCKAJ46IYoIr5WvFwRCeC5yxyXyd4M/6wCeL/2d
xAYukhM+nSNV1gpHUp4XOdc=
=oEqs
-----END PGP SIGNATURE-----


From pje at telecommunity.com  Fri May  1 17:15:08 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 01 May 2009 11:15:08 -0400
Subject: [Distutils] Installed Packages..
In-Reply-To: <a519008479f4afc8f9b96b8c2d210fa9@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<86092c314f8fbc0f32a57fbe4f47d875@preisshare.net>
	<94bdd2610904300523x57a6b872la8416af6271d19d1@mail.gmail.com>
	<6023049d001a7fcc4e9aa09489217fb2@preisshare.net>
	<94bdd2610904300543u6fe37517mee7a2cca38dbb076@mail.gmail.com>
	<1a27e131b5d7ae18a041cdf802130281@preisshare.net>
	<94bdd2610904300650g91a20dq5478ad45811c544@mail.gmail.com>
	<e85707c8e5a8f66b60f670b9173cd22a@preisshare.net>
	<94bdd2610904300748y6aac6c4fw1b87c659898aaef@mail.gmail.com>
	<a519008479f4afc8f9b96b8c2d210fa9@preisshare.net>
Message-ID: <20090501151232.C41983A4070@sparrow.telecommunity.com>

At 08:50 AM 5/1/2009 -0400, David Lyon wrote:
>It seems you are right...
>But ... when I check more closely .. I see that the above code doesn't
>really display all the installed packages in the site-lib directory.
>There are some directories in there that don't get included in the
>results. I would love to know why.
>So I am realising that the above code doesn't give the right answers.
>I thought if programmers used distutils and so forth everything would
>work "properly".

The packages have to be installed using either setuptools, or Python 
2.5+ distutils, in order for them to be listable via pkg_resources.


From ianb at colorstudy.com  Sat May  2 00:07:26 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 1 May 2009 17:07:26 -0500
Subject: [Distutils] Setuptools, namespace packages,
	--single-version-externally-managed
Message-ID: <b654cd2e0905011507j3cec56e0n589418bc866aaf82@mail.gmail.com>

So, a bit of a problem came up with pip and namespace packages.  Here's my
understanding of what's going on:

When you install a namespace package with pip, it uses install
--single-version-externally-managed, and generally the namespace directory
is empty and there's a *.nspkg.pth file that has this:

import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
*('<NAMESPACE>',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m =
not ie and sys.modules.setdefault('zope',new.module('zope')); mp = (m or [])
and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)

So the lack of an __init__.py file doesn't really matter, because it's
created right there, and has its __path__ added to.  But there's a problem
when there's another namespace package elsewhere on the path, that wasn't
installed with pip (or Setuptools) and uses pkgutils.extend_path(__path__,
__name__).  This doesn't get imported because of that .pth file, and the
.pth file doesn't itself use extend_path, so the path isn't searched.  This
is currently happening with Zope packages installed with plain distutils,
then another package installed with the zope namespace elsewhere with pip.
(When using easy_install, I think pkg_resource.declare_namespace comes into
play somewhere, and this seems to handle this case, but I'm not sure why the
installation is different with pip.)

So... what should pip be doing differently to make this work?

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090501/12d73fef/attachment.htm>

From david.lyon at preisshare.net  Sat May  2 01:07:24 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 01 May 2009 19:07:24 -0400
Subject: [Distutils] One Package Per Project
In-Reply-To: <87y6tg52i0.fsf@rhodesmill.org>
References: <87y6tg52i0.fsf@rhodesmill.org>
Message-ID: <6ba4968543df8db116d9877943dbeac8@preisshare.net>

Hi Brandon,

It sounds like a nice idea, in terms of simplifying things, but it
worries me what "evil" code would have to be written to enforce it.

My simplistic understunding of python packaging is that there are:

 - internal packages

and user packages.. divided into:

 - site-packages
 - registered site-packages
 - packages located elsewhere outside site-packages but 
   'registered' via a .pth in site-packages
 - project packages
 - project sub-packages

I would describe what you are talking about as project sub-packages.

If they aren't registered.. in site-packages.. then it doesn't
really make any difference how many there are. Or what they
are called...

What we need is restructuring sure...

But what seems to be needed is clearer thought around the
whole package registration system.

I'm all for simplification..

On Fri, 01 May 2009 10:20:07 -0400, Brandon Craig Rhodes
<brandon at rhodesmill.org> wrote:
> I think that, going forward, Python packaging tools (not installation
> tools; they should remain as they are, for backwards compatibility)
> should move to supporting only One Package Per Project.  And, each
> project should have the same name as the package inside.  In the future,
> people should have to download an old copy of distutils deliberately if
> they want to build projects with several packages inside; we should stop
> releasing tools that support or encourage it.
> 
>  1. It is easier on developers who want to "import escher" to know that
>     they can simply list "escher" as a dependency instead of having to
>     guess whether it's "Escher" or "EscherProject" or whether it's part
>     of a larger "lithographers" project or whether, heaven forbid, the
>     author decided to redundantly call the project "pyescher".
> 
>  2. This practice would make PyPI's name make actual sense.  It actually
>     claims to be (you can check the site!) the "Python *Package* Index"
>     whereas in fact it's currently nothing of the sort!  It's really an
>     index of "projects" that might have zero, one, or several packages
>     inside of them.  We should move all projects towards the good
>     behavior of the ones that already name themselves after the single
>     package that they contain.
> 
>  3. I think the whole idea of putting several packages in a project was
>     useful back when dependencies didn't exist.  It made sense, in
>     ancient days, for "ZODB" to include "transaction" because there was
>     no other way to make sure they got installed together.  But now that
>     dependencies are possible, there is no longer a need for multiple-
>     package project that outweights the costs involved.
> 
>  4. The current scheme makes it impossible to choose a "safe" package
>     name when creating and registering a new package.  Just because
>     there's no "escher" *project* when you look at PyPI doesn't mean
>     that some project doesn't have an "escher" package hidden inside.
>     You could choose a package name, distribute your product, and only
>     find out later that your users cannot install both your product and
>     another product simultaneously because the other product was, in
>     fact, already using that package name but without your knowing it.
> 
> --
> Brandon Craig Rhodes   brandon at rhodesmill.org  
> http://rhodesmill.org/brandon
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From floris.bruynooghe at gmail.com  Sat May  2 23:13:56 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 2 May 2009 22:13:56 +0100
Subject: [Distutils] setup.py build_ext --rpath behaviour
In-Reply-To: <20090416104228.GA416@laurie.devork>
References: <20090416104228.GA416@laurie.devork>
Message-ID: <20090502211356.GA10792@laurie.devork>

On Thu, Apr 16, 2009 at 11:42:28AM +0100, Floris Bruynooghe wrote:
> Now when using the --rpath option to the build_ext command
> distutils.unixcompiler.UnixCCompiler.runtime_library_dir_option() will
> use some heuristics to figure out which option to pass to compiler to
> get the runpath in (i.e. "-R" or "-Wl,-R" etc).  I'm going to argue
> that this needs to be extened pass in -Wl,--enable-new-dtags,-R if the
> GNU linker is used so that the newer and better RUNPATH gets put into
> the shared objects all the time.  (I don't yet know how to detect the
> GNU linker but would like a consensus on the desired behaviour before
> looking into this).

The patch for this is attached to http://bugs.python.org/issue5900
Feel free to give feedback/criticism if you care about this.


> The second issue with build_ext --rpath is on AIX.  Again some
> background on AIX shared objects, AIX is not SysV-like and uses the
> XCOFF binary format instead of ELF.  Therefore they don't have a RPATH
> or RUNPATH, but they do have a think called LIBPATH which does
> something similar.  The difference between XCOFF's LIBPATH and ELF's
> RUNPATH is that AIX's runtime linker does not have a default search
> path, hence the full search path needs to be encoded into the LIBPATH
> of the shared object.
> 
> Now I would propose for build_ext --rpath to encode the LIBPATH when
> used on AIX since that is the correct thing to do IMHO.

I was kind of hoping someone would pick up on this and at least argue
about how it is not a good thing to try and convert an option for one
concept (RPATH/RUNPATH) onto another one that isn't quite the same
(LIBPATH).  The side-effects are just not as clearly defined and open
to interpretation.

Anyway, having given this some more tought I now think it's not a good
thing.  Instead I'm going to make a patch that raises and exception if
the --rpath option is used on AIX as it's behaviour is undefined[0].  If
a user does want -blibpath:/some/path to be passed on to the linker
they can still use LDFLAGS.


Other opinions would be appreciated.

Regards
Floris


[0] The linker will actually simply ignore it lurring the user into
    false security.  Only when -bsvr4 is passed to the linker too will
    it be treated identical as -blibpath, but this is again the same
    fuzzy mapping as a SysV user will not be used to passing in the
    default search path to -R/--rpath and hence probably not end up
    with what they wanted too.

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Sun May  3 12:03:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 3 May 2009 12:03:26 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
Message-ID: <94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>

On Thu, Apr 30, 2009 at 9:34 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi,
>
> I have reworked the PEP a little bit with people feedback.
>
> It needs more feedback : http://svn.python.org/projects/peps/trunk/pep-0376.txt
>
> - install/uninstall script
>
> ?I think the best solution is not to provide an install script since
> third-party tool do it. Furthermore,
> ?there's already the simplest install script available today: you can
> run "python setup.py install" on a given package
> ?so it gets installed.
>
> ?So what about adding just a global uninstall feature, that
> uninstalls the files installed for a package, using the record
> ?file, and let the third party tool have better features.
>
> - MANIFEST (SOURCES.txt)
>
> ?Ronald pointed out that it was not necessary to have a MANIFEST file
> included if we are going to have a RECORD
> ?file, as described in the PEP.
>
> ?The MANIFEST (which in a way is equivalent to SOURCES.txt pip or
> easy_install adds) is the list of source files,
> ?whereas the RECORD is the list if installed files (which might
> include more elements in case of compilations)
>
> ?What would be the interest of having the list of source files in egg-info ?
>

There's another thing I have thought of, for the file list in egg-info
: make it configurable.

So let's:

- make the change in the "install_egg_info" command to create the
egg-info directory
- push the PKG-INFO file into it
- provide a new option for the command for people to point other files to add
- change the "install" command so it adds the RECORD file in this list

With that change, third-party tools will be able to configure
"install_egg_info" in order
to list extra files to add into the egg-info directory when the
package is installed,
whitout having to override that command.

The option would be a simple list of files path. Each path can be:

- a path relative to the root of the project that is being installed,
which allows you to provide a file that is present
  in your distribution tree
- an absolute path,

The name of each file will have to be normalized: all upper case with
no extensions.

Any opinions ?

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Sun May  3 19:15:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 03 May 2009 13:15:10 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
Message-ID: <20090503171236.7D21D3A4080@sparrow.telecommunity.com>

At 12:03 PM 5/3/2009 +0200, Tarek Ziad? wrote:
>The name of each file will have to be normalized: all upper case with
>no extensions.
>
>Any opinions ?

I don't see any point to the normalization.  However, being able to 
install arbitrary files in .egg-info is currently supported by 
setuptools, and that capability is used by e.g. the EggTranslations project.

I would therefore suggest that perhaps the ability to specify file 
trees for egg-info inclusion would be better, if the goal is to 
eventually replace setuptools.


From ziade.tarek at gmail.com  Sun May  3 19:34:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 3 May 2009 19:34:49 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090503171236.7D21D3A4080@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
Message-ID: <94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>

2009/5/3 P.J. Eby <pje at telecommunity.com>:
> At 12:03 PM 5/3/2009 +0200, Tarek Ziad? wrote:
>>
>> The name of each file will have to be normalized: all upper case with
>> no extensions.
>>
>> Any opinions ?
>
> I don't see any point to the normalization.

To avoid different naming conventions like:

PKG-INFO, requires.txt, SOURCES.txt

> ?However, being able to install
> arbitrary files in .egg-info is currently supported by setuptools, and that
> capability is used by e.g. the EggTranslations project.
>
> I would therefore suggest that perhaps the ability to specify file trees for
> egg-info inclusion would be better,

Are you talking about the directory that is built by egg_info in setuptools ?
or something else ?

I mean, do we want to have subdirectories in .egg-info ?

>  if the goal is to eventually replace
> setuptools.

The goal is to provide an install_egg_info command that can be configured
without the need to replace/override it.

I guess, this can be done by taking part of the code done in setuptools
install_egg_info and provide a way for setuptools or another project
to add more files
through configuration.

Now, if entry_points were added in Distutils, things would be more
simple to extend
every command I guess. (like you have pointed)

That's some other work we started here :
http://wiki.python.org/moin/Distutils/PluginSystem

But my mail to python-ideas about adding a plugin system into Python
didn't get a lot
of interest.

Maybe this straight-forward sentence would work better:

"let's add entry points in Distutils" ?


Tarek

-- 
Tarek Ziad? | http://ziade.org

From yanegomi at gmail.com  Mon May  4 11:43:42 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Mon, 4 May 2009 02:43:42 -0700
Subject: [Distutils] Anyone stuck with easy_install / has .pth file issues
	-- this is for you!
Message-ID: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>

Hi guys,
    Just thought I'd might provide this script to fellow developers
which fixes .pth files (easy-install.pth / .egg was the prime target
-- see the comments for more details):
<http://yaneurabeya.livejournal.com/3929.html>.
    Comments are more than welcome. If I get any, I'll try to get back
in touch sometime before next weekend (my weekdays have been extremely
busy lately thanks to a pending work release schedule -_-...).
HTH and thanks!
-Garrett

From pje at telecommunity.com  Mon May  4 17:20:13 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 11:20:13 -0400
Subject: [Distutils] Anyone stuck with easy_install / has .pth file
 issues -- this is for you!
In-Reply-To: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com
 >
References: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>
Message-ID: <20090504151737.93AD33A40A5@sparrow.telecommunity.com>

At 02:43 AM 5/4/2009 -0700, Garrett Cooper wrote:
>Hi guys,
>     Just thought I'd might provide this script to fellow developers
>which fixes .pth files (easy-install.pth / .egg was the prime target
>-- see the comments for more details):
><http://yaneurabeya.livejournal.com/3929.html>.
>     Comments are more than welcome.

As far as I can tell, it doesn't do anything that "easy_install -mxN" 
doesn't, although it appears to also convert paths of this form:

    /foo/bar/baz/foo/bar/spam

into:

    ./baz./spam

if I'm reading the code correctly.  It also seems to have no 
protection against adding multiple versions of the same project to a 
.pth file, and to ignore development eggs, whether or not their 
directories still exist.

In contrast, easy_install already removes non-existent 
files/directories whenever it touches easy_install.pth, and if you 
gave it a command line globbing the same files as this tool (i.e., 
just "easy_install [list of eggs]"), you'd at least end up without 
any duplicates in the .pth file.

In short, AFAICT, you could replace the entire tool with a short note 
on how to accomplish the same things using easy_install, or by simply 
having it invoke easy_install internally.


From ziade.tarek at gmail.com  Mon May  4 17:23:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 4 May 2009 17:23:40 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
Message-ID: <94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>

There's another point I was thinking about in PEP 376

What about dropping the 'egg' part in 'PROJECT.egg-info'  ? and replace it with

'PROJECT.info'

(and make the 2.7 version compatible with PROJECT.egg-info )

I know it's a minor change, but it seems that a lot of people are
confused with this,
when they don't know what is an egg, etc.

Tarek

On Thu, Apr 30, 2009 at 9:34 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi,
>
> I have reworked the PEP a little bit with people feedback.
>
> It needs more feedback : http://svn.python.org/projects/peps/trunk/pep-0376.txt
>
> - install/uninstall script
>
> ?I think the best solution is not to provide an install script since
> third-party tool do it. Furthermore,
> ?there's already the simplest install script available today: you can
> run "python setup.py install" on a given package
> ?so it gets installed.
>
> ?So what about adding just a global uninstall feature, that
> uninstalls the files installed for a package, using the record
> ?file, and let the third party tool have better features.
>
> - MANIFEST (SOURCES.txt)
>
> ?Ronald pointed out that it was not necessary to have a MANIFEST file
> included if we are going to have a RECORD
> ?file, as described in the PEP.
>
> ?The MANIFEST (which in a way is equivalent to SOURCES.txt pip or
> easy_install adds) is the list of source files,
> ?whereas the RECORD is the list if installed files (which might
> include more elements in case of compilations)
>
> ?What would be the interest of having the list of source files in egg-info ?
>
>
> Cheers
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
>



-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Mon May  4 17:46:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 11:46:37 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
Message-ID: <20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>

At 05:23 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>There's another point I was thinking about in PEP 376
>
>What about dropping the 'egg' part in 'PROJECT.egg-info'  ? and 
>replace it with
>
>'PROJECT.info'
>
>(and make the 2.7 version compatible with PROJECT.egg-info )
>
>I know it's a minor change,

Actually, it's a major change, since it means dropping backward 
compatibility with existing tools.


From pje at telecommunity.com  Mon May  4 17:48:50 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 11:48:50 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com
 >
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>
Message-ID: <20090504154612.060E53A40A5@sparrow.telecommunity.com>

At 07:34 PM 5/3/2009 +0200, Tarek Ziad? wrote:
>2009/5/3 P.J. Eby <pje at telecommunity.com>:
> > At 12:03 PM 5/3/2009 +0200, Tarek Ziad? wrote:
> >>
> >> The name of each file will have to be normalized: all upper case with
> >> no extensions.
> >>
> >> Any opinions ?
> >
> > I don't see any point to the normalization.
>
>To avoid different naming conventions like:
>
>PKG-INFO, requires.txt, SOURCES.txt

And the problem with that is...?


> >  However, being able to install
> > arbitrary files in .egg-info is currently supported by setuptools, and that
> > capability is used by e.g. the EggTranslations project.
> >
> > I would therefore suggest that perhaps the ability to specify 
> file trees for
> > egg-info inclusion would be better,
>
>Are you talking about the directory that is built by egg_info in setuptools ?
>or something else ?
>
>I mean, do we want to have subdirectories in .egg-info ?

They already exist.


> >  if the goal is to eventually replace
> > setuptools.
>
>The goal is to provide an install_egg_info command that can be configured
>without the need to replace/override it.
>
>I guess, this can be done by taking part of the code done in setuptools
>install_egg_info and provide a way for setuptools or another project
>to add more files
>through configuration.

setuptools simply copies a pre-built egg-info directory from the 
source tree, that is built by the "egg_info" command.  But people can 
also put arbitrary files and directories there, besides the contents 
built by the command itself.



>Maybe this straight-forward sentence would work better:
>
>"let's add entry points in Distutils" ?

+1


From ziade.tarek at gmail.com  Mon May  4 17:54:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 4 May 2009 17:54:59 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>

2009/5/4 P.J. Eby <pje at telecommunity.com>:
> At 05:23 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>>
>> There's another point I was thinking about in PEP 376
>>
>> What about dropping the 'egg' part in 'PROJECT.egg-info' ?? and replace it
>> with
>>
>> 'PROJECT.info'
>>
>> (and make the 2.7 version compatible with PROJECT.egg-info )
>>
>> I know it's a minor change,
>
> Actually, it's a major change, since it means dropping backward
> compatibility with existing tools.

well if Distutils 2.7 / 3.1 keeps backward compatibility like I said,
that would give 3 years for existing
tools to change..

But what about the idea ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Mon May  4 18:01:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 4 May 2009 18:01:06 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090504154612.060E53A40A5@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>
	<20090504154612.060E53A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905040901v420efe94y751f799c3c9231e3@mail.gmail.com>

On Mon, May 4, 2009 at 5:48 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > I don't see any point to the normalization.
>>
>> To avoid different naming conventions like:
>>
>> PKG-INFO, requires.txt, SOURCES.txt
>
> And the problem with that is...?

inconsistency, but right, it makes no sense if any file/dir can be added there.

What about SOURCES.txt btw ? What is the reason to add it ?

> setuptools simply copies a pre-built egg-info directory from the source
> tree, that is built by the "egg_info" command. ?But people can also put
> arbitrary files and directories there, besides the contents built by the
> command itself.

Ok

>
>
>
>> Maybe this straight-forward sentence would work better:
>>
>> "let's add entry points in Distutils" ?
>
> +1

I think this can be a subpart of this PEP proposal then,



-- 
Tarek Ziad? | http://ziade.org

From floris.bruynooghe at gmail.com  Mon May  4 18:12:43 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 4 May 2009 17:12:43 +0100
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
Message-ID: <20090504161243.GA15980@laurie.devork>

On Mon, May 04, 2009 at 05:54:59PM +0200, Tarek Ziad? wrote:
> 2009/5/4 P.J. Eby <pje at telecommunity.com>:
> > At 05:23 PM 5/4/2009 +0200, Tarek Ziad? wrote:
> >>
> >> There's another point I was thinking about in PEP 376
> >>
> >> What about dropping the 'egg' part in 'PROJECT.egg-info' ?? and replace it
> >> with
> >>
> >> 'PROJECT.info'
> >>
> >> (and make the 2.7 version compatible with PROJECT.egg-info )
> >>
> >> I know it's a minor change,
> >
> > Actually, it's a major change, since it means dropping backward
> > compatibility with existing tools.
> 
> well if Distutils 2.7 / 3.1 keeps backward compatibility like I said,
> that would give 3 years for existing
> tools to change..
> 
> But what about the idea ?


How can we be sure that we won't want to change it again in the
future?  As for PROJECT.info, that still doesn't say that the .info
directory describes a project to me.  Not that I like .egg-info, but I
guess PJE's argument that it is a major change is valid and just .info
isn't quite better enough for me to justify the pain (i.e. the name
doesn't sound so right that no one will ever want to change it again).


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Mon May  4 18:31:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 4 May 2009 18:31:09 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090504161243.GA15980@laurie.devork>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
	<20090504161243.GA15980@laurie.devork>
Message-ID: <94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>

On Mon, May 4, 2009 at 6:12 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
>>
>> But what about the idea ?
>
>
> How can we be sure that we won't want to change it again in the
> future?

well I think it's now or never, since we are defining a standard here
for this directory.


> As for PROJECT.info, that still doesn't say that the .info
> directory describes a project to me. ?Not that I like .egg-info, but I
> guess PJE's argument that it is a major change is valid and just .info
> isn't quite better enough for me to justify the pain (i.e. the name
> doesn't sound so right that no one will ever want to change it again).

Ok then, we will have to provide extra documentation to make people
understand that the '.egg-info' directory has absolutely nothing to do
with egg-the-format
but is rather a metadata container.

'egg-info' was introduced with adding the whole 'egg' thing in Python
in mind at some point I believe.

And it seems that the .egg directory/zip file for projects setuptools
provides will not make it into Python
and is still very controversial
(http://www.reddit.com/r/Python/comments/8he79/am_i_alone_in_feeling_like_python_got_a_whole_lot/)

So removing the 'egg' part of 'egg-info' seemed natural to me at this point.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Mon May  4 19:43:23 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 13:43:23 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
Message-ID: <20090504174047.16DE83A40A5@sparrow.telecommunity.com>

At 05:54 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>2009/5/4 P.J. Eby <pje at telecommunity.com>:
> > At 05:23 PM 5/4/2009 +0200, Tarek Ziad? wrote:
> >>
> >> There's another point I was thinking about in PEP 376
> >>
> >> What about dropping the 'egg' part in 'PROJECT.egg-info'  ? and replace it
> >> with
> >>
> >> 'PROJECT.info'
> >>
> >> (and make the 2.7 version compatible with PROJECT.egg-info )
> >>
> >> I know it's a minor change,
> >
> > Actually, it's a major change, since it means dropping backward
> > compatibility with existing tools.
>
>well if Distutils 2.7 / 3.1 keeps backward compatibility like I said,
>that would give 3 years for existing
>tools to change..

Actually, writing '.info' files means that existing tools that expect 
.egg-info files or directories would stop working correctly *right away*.


From pje at telecommunity.com  Mon May  4 19:44:50 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 13:44:50 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
Message-ID: <20090504174216.8252D3A40A5@sparrow.telecommunity.com>

At 06:31 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>Ok then, we will have to provide extra documentation to make people
>understand that the '.egg-info' directory has absolutely nothing to do
>with egg-the-format
>but is rather a metadata container.

On the contrary; .egg-info *is* an egg format; see the EggFormats 
documentation.


From hannosch at hannosch.eu  Mon May  4 19:42:48 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Mon, 04 May 2009 19:42:48 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
Message-ID: <gtn9eo$u2m$1@ger.gmane.org>

Tarek Ziad? wrote:
> On Mon, May 4, 2009 at 6:12 PM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com> wrote:
> 
> Ok then, we will have to provide extra documentation to make people
> understand that the '.egg-info' directory has absolutely nothing to do
> with egg-the-format
> but is rather a metadata container.

> So removing the 'egg' part of 'egg-info' seemed natural to me at this point.

That seems reasonable to me. Just name it something a little more
verbose, like pkg-info or metadata or whatever it actually is that it
contains.

And since this is such a wonderful bikeshed opportunity, I'd suggest you
just pick whatever concept-neutral name you personally like the best ;)

Hanno


From ianb at colorstudy.com  Mon May  4 19:43:14 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 4 May 2009 12:43:14 -0500
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com> 
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com> 
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com> 
	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
Message-ID: <b654cd2e0905041043s222a0562t35a14a6da71261da@mail.gmail.com>

On Mon, May 4, 2009 at 11:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Ok then, we will have to provide extra documentation to make people
> understand that the '.egg-info' directory has absolutely nothing to do
> with egg-the-format
> but is rather a metadata container.
>
> 'egg-info' was introduced with adding the whole 'egg' thing in Python
> in mind at some point I believe.
>
> And it seems that the .egg directory/zip file for projects setuptools
> provides will not make it into Python
> and is still very controversial
> (
> http://www.reddit.com/r/Python/comments/8he79/am_i_alone_in_feeling_like_python_got_a_whole_lot/
> )
>
> So removing the 'egg' part of 'egg-info' seemed natural to me at this
> point.
>

Egg-the-format is what we are recreating in distutils, isn't it?  Obviously
some people are unhappy with some things related to packaging, but I don't
think egg-the-format is something people actually mind (if they know what it
is).  Of course few people really know what the format is, or are able to
distinguish it from other parts of the Setuptools stack, but that doesn't
change just because you rename the extension.  Eggs don't even carry any
particular naming attachment to Setuptools.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090504/afa5d9f8/attachment.htm>

From pje at telecommunity.com  Mon May  4 19:51:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 13:51:04 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905040901v420efe94y751f799c3c9231e3@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>
	<20090504154612.060E53A40A5@sparrow.telecommunity.com>
	<94bdd2610905040901v420efe94y751f799c3c9231e3@mail.gmail.com>
Message-ID: <20090504174829.93FBC3A40A5@sparrow.telecommunity.com>

At 06:01 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>On Mon, May 4, 2009 at 5:48 PM, P.J. Eby <pje at telecommunity.com> wrote:
> >> > I don't see any point to the normalization.
> >>
> >> To avoid different naming conventions like:
> >>
> >> PKG-INFO, requires.txt, SOURCES.txt
> >
> > And the problem with that is...?
>
>inconsistency, but right, it makes no sense if any file/dir can be 
>added there.
>
>What about SOURCES.txt btw ? What is the reason to add it ?

It's for source distributions.  It allows them to be able to rebuild 
an identical source distribution in the absence of source control metadata.

It's not really necessary for the installation process, although it's 
used to figure out which files to install if you use include_package_data=True. 


From yanegomi at gmail.com  Mon May  4 22:11:52 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Mon, 4 May 2009 13:11:52 -0700
Subject: [Distutils] Anyone stuck with easy_install / has .pth file
	issues -- this is for you!
In-Reply-To: <20090504151737.93AD33A40A5@sparrow.telecommunity.com>
References: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>
	<20090504151737.93AD33A40A5@sparrow.telecommunity.com>
Message-ID: <364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com>

Hi P.J.!

On Mon, May 4, 2009 at 8:20 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 02:43 AM 5/4/2009 -0700, Garrett Cooper wrote:
>>
>> Hi guys,
>> ? ?Just thought I'd might provide this script to fellow developers
>> which fixes .pth files (easy-install.pth / .egg was the prime target
>> -- see the comments for more details):
>> <http://yaneurabeya.livejournal.com/3929.html>.
>> ? ?Comments are more than welcome.
>
> As far as I can tell, it doesn't do anything that "easy_install -mxN"
> doesn't, although it appears to also convert paths of this form:
>
> ? /foo/bar/baz/foo/bar/spam
>
> into:
>
> ? ./baz./spam
>
> if I'm reading the code correctly. ?It also seems to have no protection
> against adding multiple versions of the same project to a .pth file

You're right -- it doesn't protect against the following (><):

/full/path/to/package.egg
./package.egg

It does protect against ./package.egg in .pth not existing in the
filesystem, and vice versa, depending on the glob of choice employed.
I'll fix that gap tonight :).

BTW, wasn't -m deprecated? I mean, -m's `delivered' functionality
doesn't even work because python stores all loaded package / module
entries in .pth files as keys in a dictionary _anyhow_ (hence the
proposal I made a few weeks ago with __import__ to do version
checking)..

setuptools can't even fix this behavior (python does a BFS-like
package search based on the module / package name and returns the
first entry found), even though it's _sort_ of supposed to (look for
"A more complete solution" on
<http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html>).
Feel free to check sys.modules if you don't believe me :). Hence the
need for virtualenv.

I have no idea how -xN factor into this though -- maybe I'm missing an
important easy-install usage note... either way, we're stuck with
0.6c3 at work because the folks we inherited it from hacked the code
(in an ugly way... ew) and I don't have time to fix it now, so does
this still apply ;\...?

> and to ignore development eggs, whether or not their directories still exist.

By default, no. If one wants to avoid a series of directories, they
can omit entries with [a] relevant expression(s) via the -g / -r
option(s).

> In contrast, easy_install already removes non-existent files/directories
> whenever it touches easy_install.pth, and if you gave it a command line
> globbing the same files as this tool (i.e., just "easy_install [list of
> eggs]"), you'd at least end up without any duplicates in the .pth file.
>
> In short, AFAICT, you could replace the entire tool with a short note on how
> to accomplish the same things using easy_install, or by simply having it
> invoke easy_install internally.

- The duplicates erasure was just gravy, and purely an exercise on my
part getting used to set's in python. The real point was the missing
package in .pth functionality, caused by easy_install / setuptools
lacking proper MP-safe logic.
- It's a simple tool (took a while to think up, but took 1.5 hours to
code / validate) designed to function outside of easy_install /
setuptools, even though that was the piece I was trying to target...
so if someone hand edits a .pth file (heaven forbid) it would note
that the file needed to be fixed / fix the file =].

Thanks for the comments!
-Garrett

From yanegomi at gmail.com  Mon May  4 22:17:30 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Mon, 4 May 2009 13:17:30 -0700
Subject: [Distutils] Anyone stuck with easy_install / has .pth file
	issues -- this is for you!
In-Reply-To: <364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com>
References: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>
	<20090504151737.93AD33A40A5@sparrow.telecommunity.com>
	<364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com>
Message-ID: <364299f40905041317r600889can1b6c9ad470c034ff@mail.gmail.com>

On Mon, May 4, 2009 at 1:11 PM, Garrett Cooper <yanegomi at gmail.com> wrote:
> Hi P.J.!
>
> On Mon, May 4, 2009 at 8:20 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 02:43 AM 5/4/2009 -0700, Garrett Cooper wrote:
>>>
>>> Hi guys,
>>> ? ?Just thought I'd might provide this script to fellow developers
>>> which fixes .pth files (easy-install.pth / .egg was the prime target
>>> -- see the comments for more details):
>>> <http://yaneurabeya.livejournal.com/3929.html>.
>>> ? ?Comments are more than welcome.
>>
>> As far as I can tell, it doesn't do anything that "easy_install -mxN"
>> doesn't, although it appears to also convert paths of this form:
>>
>> ? /foo/bar/baz/foo/bar/spam
>>
>> into:
>>
>> ? ./baz./spam
>>
>> if I'm reading the code correctly. ?It also seems to have no protection
>> against adding multiple versions of the same project to a .pth file
>
> You're right -- it doesn't protect against the following (><):
>
> /full/path/to/package.egg
> ./package.egg
>
> It does protect against ./package.egg in .pth not existing in the
> filesystem, and vice versa, depending on the glob of choice employed.
> I'll fix that gap tonight :).
>
> BTW, wasn't -m deprecated? I mean, -m's `delivered' functionality
> doesn't even work because python stores all loaded package / module
> entries in .pth files as keys in a dictionary _anyhow_ (hence the
> proposal I made a few weeks ago with __import__ to do version
> checking)..
>
> setuptools can't even fix this behavior (python does a BFS-like
> package search based on the module / package name and returns the
> first entry found), even though it's _sort_ of supposed to (look for
> "A more complete solution" on
> <http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html>).
> Feel free to check sys.modules if you don't believe me :). Hence the
> need for virtualenv.
>
> I have no idea how -xN factor into this though -- maybe I'm missing an
> important easy-install usage note... either way, we're stuck with
> 0.6c3 at work because the folks we inherited it from hacked the code
> (in an ugly way... ew) and I don't have time to fix it now, so does
> this still apply ;\...?
>
>> and to ignore development eggs, whether or not their directories still exist.
>
> By default, no. If one wants to avoid a series of directories, they
> can omit entries with [a] relevant expression(s) via the -g / -r
> option(s).
>
>> In contrast, easy_install already removes non-existent files/directories
>> whenever it touches easy_install.pth, and if you gave it a command line
>> globbing the same files as this tool (i.e., just "easy_install [list of
>> eggs]"), you'd at least end up without any duplicates in the .pth file.
>>
>> In short, AFAICT, you could replace the entire tool with a short note on how
>> to accomplish the same things using easy_install, or by simply having it
>> invoke easy_install internally.
>
> - The duplicates erasure was just gravy, and purely an exercise on my
> part getting used to set's in python. The real point was the missing
> package in .pth functionality, caused by easy_install / setuptools
> lacking proper MP-safe logic.

I do need to check for commented entries though / expand the search
criteria or something... excellent point made P.J.

> - It's a simple tool (took a while to think up, but took 1.5 hours to
> code / validate) designed to function outside of easy_install /
> setuptools, even though that was the piece I was trying to target...
> so if someone hand edits a .pth file (heaven forbid) it would note
> that the file needed to be fixed / fix the file =].
>
> Thanks for the comments!
> -Garrett

From pje at telecommunity.com  Mon May  4 22:46:14 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 16:46:14 -0400
Subject: [Distutils] Anyone stuck with easy_install / has .pth file
 issues -- this is for you!
In-Reply-To: <364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com
 >
References: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>
	<20090504151737.93AD33A40A5@sparrow.telecommunity.com>
	<364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com>
Message-ID: <20090504204338.115EF3A40A5@sparrow.telecommunity.com>

At 01:11 PM 5/4/2009 -0700, Garrett Cooper wrote:
>You're right -- it doesn't protect against the following (><):
>
>/full/path/to/package.egg
>./package.egg

By duplicates, I meant 'package-1.0.egg' and 'package-1.1.egg', not 
alternate paths to the same file.

(As for the '.' replacement, you might want to look at what I said 
more closely.)


>BTW, wasn't -m deprecated?

No.

>  I mean, -m's `delivered' functionality
>doesn't even work

You'll need to be more specific there: I don't know what you're 
saying "doesn't work", let alone *how*, what behavior you expect, or 
what you're seeing instead.


>because python stores all loaded package / module
>entries in .pth files as keys in a dictionary _anyhow_

I also don't know what you mean here.  .pth file contents are added 
to sys.path, not a dictionary.  Unless you're talking about 
sys.path_importer_cache or something, in which case I still don't see 
the connection to anything else at hand.


>I have no idea how -xN factor into this though

The original docs said to use -m before uninstalling, but they really 
should say -mxN to exclude scripts and avoid installing (or 
uninstalling!) dependencies.  The docs are broken (or at any rate, 
suboptimal) and should be fixed.


>- The duplicates erasure was just gravy, and purely an exercise on my
>part getting used to set's in python. The real point was the missing
>package in .pth functionality, caused by easy_install / setuptools
>lacking proper MP-safe logic.

I'm going to guess that by "MP" you mean multiprocessing, in which 
case I would say, yes, easy_install assumes that you are not running 
multiple simultaneous installations to the same directory.


From ziade.tarek at gmail.com  Tue May  5 00:26:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 00:26:43 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090504174216.8252D3A40A5@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
	<20090504174216.8252D3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905041526m42e949b9t3c8cb6bdd23a9518@mail.gmail.com>

2009/5/4 P.J. Eby <pje at telecommunity.com>:
> At 06:31 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>>
>> Ok then, we will have to provide extra documentation to make people
>> understand that the '.egg-info' directory has absolutely nothing to do
>> with egg-the-format
>> but is rather a metadata container.
>
> On the contrary; .egg-info *is* an egg format; see the EggFormats
> documentation.

Sorry, when I said "egg-the-format" I was talking about  ".egg"  the directory:

"directory or zipfile containing the project's code and resources,
along with an EGG-INFO subdirectory that contains the project's
metadata"...

Tarek
-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 00:31:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 00:31:10 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <b654cd2e0905041043s222a0562t35a14a6da71261da@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
	<b654cd2e0905041043s222a0562t35a14a6da71261da@mail.gmail.com>
Message-ID: <94bdd2610905041531j45687957h4415a47bcd81b643@mail.gmail.com>

On Mon, May 4, 2009 at 7:43 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> On Mon, May 4, 2009 at 11:31 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Ok then, we will have to provide extra documentation to make people
>> understand that the '.egg-info' directory has absolutely nothing to do
>> with egg-the-format
>> but is rather a metadata container.
>>
>> 'egg-info' was introduced with adding the whole 'egg' thing in Python
>> in mind at some point I believe.
>>
>> And it seems that the .egg directory/zip file for projects setuptools
>> provides will not make it into Python
>> and is still very controversial
>>
>> (http://www.reddit.com/r/Python/comments/8he79/am_i_alone_in_feeling_like_python_got_a_whole_lot/)
>>
>> So removing the 'egg' part of 'egg-info' seemed natural to me at this
>> point.
>
> Egg-the-format is what we are recreating in distutils, isn't it?

for the .egg-info format yes, but the .egg format is not in the plan.
(that what I called egg-the-format
but I was wrong since .egg-info is also called a format in setuptools doc)

but what is really is (even if it's part of the egg format) is a
metadata container,
(also present in .egg/EGG-INFO)


> Obviously
> some people are unhappy with some things related to packaging, but I don't
> think egg-the-format is something people actually mind (if they know what it
> is).? Of course few people really know what the format is, or are able to
> distinguish it from other parts of the Setuptools stack, but that doesn't
> change just because you rename the extension.? Eggs don't even carry any
> particular naming attachment to Setuptools.

Sure that makes sense,

Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 00:50:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 00:50:35 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090504174829.93FBC3A40A5@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>
	<20090504154612.060E53A40A5@sparrow.telecommunity.com>
	<94bdd2610905040901v420efe94y751f799c3c9231e3@mail.gmail.com>
	<20090504174829.93FBC3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905041550r4aa9dfffld25536f184f91232@mail.gmail.com>

On Mon, May 4, 2009 at 7:51 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:01 PM 5/4/2009 +0200, Tarek Ziad? wrote:
>>
>> On Mon, May 4, 2009 at 5:48 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> >> > I don't see any point to the normalization.
>> >>
>> >> To avoid different naming conventions like:
>> >>
>> >> PKG-INFO, requires.txt, SOURCES.txt
>> >
>> > And the problem with that is...?
>>
>> inconsistency, but right, it makes no sense if any file/dir can be added
>> there.
>>
>> What about SOURCES.txt btw ? What is the reason to add it ?
>
> It's for source distributions. ?It allows them to be able to rebuild an
> identical source distribution in the absence of source control metadata.
>
> It's not really necessary for the installation process, although it's used
> to figure out which files to install if you use include_package_data=True.
>

Any particular reason to call it "SOURCES.txt"  ?

Or we can call it MANIFEST (with '/'-separated relative path)


-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 00:54:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 00:54:26 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <gtn9eo$u2m$1@ger.gmane.org>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905040823h444253adm150e203ef91e15cd@mail.gmail.com>
	<20090504154401.A5F5D3A40A5@sparrow.telecommunity.com>
	<94bdd2610905040854m76d000adj2df6d9e2f5ab4812@mail.gmail.com>
	<20090504161243.GA15980@laurie.devork>
	<94bdd2610905040931n41f46946j8798da3696684ff1@mail.gmail.com>
	<gtn9eo$u2m$1@ger.gmane.org>
Message-ID: <94bdd2610905041554i51d44f2eqa0c766a67ae6f460@mail.gmail.com>

On Mon, May 4, 2009 at 7:42 PM, Hanno Schlichting <hannosch at hannosch.eu> wrote:
> Tarek Ziad? wrote:
>> On Mon, May 4, 2009 at 6:12 PM, Floris Bruynooghe
>> <floris.bruynooghe at gmail.com> wrote:
>>
>> Ok then, we will have to provide extra documentation to make people
>> understand that the '.egg-info' directory has absolutely nothing to do
>> with egg-the-format
>> but is rather a metadata container.
>
>> So removing the 'egg' part of 'egg-info' seemed natural to me at this point.
>
> That seems reasonable to me. Just name it something a little more
> verbose, like pkg-info or metadata or whatever it actually is that it
> contains.
>
> And since this is such a wonderful bikeshed opportunity, I'd suggest you
> just pick whatever concept-neutral name you personally like the best ;)

Yes that was my point, but I can understand now why it is not the best move

Tarek

From pje at telecommunity.com  Tue May  5 01:18:48 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 19:18:48 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905041550r4aa9dfffld25536f184f91232@mail.gmail.co
 m>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905031034n10b0015frb1bb083a937217f@mail.gmail.com>
	<20090504154612.060E53A40A5@sparrow.telecommunity.com>
	<94bdd2610905040901v420efe94y751f799c3c9231e3@mail.gmail.com>
	<20090504174829.93FBC3A40A5@sparrow.telecommunity.com>
	<94bdd2610905041550r4aa9dfffld25536f184f91232@mail.gmail.com>
Message-ID: <20090504231613.4EB8A3A40A5@sparrow.telecommunity.com>

At 12:50 AM 5/5/2009 +0200, Tarek Ziad? wrote:
>On Mon, May 4, 2009 at 7:51 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 06:01 PM 5/4/2009 +0200, Tarek Ziad? wrote:
> >>
> >> On Mon, May 4, 2009 at 5:48 PM, P.J. Eby <pje at telecommunity.com> wrote:
> >> >> > I don't see any point to the normalization.
> >> >>
> >> >> To avoid different naming conventions like:
> >> >>
> >> >> PKG-INFO, requires.txt, SOURCES.txt
> >> >
> >> > And the problem with that is...?
> >>
> >> inconsistency, but right, it makes no sense if any file/dir can be added
> >> there.
> >>
> >> What about SOURCES.txt btw ? What is the reason to add it ?
> >
> > It's for source distributions.  It allows them to be able to rebuild an
> > identical source distribution in the absence of source control metadata.
> >
> > It's not really necessary for the installation process, although it's used
> > to figure out which files to install if you use include_package_data=True.
> >
>
>Any particular reason to call it "SOURCES.txt"  ?
>
>Or we can call it MANIFEST (with '/'-separated relative path)

I called it SOURCES.txt because MANIFEST is ambiguous as to what it's 
a manifest *of*, and also to distinguish it from any user-generated 
MANIFEST file.  (If you don't know about those, you're probably going 
to break backward compatibility w/somebody, btw.  Distutils usage is 
a diverse collection of nightmares.)

Anyway, the name of the file has no bearing on what distutils does, 
unless distutils is trying to implement the same feature as 
setuptools: i.e. round-trippable sdists whose original manifest was 
generated via revision control.


From ziade.tarek at gmail.com  Tue May  5 01:37:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 01:37:34 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <20090503171236.7D21D3A4080@sparrow.telecommunity.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
Message-ID: <94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>

2009/5/3 P.J. Eby <pje at telecommunity.com>:
> At 12:03 PM 5/3/2009 +0200, Tarek Ziad? wrote:
>>
>> The name of each file will have to be normalized: all upper case with
>> no extensions.
>>
>> Any opinions ?
>
> I don't see any point to the normalization. ?However, being able to install
> arbitrary files in .egg-info is currently supported by setuptools, and that
> capability is used by e.g. the EggTranslations project.

I am discovering a new callback mechanism here, I am confused.

I was looking at another package last week called "eggtestinfo", that
uses the 'egg_info.writers'
entry point to add package in the egg-info dir created by the egg_info command

So the callbacks are used at installation time and the entry points
only at packaging time,

but I don't understand how EggTranslations is used since it registers
callback only when the code is called

Furtermore, if we provide the ability to fill egg-info with third
party packages registered through
a plugin system, it make sense to prepare it at packaging time to
avoid having to install this third party package
on the target system.

So 'egg_info.writers' + the egg_info command is what makes sense right
now to me.

Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 01:46:21 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 01:46:21 +0200
Subject: [Distutils] Adding entry points into Distutils ?
Message-ID: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>

Hello

I am making a separate email for this topic to make sure no one misses it.

There are *many* benefits of adding entry points into Distutils. In
fact, adding a plugin system in there,
will help make the commands more extendable and therefore will help us
in the long term to remove things
out of Distutils.

So any strong opinion against them ?

If not, I'll add a section in PEP 376 - and the first use case I can
see is having a pluggable way to extend
the list of files contained in .egg-info.

The work to be done would be to extract entry points from
pkg_resources (the discovery work that consists of looking
for entry_points.txt files will be covered by the APIs already
described in that PEP) This is not hard.


Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Tue May  5 01:47:56 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 04 May 2009 19:47:56 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
Message-ID: <154a4c839797fdf9eaa60501871a90e3@preisshare.net>


Hi Tarek,


On Tue, 5 May 2009 01:37:34 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
 
> Furtermore, if we provide the ability to fill egg-info with third
> party packages registered through
> a plugin system, it make sense to prepare it at packaging time to
> avoid having to install this third party package
> on the target system.

I'm trying to understand why there is a need for a "new" plug-in
system.

Isn't the site-packages directory already a very simple and
effective plug-in system?

Does not the .PTH file provision... with it's "import .." prefix
capability, allow any code to be run through the .PTH?

I'm not suggesting don't change anything. I'm just wondering
about the implications on my own project and trying to support
both old and new installations.

A lot of windows packages just come with their own windows
installer... how would your proposed system work with these?

Regards

David







From david.lyon at preisshare.net  Tue May  5 01:50:10 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 04 May 2009 19:50:10 -0400
Subject: [Distutils] =?utf-8?q?Adding_entry_points_into_Distutils_=3F?=
In-Reply-To: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
Message-ID: <e29d85fd1780cf2eb5a4950627bb57ad@preisshare.net>



On Tue, 5 May 2009 01:46:21 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> Hello
> 
> I am making a separate email for this topic to make sure no one misses
it.
> 
> There are *many* benefits of adding entry points into Distutils. In
> fact, adding a plugin system in there,
> will help make the commands more extendable and therefore will help us
> in the long term to remove things
> out of Distutils.
> 
> So any strong opinion against them ?

but what does it provide that we don't already get with the site-packages
directory? and .PTH?

isn't there already a plug-in system for packages in python?

David


From ianb at colorstudy.com  Tue May  5 01:57:31 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 4 May 2009 18:57:31 -0500
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
Message-ID: <b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>

On Mon, May 4, 2009 at 6:46 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello
>
> I am making a separate email for this topic to make sure no one misses it.
>
> There are *many* benefits of adding entry points into Distutils. In
> fact, adding a plugin system in there,
> will help make the commands more extendable and therefore will help us
> in the long term to remove things
> out of Distutils.
>
> So any strong opinion against them ?
>

Not strong, but I have a few issues with how they are currently defined:

* There's the issue of activated and unactivated eggs, of course, but I
guess that will be moot since there's no activation with just distutils?
* I'm uncomfortable with the way entry points are scanned.  I haven't looked
close enough to back it up with numbers, but I think there's a noticeable
performance degradation when the number of installed packages becomes
large.  (Given the algorithm this would be expected.)
* Scanning for entry points by name makes me uncomfortable, but I feel like
the API kind of encourages it.
* There's no idea of explicitly enabling an entry point, simply installing a
package makes the entry point show up.  Implicit plugins make me
uncomfortable.
* There's not much in the way of entry point documentation built into the
system.  The __doc__ of the entry point objects is one way, but there's no
way to document entry point groups.
* Entry points that provide functionality to the installation system itself
make me very uncomfortable, except insofar as they describe the package.
[console_scripts] is okay, but some of the other Setuptools extension points
bother me because of the self-referential nature, e.g., egg_file_writers.



-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090504/9c36bc69/attachment-0001.htm>

From ziade.tarek at gmail.com  Tue May  5 01:58:57 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 01:58:57 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <154a4c839797fdf9eaa60501871a90e3@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
	<154a4c839797fdf9eaa60501871a90e3@preisshare.net>
Message-ID: <94bdd2610905041658w33a2b8ddq78553583a25b1a5d@mail.gmail.com>

On Tue, May 5, 2009 at 1:47 AM, David Lyon <david.lyon at preisshare.net> wrote:
>
> Hi Tarek,
>
>
> On Tue, 5 May 2009 01:37:34 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>
>> Furtermore, if we provide the ability to fill egg-info with third
>> party packages registered through
>> a plugin system, it make sense to prepare it at packaging time to
>> avoid having to install this third party package
>> on the target system.
>
> I'm trying to understand why there is a need for a "new" plug-in
> system.
>
> Isn't the site-packages directory already a very simple and
> effective plug-in system?
>
> Does not the .PTH file provision... with it's "import .." prefix
> capability, allow any code to be run through the .PTH?

This is not a registery that indexes code under specific markers :
you are not able for instance to ask "give me all classes that are
filling the .egg-info
dir so I can use them on this 'foo' package"

for this you need a system that let you register your classes under
the 'write-into-egg-info'
group, with an unique name for each, no matter where the class is
located in your package.

That is what entry points are providing : the ability to mark a code
locate anywhere in your installation
and to load it when needed in your execution context.


>
> I'm not suggesting don't change anything. I'm just wondering
> about the implications on my own project and trying to support
> both old and new installations.
>
> A lot of windows packages just come with their own windows
> installer... how would your proposed system work with these?

I don't think it affects any of these topics.

Tarek


-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Tue May  5 02:00:28 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 04 May 2009 20:00:28 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905041658w33a2b8ddq78553583a25b1a5d@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
	<154a4c839797fdf9eaa60501871a90e3@preisshare.net>
	<94bdd2610905041658w33a2b8ddq78553583a25b1a5d@mail.gmail.com>
Message-ID: <62ede39f4252967687585fb1e1e2177f@preisshare.net>



On Tue, 5 May 2009 01:58:57 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> That is what entry points are providing : the ability to mark a code
> locate anywhere in your installation
> and to load it when needed in your execution context.

ok - but don't we already have this in site.py ? inside the
interpreter. 

>From my reading, it already does just that every time the
python interpreter starts up.

David





From ziade.tarek at gmail.com  Tue May  5 02:11:36 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 02:11:36 +0200
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <62ede39f4252967687585fb1e1e2177f@preisshare.net>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
	<154a4c839797fdf9eaa60501871a90e3@preisshare.net>
	<94bdd2610905041658w33a2b8ddq78553583a25b1a5d@mail.gmail.com>
	<62ede39f4252967687585fb1e1e2177f@preisshare.net>
Message-ID: <94bdd2610905041711l4e80a6baifd25c6f1e425018e@mail.gmail.com>

On Tue, May 5, 2009 at 2:00 AM, David Lyon <david.lyon at preisshare.net> wrote:
>
>
> On Tue, 5 May 2009 01:58:57 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>
>> That is what entry points are providing : the ability to mark a code
>> locate anywhere in your installation
>> and to load it when needed in your execution context.
>
> ok - but don't we already have this in site.py ? inside the
> interpreter.
>
> From my reading, it already does just that every time the
> python interpreter starts up.

I am not sure to understand what you are explaining - when the python
interpreter starts up, it doesn't load every installed package
in memory. the loading happens when you do "import foo"

But entry points provides a way to tell you for example that the "bar"
function located in the "foo" module is a plugin,
*without having to load this module in your interpreter*, because this
info is writtent in a static text file located in the .egg-info
 directory. After, you will eventually load it (by using a load()
function in the entry point, that is basically an import statement)

Tarek

>
> David
>
>
>
>
>



-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Tue May  5 02:32:43 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 04 May 2009 20:32:43 -0400
Subject: [Distutils] RFC : PEP 376 - egg.info
In-Reply-To: <94bdd2610905041711l4e80a6baifd25c6f1e425018e@mail.gmail.com>
References: <94bdd2610904300034s738bea14h7250279e66ee0553@mail.gmail.com>
	<94bdd2610905030303h2258f106o249af902c6cbc27b@mail.gmail.com>
	<20090503171236.7D21D3A4080@sparrow.telecommunity.com>
	<94bdd2610905041637s4bf24e7ya8a9e2a336828095@mail.gmail.com>
	<154a4c839797fdf9eaa60501871a90e3@preisshare.net>
	<94bdd2610905041658w33a2b8ddq78553583a25b1a5d@mail.gmail.com>
	<62ede39f4252967687585fb1e1e2177f@preisshare.net>
	<94bdd2610905041711l4e80a6baifd25c6f1e425018e@mail.gmail.com>
Message-ID: <16733739b9ccd273c1d512ba80c482c8@preisshare.net>



On Tue, 5 May 2009 02:11:36 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I am not sure to understand what you are explaining - when the python
> interpreter starts up, it doesn't load every installed package
> in memory. the loading happens when you do "import foo"

Correct. I didn't say it loads them all. 

The code however seems to load up all the names of the packages
from packages along the python-path. It then executes any
line starting with 'import'.

--- http://svn.python.org/projects/python/trunk/Lib/site.py -----
def addpackage(sitedir, name, known_paths):
    """Process a .pth file within the site-packages directory:
       For each line in the file, either combine it with sitedir to a path
       and add that to known_paths, or execute it if it starts with 'import
'.
    """
    if known_paths is None:
        _init_pathinfo()
        reset = 1
    else:
        reset = 0
    fullname = os.path.join(sitedir, name)
    try:
        f = open(fullname, "rU")
    except IOError:
        return
    with f:
        for line in f:
            if line.startswith("#"):
                continue
            if line.startswith(("import ", "import\t")):
                exec line
                continue
            line = line.rstrip()
            dir, dircase = makepath(sitedir, line)
            if not dircase in known_paths and os.path.exists(dir):
                sys.path.append(dir)
                known_paths.add(dircase)
    if reset:
        known_paths = None
    return known_paths

-------------------------------------------------------------------------

> *without having to load this module in your interpreter*, because this
> info is writtent in a static text file located in the .egg-info
>  directory. After, you will eventually load it (by using a load()
> function in the entry point, that is basically an import statement)

Yes .. that perfectly describes existing functionality.

> But entry points provides a way to tell you for example that the "bar"
> function located in the "foo" module is a plugin,

I see now what you are getting at...

But packages along the site-packages directory are already a form of
a plug-in system. It just doesn't have the modern name.

The code is already implemented as shown above.. it all works "today"..

What really is missing is the sophisticated API to properly manage
it all. There's no way to find out what's installed... remove anything..

The whole system is very brittle...

I think it's possible to come up with something "new" by just addressing
many of the major deficiencies that already exist in distutils. 

For me personally - I'm a big admirer of the existing code..

problem to me seems to be there's just not enough of it..

David








From pje at telecommunity.com  Tue May  5 03:02:23 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 04 May 2009 21:02:23 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.co
 m>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
Message-ID: <20090505005955.7C6ED3A40A5@sparrow.telecommunity.com>

At 06:57 PM 5/4/2009 -0500, Ian Bicking wrote:
>* I'm uncomfortable with the way entry points are scanned.?  I 
>haven't looked close enough to back it up with numbers, but I think 
>there's a noticeable performance degradation when the number of 
>installed packages becomes large.?  (Given the algorithm this would 
>be expected.)

It's linear in the number of entry_points.txt files, yes, but in most 
apps it should occur at most once, since pkg_resources has a single 
WorkingSet object holding Distribution objects which cache their 
entry point data upon first access.

There are all sorts of ways you could make different tradeoffs, but 
this particular set of tradeoffs was optimized for a 
single-application environment, rather than a massive global shared 
site-packages where there are plugins for every application on the 
system.  It was also optimized for the zipimport case, because you 
can tell whether a project has entry points from its cached zip 
directory, that's needed at startup anyhow.


From ben+python at benfinney.id.au  Tue May  5 03:03:11 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 05 May 2009 11:03:11 +1000
Subject: [Distutils] Adding entry points into Distutils ?
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
Message-ID: <87my9sicog.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> There are *many* benefits of adding entry points into Distutils. In
> fact, adding a plugin system in there, will help make the commands
> more extendable and therefore will help us in the long term to remove
> things out of Distutils.

I don't see what those advantages are. From what I can see, commands
still need to be implemented in separate programs in order to be used as
commands.

> So any strong opinion against them ?

Until I understand what they do, I don't know what objections I might
have :-)

-- 
 \                            ?Only the educated are free.? ?Epictetus |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From yanegomi at gmail.com  Tue May  5 05:53:50 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Mon, 4 May 2009 20:53:50 -0700
Subject: [Distutils] Anyone stuck with easy_install / has .pth file
	issues -- this is for you!
In-Reply-To: <20090504204338.115EF3A40A5@sparrow.telecommunity.com>
References: <364299f40905040243p37aa52aj1560b27bf1d04f56@mail.gmail.com>
	<20090504151737.93AD33A40A5@sparrow.telecommunity.com>
	<364299f40905041311m10f584e6i477120fa64d9edf@mail.gmail.com>
	<20090504204338.115EF3A40A5@sparrow.telecommunity.com>
Message-ID: <364299f40905042053v1a6b7524td33e52838af6ebfc@mail.gmail.com>

On Mon, May 4, 2009 at 1:46 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 01:11 PM 5/4/2009 -0700, Garrett Cooper wrote:
>>
>> You're right -- it doesn't protect against the following (><):
>>
>> /full/path/to/package.egg
>> ./package.egg
>
> By duplicates, I meant 'package-1.0.egg' and 'package-1.1.egg', not
> alternate paths to the same file.
>
> (As for the '.' replacement, you might want to look at what I said more
> closely.)

Yeah, I realize where I screwed up now. Thanks :).

>> BTW, wasn't -m deprecated?
>
> No.

Ah, you're right. Maybe that was another option...

>> ?I mean, -m's `delivered' functionality
>> doesn't even work
>
> You'll need to be more specific there: I don't know what you're saying
> "doesn't work", let alone *how*, what behavior you expect, or what you're
> seeing instead.

Well, if egg 1 and egg 2 define the same module, then the 1st egg's
copy of the module will be loaded and not the second, regardless of
the version criteria defined in the .egg, because sys.modules uses a
dictionary for storage:

[gcooper at orangebox /usr/home/gcooper]$ python -c 'import sys; print
sys.modules'
{'copy_reg': <module 'copy_reg' from
'/usr/local/lib/python2.5/copy_reg.pyc'>, '__main__': <module
'__main__' (built-in)>, 'site': <module 'site' from
'/usr/local/lib/python2.5/site.pyc'>, '__builtin__': <module
'__builtin__' (built-in)>, 'encodings': <module 'encodings' from
'/usr/local/lib/python2.5/encodings/__init__.pyc'>,
'encodings.encodings': None, 'posixpath': <module 'posixpath' from
'/usr/local/lib/python2.5/posixpath.pyc'>, 'errno': <module 'errno'
(built-in)>, 'encodings.codecs': None, 'os.path': <module 'posixpath'
from '/usr/local/lib/python2.5/posixpath.pyc'>, '_codecs': <module
'_codecs' (built-in)>, 'stat': <module 'stat' from
'/usr/local/lib/python2.5/stat.pyc'>, 'zipimport': <module 'zipimport'
(built-in)>, 'warnings': <module 'warnings' from
'/usr/local/lib/python2.5/warnings.pyc'>, 'encodings.types': None,
'UserDict': <module 'UserDict' from
'/usr/local/lib/python2.5/UserDict.pyc'>, 'encodings.ascii': <module
'encodings.ascii' from
'/usr/local/lib/python2.5/encodings/ascii.pyc'>, 'sys': <module 'sys'
(built-in)>, 'codecs': <module 'codecs' from
'/usr/local/lib/python2.5/codecs.pyc'>, 'types': <module 'types' from
'/usr/local/lib/python2.5/types.pyc'>, '_types': <module '_types'
(built-in)>, 'signal': <module 'signal' (built-in)>, 'linecache':
<module 'linecache' from '/usr/local/lib/python2.5/linecache.pyc'>,
'posix': <module 'posix' (built-in)>, 'encodings.aliases': <module
'encodings.aliases' from
'/usr/local/lib/python2.5/encodings/aliases.pyc'>, 'exceptions':
<module 'exceptions' (built-in)>, 'os': <module 'os' from
'/usr/local/lib/python2.5/os.pyc'>}

>> because python stores all loaded package / module
>> entries in .pth files as keys in a dictionary _anyhow_
>
> I also don't know what you mean here. ?.pth file contents are added to
> sys.path, not a dictionary. ?Unless you're talking about
> sys.path_importer_cache or something, in which case I still don't see the
> connection to anything else at hand.

It turned into a bikeshed item I suppose, but that was what I was
stating about the -m option... meh.

>> I have no idea how -xN factor into this though
>
> The original docs said to use -m before uninstalling, but they really should
> say -mxN to exclude scripts and avoid installing (or uninstalling!)
> dependencies. ?The docs are broken (or at any rate, suboptimal) and should
> be fixed.

Ah, ok. Cool -- thanks :).

>> - The duplicates erasure was just gravy, and purely an exercise on my
>> part getting used to set's in python. The real point was the missing
>> package in .pth functionality, caused by easy_install / setuptools
>> lacking proper MP-safe logic.
>
> I'm going to guess that by "MP" you mean multiprocessing, in which case I
> would say, yes, easy_install assumes that you are not running multiple
> simultaneous installations to the same directory.

Yesh... that's the problem -- but why should I have to run
easy_install twice :\?

Thanks for the feedback!
-Garrett

From ziade.tarek at gmail.com  Tue May  5 10:49:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 10:49:35 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
Message-ID: <94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>

On Tue, May 5, 2009 at 1:57 AM, Ian Bicking <ianb at colorstudy.com> wrote:
> Not strong, but I have a few issues with how they are currently defined:
>
> * There's the issue of activated and unactivated eggs, of course, but I
> guess that will be moot since there's no activation with just distutils?

Yes

> * There's no idea of explicitly enabling an entry point, simply installing a
> package makes the entry point show up.? Implicit plugins make me
> uncomfortable.

I don't see entry points as plugins, but rather the registering of a
given piece of code,
under a unique name.

If you add explicit enabling, who will do it ? the package that has
the entry point ?
The applications that consumes them ?

The way I see entry points is "potential" plugins, an application can
decide to consume,
and turn into a real plugin when it uses it.

And an entry point that would be "disabled" is an entry point that is not used
from the application A point of view, but might be used in the application B.

(unless I misunderstood the concept of "group")

So enabling/disabling an entry point and keeping track of the
activation state should be done
by the host application ihmo.



> * There's not much in the way of entry point documentation built into the
> system.? The __doc__ of the entry point objects is one way, but there's no
> way to document entry point groups.

very good point,


-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 10:55:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 10:55:32 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090505005955.7C6ED3A40A5@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<20090505005955.7C6ED3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905050155u668fb1fav82262f46db73c782@mail.gmail.com>

2009/5/5 P.J. Eby <pje at telecommunity.com>:
> At 06:57 PM 5/4/2009 -0500, Ian Bicking wrote:
>>
>> * I'm uncomfortable with the way entry points are scanned.? ?I haven't
>> looked close enough to back it up with numbers, but I think there's a
>> noticeable performance degradation when the number of installed packages
>> becomes large.? ?(Given the algorithm this would be expected.)
>
> It's linear in the number of entry_points.txt files, yes, but in most apps
> it should occur at most once, since pkg_resources has a single WorkingSet
> object holding Distribution objects which cache their entry point data upon
> first access.
>
> There are all sorts of ways you could make different tradeoffs, but this
> particular set of tradeoffs was optimized for a single-application
> environment, rather than a massive global shared site-packages where there
> are plugins for every application on the system. ?It was also optimized for
> the zipimport case, because you can tell whether a project has entry points
> from its cached zip directory, that's needed at startup anyhow.
>

Would it make sense then to maintain an global index of all entry
points, that would be updated
upon installation / uninstallation (if it's added like we wrote in PEP 376)
rather than scanning the paths everytime ?

we could have one index file per site-package-like directory, and
discover index files
rather than all directories / zip files.

Extra paths added in sys.path would be omited but I don't see it as a problem



-- 
Tarek Ziad? | http://ziade.org

From noah.gift at gmail.com  Tue May  5 11:20:13 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Tue, 5 May 2009 21:20:13 +1200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050155u668fb1fav82262f46db73c782@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<20090505005955.7C6ED3A40A5@sparrow.telecommunity.com>
	<94bdd2610905050155u668fb1fav82262f46db73c782@mail.gmail.com>
Message-ID: <e91cc0270905050220v2ce55151lc2b4814db3d99651@mail.gmail.com>

On Tue, May 5, 2009 at 8:55 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> 2009/5/5 P.J. Eby <pje at telecommunity.com>:
>> At 06:57 PM 5/4/2009 -0500, Ian Bicking wrote:
>>>
>>> * I'm uncomfortable with the way entry points are scanned.? ?I haven't
>>> looked close enough to back it up with numbers, but I think there's a
>>> noticeable performance degradation when the number of installed packages
>>> becomes large.? ?(Given the algorithm this would be expected.)
>>
>> It's linear in the number of entry_points.txt files, yes, but in most apps
>> it should occur at most once, since pkg_resources has a single WorkingSet
>> object holding Distribution objects which cache their entry point data upon
>> first access.
>>
>> There are all sorts of ways you could make different tradeoffs, but this
>> particular set of tradeoffs was optimized for a single-application
>> environment, rather than a massive global shared site-packages where there
>> are plugins for every application on the system. ?It was also optimized for
>> the zipimport case, because you can tell whether a project has entry points
>> from its cached zip directory, that's needed at startup anyhow.
>>
>
> Would it make sense then to maintain an global index of all entry
> points, that would be updated
> upon installation / uninstallation (if it's added like we wrote in PEP 376)
> rather than scanning the paths everytime ?

I think so.  In an environment like the one I work in, a massive NFS
infrastructure with thousands of hosts, scanning upon startup is
painful.  I think writing the package list out to a file is the way to
go, and then scanning that file upon startup.  While you are at it, I
think someone also needs to look at some of things that are happening
up startup like looking for .pyc .so etc.  Unfortunately, file servers
are stupid and penalize you each time you stat a file that isn't
there.  I would like an expert mode that allowed me control exactly
what was looked at when the Python interpreter launched.  I blogged
about it here:

http://artificialcode.blogspot.com/2009/04/short-circuiting-python-module-lookup.html

If you run the python with strace you will see some interesting
things.  This is something that I feel is in dire need of optimization
and makes Python look pretty bad...

>
> we could have one index file per site-package-like directory, and
> discover index files
> rather than all directories / zip files.

I again like the expert mode idea, in which you could literally say,
"dammit, this is where stuff is and you aren't looking anywhere
else...I really mean it...I know what I am doing..".
>
> Extra paths added in sys.path would be omited but I don't see it as a problem
>
>
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Cheers,

Noah

From doug.hellmann at gmail.com  Tue May  5 13:57:22 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Tue, 5 May 2009 07:57:22 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
Message-ID: <A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>


On May 5, 2009, at 4:49 AM, Tarek Ziad? wrote:

> On Tue, May 5, 2009 at 1:57 AM, Ian Bicking <ianb at colorstudy.com>  
> wrote:
>> Not strong, but I have a few issues with how they are currently  
>> defined:
>>
>> * There's the issue of activated and unactivated eggs, of course,  
>> but I
>> guess that will be moot since there's no activation with just  
>> distutils?
>
> Yes
>
>> * There's no idea of explicitly enabling an entry point, simply  
>> installing a
>> package makes the entry point show up.  Implicit plugins make me
>> uncomfortable.
>
> I don't see entry points as plugins, but rather the registering of a
> given piece of code,
> under a unique name.

I don't understand that.  I thought the purpose of entry points was to  
register code such as plugins so that applications didn't have to be  
manually configured.  I think I'm with Ian on that one: Explicit is  
better than implicit.  If I have to "turn on" the plugin, then what  
benefit does an entry point registry give me?  I could just as easily  
provide that information to the application directly.

> If you add explicit enabling, who will do it ? the package that has
> the entry point ?
> The applications that consumes them ?

The user who wants the application to consume the plugin.

> The way I see entry points is "potential" plugins, an application can
> decide to consume,
> and turn into a real plugin when it uses it.
>
>
> And an entry point that would be "disabled" is an entry point that  
> is not used
> from the application A point of view, but might be used in the  
> application B.

But if it's not being used by A, why should A see it at all?

Doug


From ziade.tarek at gmail.com  Tue May  5 14:15:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 14:15:40 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
Message-ID: <94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>

On Tue, May 5, 2009 at 1:57 PM, Doug Hellmann <doug.hellmann at gmail.com> wrote:

> ?If I have to "turn on" the plugin, then what benefit does an entry point registry give me?

I don't understand this sentence, since you say later that you want the user to
manually turn a plugin "on" for an application to soncume it.

>> If you add explicit enabling, who will do it ? the package that has
>> the entry point ?
>> The applications that consumes them ?
>
> The user who wants the application to consume the plugin.
>

I am confused with the role of this "man in the middle". In my mind
there are plugins on one side,
and host applications that consumes them if they wish on they other side.

Do you have a use case we can share for mutual comprehension ?


>>
>> And an entry point that would be "disabled" is an entry point that is not
>> used
>> from the application A point of view, but might be used in the application
>> B.
>
> But if it's not being used by A, why should A see it at all?

A, B or any app can browse all entry points. Entry points are defined
by the (group, name)
couple.

Basically if I want to create a pluggable feature called "myfeature",
the group will be
"myfeature" and plugins that implement that feature will register
themselves under that group.

Then my application will browse and consume entry points for the group
"myfeature" and do
whatever they want with them.

In pseudo code:

   >>> plugins = iter_entry_points(group="myfeature")

So if A doesn't need the plugins that are under the group "myfeature",
it will just ignore the entry points
that are in this group. e.g. ignore the group.

Maybe A will consume entry_points that are under another group. But I
have never browsed *all* entry points
from an application.

I think the best practice for entry points is to use the most explicit
group names possible, but having plugins
that can be consumed by several applications is a win ihmo.

For instance, if I need to write a specific extensible installation
script, I'll probably see if I can consume
zc.buildout recipes through the "zc.buildout.recipe" group.


>
> Doug
>
>



-- 
Tarek Ziad? | http://ziade.org

From doug.hellmann at gmail.com  Tue May  5 14:41:50 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Tue, 5 May 2009 08:41:50 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
Message-ID: <9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>


On May 5, 2009, at 8:15 AM, Tarek Ziad? wrote:

> On Tue, May 5, 2009 at 1:57 PM, Doug Hellmann  
> <doug.hellmann at gmail.com> wrote:
>
>>  If I have to "turn on" the plugin, then what benefit does an entry  
>> point registry give me?
>
> I don't understand this sentence, since you say later that you want  
> the user to
> manually turn a plugin "on" for an application to soncume it.

I want each application to be configured separately, rather than  
having a global registry of plugins.  So having a "plugin registry"  
*library* in stdlib may make sense (so that apps can build their  
registry databases in a consistent way), but automatically registering  
entry points just because a package is installed does not.

>>> If you add explicit enabling, who will do it ? the package that has
>>> the entry point ?
>>> The applications that consumes them ?
>>
>> The user who wants the application to consume the plugin.
>>
>
> I am confused with the role of this "man in the middle". In my mind
> there are plugins on one side,
> and host applications that consumes them if they wish on they other  
> side.
>
> Do you have a use case we can share for mutual comprehension ?

I think we have a different view about how the plugins should work.   
It sounds like you're advocating a model where all plugins are  
registered globally, an application can search the global registry for  
plugins based on categories, and then some administrator enables/ 
disables them locally for each app.

I don't want new functionality available to an application just  
because someone has permission to install a package somewhere in the  
PYTHONPATH.  I would rather have plugins added to an app through an  
explicit configuration step of some sort.

> So if A doesn't need the plugins that are under the group "myfeature",
> it will just ignore the entry points
> that are in this group. e.g. ignore the group.
>
> Maybe A will consume entry_points that are under another group. But I
> have never browsed *all* entry points
> from an application.

That depends on how the database of entry points is maintained, but  
I'll grant that point.

> I think the best practice for entry points is to use the most explicit
> group names possible, but having plugins
> that can be consumed by several applications is a win ihmo.
>
> For instance, if I need to write a specific extensible installation
> script, I'll probably see if I can consume
> zc.buildout recipes through the "zc.buildout.recipe" group.
>
>
>>
>> Doug
>>
>>
>
>
>
> -- 
> Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Tue May  5 15:33:03 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 15:33:03 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
Message-ID: <94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>

On Tue, May 5, 2009 at 2:41 PM, Doug Hellmann <doug.hellmann at gmail.com> wrote:
>> I am confused with the role of this "man in the middle". In my mind
>> there are plugins on one side,
>> and host applications that consumes them if they wish on they other side.
>>
>> Do you have a use case we can share for mutual comprehension ?
>
> I think we have a different view about how the plugins should work. ?It
> sounds like you're advocating a model where all plugins are registered
> globally, an application can search the global registry for plugins based on
> categories, and then some administrator enables/disables them locally for
> each app.
>
> I don't want new functionality available to an application just because
> someone has permission to install a package somewhere in the PYTHONPATH. ?I
> would rather have plugins added to an app through an explicit configuration
> step of some sort.

That is basically how host applications are dealing with entry points:
an explicit configuration. The implicit part is just happening when
you look for the plugins.

Let's take a real example : I am working on a program called Atomisator
which can be extended through plugins.

For example people can provide a plugin to read data that are located
somewhere (like a rss feed), and return
a sequence of entries.

So in atomisator, I have a configuration file where I decide which
plugin to use to read my data:

"""
   [atomisator]

   reader = rss
"""

When reading it, Atomisator knows that it needs the "rss" plugin, and
will browse in entry points
with a specific group called "atomisator.rss". If it finds it, it uses
it, otherwise it throws an exception. ("Install it!")

That allows people to create their own plugins in separate packages,
and use them by tweaking the configuration
file. The only think that entry point provided here is an automatic
registration at installation time of the "rss" plugins,
so my Atomisatior application can discover then load it at run time.

So in your way of seeing thing, you'd rather see this registration
mechanism at the application level,
but the you need to provide specific installation instructions for
people that want to add plugins.
e.g. "put your package in this /plugins directory" for example.

I am advocating that the entry point mechanism is handy because it
relies on an existing mechanism : "install your package"
and it allows plugin sharing amongst applications.

But I see the caveats you are explaining, and I understand them now;

So, what if we didn't have these entry points installed globally when
the package is installed,
but just a configuration file in the host application level to point them ?

a configuration file that reunites all entry points an application
uses. For the Atomisator example:

  [atomisator.reader]
  rss = somepackage.somemodule:MyPluginClass

This would let the application consume the plugins pointed by this
configuration file
and this would remove the implicit part you don't like. I'd be very
happy with such a plugin system on my side.

And this would fit I think in Distutils needs since we can configure
it through three levels of configuration files
distutils.cfg, pydistutils.cfg and setup.cfg



Tarek

-- 
Tarek Ziad? | http://ziade.org

From seb.binet at gmail.com  Tue May  5 15:55:44 2009
From: seb.binet at gmail.com (Sebastien Binet)
Date: Tue, 5 May 2009 15:55:44 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
Message-ID: <200905051555.44757.binet@cern.ch>

hi,

> That allows people to create their own plugins in separate packages,
> and use them by tweaking the configuration
> file. The only think that entry point provided here is an automatic
> registration at installation time of the "rss" plugins,
> so my Atomisatior application can discover then load it at run time.
>
> So in your way of seeing thing, you'd rather see this registration
> mechanism at the application level,
> but the you need to provide specific installation instructions for
> people that want to add plugins.
> e.g. "put your package in this /plugins directory" for example.

wouldn't this be tackled by providing a plugin-discoverer plugin ?
kind of like the new import hooks of pep-302 [1]  where you have module 
finders separated from module loaders, or like this library I have been using 
in my field [2]

cheers,
sebastien.

[1] http://www.python.org/dev/peps/pep-0302/
[2] http://cdsweb.cern.ch/record/865639?ln=pl
      http://wlav.web.cern.ch/wlav/pybus/index.html
      http://seal.cvs.cern.ch:80/cgi-bin/seal.cgi/seal/Scripting/PyBus
-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################


From doug.hellmann at gmail.com  Tue May  5 16:29:20 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Tue, 5 May 2009 10:29:20 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
Message-ID: <90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>


On May 5, 2009, at 9:33 AM, Tarek Ziad? wrote:

> On Tue, May 5, 2009 at 2:41 PM, Doug Hellmann  
> <doug.hellmann at gmail.com> wrote:
>>> I am confused with the role of this "man in the middle". In my mind
>>> there are plugins on one side,
>>> and host applications that consumes them if they wish on they  
>>> other side.
>>>
>>> Do you have a use case we can share for mutual comprehension ?
>>
>> I think we have a different view about how the plugins should  
>> work.  It
>> sounds like you're advocating a model where all plugins are  
>> registered
>> globally, an application can search the global registry for plugins  
>> based on
>> categories, and then some administrator enables/disables them  
>> locally for
>> each app.
>>
>> I don't want new functionality available to an application just  
>> because
>> someone has permission to install a package somewhere in the  
>> PYTHONPATH.  I
>> would rather have plugins added to an app through an explicit  
>> configuration
>> step of some sort.
>
> That is basically how host applications are dealing with entry points:
> an explicit configuration. The implicit part is just happening when
> you look for the plugins.
>
> Let's take a real example : I am working on a program called  
> Atomisator
> which can be extended through plugins.
>
> For example people can provide a plugin to read data that are located
> somewhere (like a rss feed), and return
> a sequence of entries.
>
> So in atomisator, I have a configuration file where I decide which
> plugin to use to read my data:
>
> """
>   [atomisator]
>
>   reader = rss
> """
>
> When reading it, Atomisator knows that it needs the "rss" plugin, and
> will browse in entry points
> with a specific group called "atomisator.rss". If it finds it, it uses
> it, otherwise it throws an exception. ("Install it!")

> That allows people to create their own plugins in separate packages,
> and use them by tweaking the configuration
> file. The only think that entry point provided here is an automatic
> registration at installation time of the "rss" plugins,
> so my Atomisatior application can discover then load it at run time.

I don't think I understand the difference between the step you're  
calling "discover", scanning the registry, and actually loading the  
plugin.  Does "discovering" the plugin involve importing any of its  
code?

> So in your way of seeing thing, you'd rather see this registration
> mechanism at the application level,
> but the you need to provide specific installation instructions for
> people that want to add plugins.
> e.g. "put your package in this /plugins directory" for example.

Sort of.  I see the installation and configuration of entry points as  
2 steps:

1. some variation of "python setup.py install"
2. update the configuration of app to tell it where to find the plugin

Some applications may provide automated tools for step 2, and I could  
see great benefit to making the loading and registration of entry  
points part of the standard library, as long as the registration is  
maintained per-application instead of site-wide.

> I am advocating that the entry point mechanism is handy because it
> relies on an existing mechanism : "install your package"
> and it allows plugin sharing amongst applications.

How then do I install a package but *not* have it available for an  
application?  Suppose, for example, that I have several copies of a  
web app installed for different customers, but I want to prevent some  
of them from using an advanced plugin of some sort.

> But I see the caveats you are explaining, and I understand them now;
>
> So, what if we didn't have these entry points installed globally when
> the package is installed,
> but just a configuration file in the host application level to point  
> them ?
>
> a configuration file that reunites all entry points an application
> uses. For the Atomisator example:
>
>  [atomisator.reader]
>  rss = somepackage.somemodule:MyPluginClass

Yes!  We can figure out the exact spelling, but we're talking about  
the same thing now.  If we use dotted notation all the way  
("somepackage.somemodule.MyPluginClass") then it's simple to just  
import the thing directly.

> This would let the application consume the plugins pointed by this
> configuration file
> and this would remove the implicit part you don't like. I'd be very
> happy with such a plugin system on my side.

If you follow the model of the logging module and provide an easy  
"load plugins from an ini file", then all python apps have the feature  
of making it easy to load plugins, developers have a standard API to  
code to, and administrators don't have to worry about plugins  
"sneaking" into an app.

> And this would fit I think in Distutils needs since we can configure
> it through three levels of configuration files
> distutils.cfg, pydistutils.cfg and setup.cfg

That sounds good.

Doug


From pje at telecommunity.com  Tue May  5 16:38:50 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 May 2009 10:38:50 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
Message-ID: <20090505143615.5B6F13A4080@sparrow.telecommunity.com>

At 08:41 AM 5/5/2009 -0400, Doug Hellmann wrote:
>I don't want new functionality available to an application just
>because someone has permission to install a package somewhere in the
>PYTHONPATH.  I would rather have plugins added to an app through an
>explicit configuration step of some sort.

Note that this is not incompatible with entry points; an application 
can simply treat entry points as a list of *available* plugins, 
rather than as a list of *active* plugins.  Chandler, for example, 
does this for user-level plugins. 


From doug.hellmann at gmail.com  Tue May  5 16:40:51 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Tue, 5 May 2009 10:40:51 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090505143615.5B6F13A4080@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<20090505143615.5B6F13A4080@sparrow.telecommunity.com>
Message-ID: <489A2A03-873B-4022-903F-145746B6A983@gmail.com>


On May 5, 2009, at 10:38 AM, P.J. Eby wrote:

> At 08:41 AM 5/5/2009 -0400, Doug Hellmann wrote:
>> I don't want new functionality available to an application just
>> because someone has permission to install a package somewhere in the
>> PYTHONPATH.  I would rather have plugins added to an app through an
>> explicit configuration step of some sort.
>
> Note that this is not incompatible with entry points; an application  
> can simply treat entry points as a list of *available* plugins,  
> rather than as a list of *active* plugins.  Chandler, for example,  
> does this for user-level plugins.

That's good.  I wasn't sure if registering an entry point caused it to  
be automatically loaded in an app that expressed interest in it, or if  
there was a step the app could take to verify that it was desirable  
code before doing any imports.  Earlier messages in this thread made  
me think there was no "protection" from a registered plugin.


From ziade.tarek at gmail.com  Tue May  5 17:05:28 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 17:05:28 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
Message-ID: <94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>

On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann <doug.hellmann at gmail.com> wrote
>
> I don't think I understand the difference between the step you're calling
> "discover", scanning the registry, and actually loading the plugin. ?Does
> "discovering" the plugin involve importing any of its code?

No, like Phillip said somewhere in the thread.

The discovery pseudo-code is:

entry_points = []

for path in paths:
  if path is egg-info:
      entry_points.append(load('entry_points.txt'))

Then each entry point is just a string that "locates" the plugins
("package.modul.class"  for example)

The real import occurs only explicitely when you do "entry_point.load()"


>> a configuration file that reunites all entry points an application
>> uses. For the Atomisator example:
>>
>> ?[atomisator.reader]
>> ?rss = somepackage.somemodule:MyPluginClass
>
> Yes! ?We can figure out the exact spelling, but we're talking about the same
> thing now. ?If we use dotted notation all the way
> ("somepackage.somemodule.MyPluginClass") then it's simple to just import the
> thing directly.

I think is simpler with the "somepackage.somemodule:MyPluginClass" notation

This is how setuptools does roughly:

   >>> parts = "somepackage.somemodule:MyPluginClass".split(':')
   >>> module = __import__(parts[0])
   >>> plugin = getattr(module, parts[1])

>
>> And this would fit I think in Distutils needs since we can configure
>> it through three levels of configuration files
>> distutils.cfg, pydistutils.cfg and setup.cfg
>
> That sounds good.

The only caveat I see though, is that the host app has to know the
exact location
of each plugin in the code of the third party app, whereas entry points provide
this information through the discovery API.

I could leave with both notation I believe:

location = "somepackage.somemodule:MyPluginClass"

*or*

location = iter_entry_points('atomisator.reader', 'rss')



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May  5 17:09:39 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 5 May 2009 17:09:39 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <200905051555.44757.binet@cern.ch>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<200905051555.44757.binet@cern.ch>
Message-ID: <94bdd2610905050809r20147dcdw2509fd50211a9c10@mail.gmail.com>

2009/5/5 Sebastien Binet <seb.binet at gmail.com>:
> hi,
>
>> That allows people to create their own plugins in separate packages,
>> and use them by tweaking the configuration
>> file. The only think that entry point provided here is an automatic
>> registration at installation time of the "rss" plugins,
>> so my Atomisatior application can discover then load it at run time.
>>
>> So in your way of seeing thing, you'd rather see this registration
>> mechanism at the application level,
>> but the you need to provide specific installation instructions for
>> people that want to add plugins.
>> e.g. "put your package in this /plugins directory" for example.
>
> wouldn't this be tackled by providing a plugin-discoverer plugin ?
> kind of like the new import hooks of pep-302 [1] ?where you have module
> finders separated from module loaders, or like this library I have been using
> in my field [2]

That would be better to use these hooks rather than __import__ I believe,
but I don't think it changes the problem (eg locate the plugin)

From doug.hellmann at gmail.com  Tue May  5 18:37:09 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Tue, 5 May 2009 12:37:09 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
Message-ID: <C19AA834-DB10-418A-B271-132F227C0A17@gmail.com>


On May 5, 2009, at 11:05 AM, Tarek Ziad? wrote:

> On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann  
> <doug.hellmann at gmail.com> wrote
>>
>>> a configuration file that reunites all entry points an application
>>> uses. For the Atomisator example:
>>>
>>>  [atomisator.reader]
>>>  rss = somepackage.somemodule:MyPluginClass
>>
>> Yes!  We can figure out the exact spelling, but we're talking about  
>> the same
>> thing now.  If we use dotted notation all the way
>> ("somepackage.somemodule.MyPluginClass") then it's simple to just  
>> import the
>> thing directly.
>
> I think is simpler with the "somepackage.somemodule:MyPluginClass"  
> notation

Good point, I was remembering the import syntax incorrectly.

>>> And this would fit I think in Distutils needs since we can configure
>>> it through three levels of configuration files
>>> distutils.cfg, pydistutils.cfg and setup.cfg
>>
>> That sounds good.
>
> The only caveat I see though, is that the host app has to know the
> exact location
> of each plugin in the code of the third party app, whereas entry  
> points provide
> this information through the discovery API.

True.  On the other hand, that encourages standard locations like  
"mylib.yourapp.entry_point" and "mylib.anotherapp.entry_point".

I think most of my concerns about the global registry are taken care  
of by the fact that "discovering" a plugin doesn't involve any imports  
of unknown code.  I still prefer per-app, explicit configuration of  
entry points, and think we could build a system to support that.  I  
would like to see *some* variation of this in the standard library,  
though, because I have several uses for it.

Doug


From floris.bruynooghe at gmail.com  Tue May  5 22:27:49 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 5 May 2009 21:27:49 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
Message-ID: <20090505202749.GA25343@laurie.devork>

On Tue, May 05, 2009 at 05:05:28PM +0200, Tarek Ziad? wrote:
> On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann <doug.hellmann at gmail.com> wrote
> >> a configuration file that reunites all entry points an application
> >> uses. For the Atomisator example:
> >>
> >> ?[atomisator.reader]
> >> ?rss = somepackage.somemodule:MyPluginClass
[...]
> >> And this would fit I think in Distutils needs since we can configure
> >> it through three levels of configuration files
> >> distutils.cfg, pydistutils.cfg and setup.cfg
> >
> > That sounds good.

So there would be a configuration file for each application that needs
it?  I like this a lot more then the global entry-point registry too
(it avoids name collisions for entry points too).

But how can a "python setup.py install" know where to find this
configuration file to add it's plugin?  Or should this be an explicit
manual step?  It might be nice to have a --register-plugins option to
the install command though if possible.  Something else to keep into
account is the FHS, I can imagine GNU/Linux distributions would want
to place a configuration file somewhere else, like in
/etc/PROJECT.conf instead of
/usr/lib/pythonX.Y/site-packages/PROJECT.egg-info/plugin_registry
(pathnames are fairly random examples).  The only thing I can think of
is somehow having a file in .egg-info telling you where the plugin
configuration file is, just like was proposed for all types of data
files earlier on this list.  But I think by now I'm taking this too
far and the install command of distutils should not be able to
register plugins for random projects automatically.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Tue May  5 23:34:00 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 May 2009 17:34:00 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090505202749.GA25343@laurie.devork>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<20090505202749.GA25343@laurie.devork>
Message-ID: <20090505213124.0E5933A4080@sparrow.telecommunity.com>

At 09:27 PM 5/5/2009 +0100, Floris Bruynooghe wrote:
>But how can a "python setup.py install" know where to find this
>configuration file to add it's plugin?

It doesn't.  The whole point of having two stages -- discovery and 
activation -- is that discovery is an automatic side-effect of 
installation, whereas activation can be controlled by an 
application-specific policy or configuration file.

Entry points (as currently implemented) are discovery; the 
application then determines which entry points to actually load or 
invoke, using its own configuration (or command-line options, or menu 
selections, or automatic policy, or whatever else).


From ianb at colorstudy.com  Tue May  5 23:35:19 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 May 2009 16:35:19 -0500
Subject: [Distutils] Setuptools, namespace packages,
	--single-version-externally-managed
In-Reply-To: <b654cd2e0905011507j3cec56e0n589418bc866aaf82@mail.gmail.com>
References: <b654cd2e0905011507j3cec56e0n589418bc866aaf82@mail.gmail.com>
Message-ID: <b654cd2e0905051435q7e2ceceah756e8ef289ef3c25@mail.gmail.com>

Any ideas on this?  Phillip?

On Fri, May 1, 2009 at 5:07 PM, Ian Bicking <ianb at colorstudy.com> wrote:

> So, a bit of a problem came up with pip and namespace packages.  Here's my
> understanding of what's going on:
>
> When you install a namespace package with pip, it uses install
> --single-version-externally-managed, and generally the namespace directory
> is empty and there's a *.nspkg.pth file that has this:
>
> import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
> *('<NAMESPACE>',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m =
> not ie and sys.modules.setdefault('zope',new.module('zope')); mp = (m or [])
> and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)
>
> So the lack of an __init__.py file doesn't really matter, because it's
> created right there, and has its __path__ added to.  But there's a problem
> when there's another namespace package elsewhere on the path, that wasn't
> installed with pip (or Setuptools) and uses pkgutils.extend_path(__path__,
> __name__).  This doesn't get imported because of that .pth file, and the
> .pth file doesn't itself use extend_path, so the path isn't searched.  This
> is currently happening with Zope packages installed with plain distutils,
> then another package installed with the zope namespace elsewhere with pip.
> (When using easy_install, I think pkg_resource.declare_namespace comes into
> play somewhere, and this seems to handle this case, but I'm not sure why the
> installation is different with pip.)
>
> So... what should pip be doing differently to make this work?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090505/786c9bb0/attachment.htm>

From pje at telecommunity.com  Wed May  6 02:03:23 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 May 2009 20:03:23 -0400
Subject: [Distutils] Setuptools, namespace packages,
 --single-version-externally-managed
In-Reply-To: <b654cd2e0905051435q7e2ceceah756e8ef289ef3c25@mail.gmail.co
 m>
References: <b654cd2e0905011507j3cec56e0n589418bc866aaf82@mail.gmail.com>
	<b654cd2e0905051435q7e2ceceah756e8ef289ef3c25@mail.gmail.com>
Message-ID: <20090506000047.7E8FF3A4080@sparrow.telecommunity.com>

At 04:35 PM 5/5/2009 -0500, Ian Bicking wrote:
>Any ideas on this??  Phillip?
>
>On Fri, May 1, 2009 at 5:07 PM, Ian Bicking 
><<mailto:ianb at colorstudy.com>ianb at colorstudy.com> wrote:
>So, a bit of a problem came up with pip and namespace 
>packages.?  Here's my understanding of what's going on:
>
>When you install a namespace package with pip, it uses install 
>--single-version-externally-managed, and generally the namespace 
>directory is empty and there's a *.nspkg.pth file that has this:
>
>import sys,new,os; p = 
>os.path.join(sys._getframe(1).f_locals['sitedir'], 
>*('<NAMESPACE>',)); ie = 
>os.path.exists(os.path.join(p,'__init__.py')); m = not ie and 
>sys.modules.setdefault('zope',new.module('zope')); mp = (m or []) 
>and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)
>
>So the lack of an __init__.py file doesn't really matter, because 
>it's created right there, and has its __path__ added to.?  But 
>there's a problem when there's another namespace package elsewhere 
>on the path, that wasn't installed with pip (or Setuptools) and uses 
>pkgutils.extend_path(__path__, __name__).?  This doesn't get 
>imported because of that .pth file, and the .pth file doesn't itself 
>use extend_path, so the path isn't searched.?  This is currently 
>happening with Zope packages installed with plain distutils, then 
>another package installed with the zope namespace elsewhere with 
>pip.?  (When using easy_install, I think 
>pkg_resource.declare_namespace comes into play somewhere, and this 
>seems to handle this case, but I'm not sure why the installation is 
>different with pip.)
>
>So... what should pip be doing differently to make this work?

Heck if I know.  IIUC, this should work as long as the namespace 
declaration gets invoked at some point.

But I don't know of any way to ensure that, except for pip to write 
its own __init__.py (invoking declare_namespace()) and gets rid of 
the nspkg.pth file.


From ben+python at benfinney.id.au  Wed May  6 04:03:57 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 06 May 2009 12:03:57 +1000
Subject: [Distutils] Adding entry points into Distutils ?
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
Message-ID: <87eiv3htrm.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> I think is simpler with the "somepackage.somemodule:MyPluginClass" notation
> 
> This is how setuptools does roughly:
> 
>    >>> parts = "somepackage.somemodule:MyPluginClass".split(':')
[?]

Using the standard import notation is no more difficult:

    >>> parts = "somepackage.somemodule.MyPluginClass".rsplit('.', 1)
    >>> parts
    ['somepackage.somemodule', 'MyPluginClass']

versus:

    >>> parts = "somepackage.somemodule:MyPluginClass".split(':', 1)
    >>> parts
    ['somepackage.somemodule', 'MyPluginClass']

I don't see any advantage, in the context of this discussion, to having
an additional, incompatible naming for full-path-to-a-class.

-- 
 \           ?The lift is being fixed for the day. During that time we |
  `\            regret that you will be unbearable.? ?hotel, Bucharest |
_o__)                                                                  |
Ben Finney


From pje at telecommunity.com  Wed May  6 04:50:40 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 May 2009 22:50:40 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <87eiv3htrm.fsf@benfinney.id.au>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
Message-ID: <20090506024804.1FFCC3A4080@sparrow.telecommunity.com>

At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>I don't see any advantage, in the context of this discussion, to 
>having an additional, incompatible naming for full-path-to-a-class.

Setuptools doesn't limit an entry point to being a class, function, 
or other top-level name within a module.  It can be a method of a 
class, or an attribute of an attribute.  The ':' removes any 
ambiguity as to which part of the name is the module, and which parts 
are attributes within that module.


From lists at zopyx.com  Wed May  6 09:07:37 2009
From: lists at zopyx.com (Andreas Jung)
Date: Wed, 06 May 2009 09:07:37 +0200
Subject: [Distutils] Setuptools + Subversion 1.6 - new setuptools
	release necessary
In-Reply-To: <49F6FD73.10809@zopyx.com>
References: <49F69462.6050309@zopyx.com> <49F6FBC1.8080001@simplistix.co.uk>
	<49F6FD73.10809@zopyx.com>
Message-ID: <4A013739.6020105@zopyx.com>

On 28.04.09 14:58, Andreas Jung wrote:
> On 28.04.2009 14:51 Uhr, Chris Withers wrote:
>   
>> Andreas Jung wrote:
>>     
>>> it is known that the latest setuptools version produces broken
>>> packages with SVN 1.6 checkouts. Could we get a fixed setuptools
>>> version asap - fixing this issue is essential
>>> (there is some patch floating around).
>>>       

There is obviously a bugreport and patches available for this issue:

http://bugs.python.org/setuptools/issue64

Any chance to see this patch being merged and released together
with a new setuptools version soon?

Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090506/ac9b8f79/attachment.vcf>

From a.cavallo at cavallinux.eu  Wed May  6 12:36:29 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Wed, 6 May 2009 11:36:29 +0100
Subject: [Distutils] logging into distutils
Message-ID: <200905061136.29818.a.cavallo@cavallinux.eu>

Hi,
I'm playing lately with distutils (on the 2.7a branch).

This "sudden" interest has been triggered by the python bugs:

http://bugs.python.org/issue5941
http://bugs.python.org/issue5940

What I've found (IMHO) is the lack of proper logging while stepping through 
all the loops inside the code.

In fact distutils.log (the log module used internally) It is loosely written 
resembling the logging python module but missing the full elegance in the 
api: is there any reason it could not be made very similar to it?

I think this is one of the reason is not more deeply spread into the distutils 
package (a useful tool in diagnostic, IMHO).

Because I'm out of work at the moment (It might not last forever tough) I can 
do it if there is any interest and send a patch set to be reviewed.


Beside this to the people interested I can provide a binary compiled python 
version with the patches hosted on the suse build server system:

http://download.opensuse.org/repositories/home:/cavallo71:/python-opt/

This is totally system transparent so it can be installed and de-installed 
without impact on the host system.

Let me know your toughts please,
Antonio cavallo

From ziade.tarek at gmail.com  Wed May  6 13:06:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 May 2009 13:06:20 +0200
Subject: [Distutils] logging into distutils
In-Reply-To: <200905061136.29818.a.cavallo@cavallinux.eu>
References: <200905061136.29818.a.cavallo@cavallinux.eu>
Message-ID: <94bdd2610905060406x10fdad11k3010185b94b88136@mail.gmail.com>

On Wed, May 6, 2009 at 12:36 PM, A. Cavallo <a.cavallo at cavallinux.eu> wrote:
> Hi,
> I'm playing lately with distutils (on the 2.7a branch).
>
> This "sudden" interest has been triggered by the python bugs:
>
> http://bugs.python.org/issue5941
> http://bugs.python.org/issue5940
>
> What I've found (IMHO) is the lack of proper logging while stepping through
> all the loops inside the code.
>
> In fact distutils.log (the log module used internally) It is loosely written
> resembling the logging python module but missing the full elegance in the
> api: is there any reason it could not be made very similar to it?
>

In fact the goal is to replace it with logging calls. It was created
with this in mind

> I think this is one of the reason is not more deeply spread into the distutils
> package (a useful tool in diagnostic, IMHO).
>
> Because I'm out of work at the moment (It might not last forever tough) I can
> do it if there is any interest and send a patch set to be reviewed.

Sure that would be awesome !

We started this work a while ago, http://bugs.python.org/issue3992
you might want to check the latest patch and take it from here,
the I can review it and commit it

Tarek




-- 
Tarek Ziad? | http://ziade.org

From doug.hellmann at gmail.com  Wed May  6 16:59:23 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Wed, 6 May 2009 10:59:23 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
Message-ID: <FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>


On May 5, 2009, at 10:50 PM, P.J. Eby wrote:

> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>> I don't see any advantage, in the context of this discussion, to  
>> having an additional, incompatible naming for full-path-to-a-class.
>
> Setuptools doesn't limit an entry point to being a class, function,  
> or other top-level name within a module.  It can be a method of a  
> class, or an attribute of an attribute.  The ':' removes any  
> ambiguity as to which part of the name is the module, and which  
> parts are attributes within that module.

Is that level of complexity useful in practice?  I can understand how  
it came to be implemented, but is it actually used by any projects?

Doug



From pje at telecommunity.com  Wed May  6 19:46:44 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 06 May 2009 13:46:44 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
Message-ID: <20090506174408.E0B903A410E@sparrow.telecommunity.com>

At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:

>On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
>
>>At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>>>I don't see any advantage, in the context of this discussion, to
>>>having an additional, incompatible naming for full-path-to-a-class.
>>
>>Setuptools doesn't limit an entry point to being a class, function,
>>or other top-level name within a module.  It can be a method of a
>>class, or an attribute of an attribute.  The ':' removes any
>>ambiguity as to which part of the name is the module, and which
>>parts are attributes within that module.
>
>Is that level of complexity useful in practice?  I can understand how
>it came to be implemented, but is it actually used by any projects?

I use it; I'm not sure who else does.

The particular use case I have (and that's most likely to be shared) 
is that the calling app or framework wants a callable or function, 
but the providing app or library implements that callable as a 
classmethod for convenience.


From doug.hellmann at gmail.com  Wed May  6 20:07:52 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Wed, 6 May 2009 14:07:52 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090506174408.E0B903A410E@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
	<20090506174408.E0B903A410E@sparrow.telecommunity.com>
Message-ID: <5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>


On May 6, 2009, at 1:46 PM, P.J. Eby wrote:

> At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:
>
>> On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
>>
>>> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>>>> I don't see any advantage, in the context of this discussion, to
>>>> having an additional, incompatible naming for full-path-to-a-class.
>>>
>>> Setuptools doesn't limit an entry point to being a class, function,
>>> or other top-level name within a module.  It can be a method of a
>>> class, or an attribute of an attribute.  The ':' removes any
>>> ambiguity as to which part of the name is the module, and which
>>> parts are attributes within that module.
>>
>> Is that level of complexity useful in practice?  I can understand how
>> it came to be implemented, but is it actually used by any projects?
>
> I use it; I'm not sure who else does.
>
> The particular use case I have (and that's most likely to be shared)  
> is that the calling app or framework wants a callable or function,  
> but the providing app or library implements that callable as a  
> classmethod for convenience.

That's pretty much what I expected.  It feels a little messy to have  
plugins exposing "internals" like that but not so much so that I  
propose we don't allow it. The ":" syntax seems like the right way to  
go.


From hannosch at hannosch.eu  Wed May  6 20:28:36 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Wed, 06 May 2009 20:28:36 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>	<87eiv3htrm.fsf@benfinney.id.au>	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>	<20090506174408.E0B903A410E@sparrow.telecommunity.com>
	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
Message-ID: <gtsksk$jgh$1@ger.gmane.org>

Doug Hellmann wrote:
> On May 6, 2009, at 1:46 PM, P.J. Eby wrote:
> 
>> At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:
>>
>>> On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
>>>
>>>> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>>>>> I don't see any advantage, in the context of this discussion, to
>>>>> having an additional, incompatible naming for full-path-to-a-class.
>>>>
>>>> Setuptools doesn't limit an entry point to being a class, function,
>>>> or other top-level name within a module.  It can be a method of a
>>>> class, or an attribute of an attribute.  The ':' removes any
>>>> ambiguity as to which part of the name is the module, and which
>>>> parts are attributes within that module.
>>>
>>> Is that level of complexity useful in practice?  I can understand how
>>> it came to be implemented, but is it actually used by any projects?
>>
>> I use it; I'm not sure who else does.
>>
>> The particular use case I have (and that's most likely to be shared)
>> is that the calling app or framework wants a callable or function, but
>> the providing app or library implements that callable as a classmethod
>> for convenience.
> 
> That's pretty much what I expected.  It feels a little messy to have
> plugins exposing "internals" like that but not so much so that I propose
> we don't allow it. The ":" syntax seems like the right way to go.

I'd be tempted to call this an edge-case. You should be able to expose
the internal detail you'd need via a module scope alias for the
particular case. That seems easier than providing a whole new notion.

Hanno


From tseaver at palladion.com  Wed May  6 20:32:29 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 06 May 2009 14:32:29 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <gtsksk$jgh$1@ger.gmane.org>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>	<87eiv3htrm.fsf@benfinney.id.au>	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>	<20090506174408.E0B903A410E@sparrow.telecommunity.com>	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
	<gtsksk$jgh$1@ger.gmane.org>
Message-ID: <gtsl3t$jq0$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hanno Schlichting wrote:
> Doug Hellmann wrote:
>> On May 6, 2009, at 1:46 PM, P.J. Eby wrote:
>>
>>> At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:
>>>
>>>> On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
>>>>
>>>>> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>>>>>> I don't see any advantage, in the context of this discussion, to
>>>>>> having an additional, incompatible naming for full-path-to-a-class.
>>>>> Setuptools doesn't limit an entry point to being a class, function,
>>>>> or other top-level name within a module.  It can be a method of a
>>>>> class, or an attribute of an attribute.  The ':' removes any
>>>>> ambiguity as to which part of the name is the module, and which
>>>>> parts are attributes within that module.
>>>> Is that level of complexity useful in practice?  I can understand how
>>>> it came to be implemented, but is it actually used by any projects?
>>> I use it; I'm not sure who else does.
>>>
>>> The particular use case I have (and that's most likely to be shared)
>>> is that the calling app or framework wants a callable or function, but
>>> the providing app or library implements that callable as a classmethod
>>> for convenience.
>> That's pretty much what I expected.  It feels a little messy to have
>> plugins exposing "internals" like that but not so much so that I propose
>> we don't allow it. The ":" syntax seems like the right way to go.
> 
> I'd be tempted to call this an edge-case. You should be able to expose
> the internal detail you'd need via a module scope alias for the
> particular case. That seems easier than providing a whole new notion.

I'm actually a big fan of the ':', because it makes explicit the
difference between the "import" and the "named thing", even for
module-scoped names.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKAde9+gerLs4ltQ4RAgzJAJ0RgbDdXFFW/O/mcK3u6BCKOiBW3QCfUXs3
S0lgBewN4w5PqIHBilft29Y=
=G7sv
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Wed May  6 20:58:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 May 2009 20:58:41 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
Message-ID: <94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>

Sorry to post again that mail, but it seems that this is the last remaining
issue for the versionning proposal.

Phillip, could you check my proposal for your development versions of
postreleases
use case ?

thx

On Thu, Apr 23, 2009 at 11:19 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> 2009/4/22 P.J. Eby <pje at telecommunity.com>:
>> I don't see how it can manage, e.g. a development version of a postrelease,
>> with an SVN rev or date stamp on it. ?Such versions might not be found on
>> PyPI or on RPMs, but would be needed in development.
>
> So, instead of having 'dev' and 'post', we would require a third case
> for "pos+dev" version
>
> so ?(dev|post)N+ could become, ((dev|post)N+)|(postN+devN+)
>
> example:
>
> 1.0.dev459 < 1.0 < 1.0.post456dev463 < 1.0.post456 < 1.0.post489
>
>
>>
>> (Btw, the wiki page pseudo-regex doesn't match what the code actually
>> parses, either.)
>>
>
> i'll check it thx
>
>>
>
>
>
> --
> Tarek Ziad? | http://ziade.org
>



-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Wed May  6 21:18:20 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 06 May 2009 15:18:20 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <gtsksk$jgh$1@ger.gmane.org>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b654cd2e0905041657y6593a9f5tb3a391589485cb58@mail.gmail.com>
	<94bdd2610905050149i38344988jc1efa9b6748f033c@mail.gmail.com>
	<A1B8B6E3-564C-4CE1-9F59-B1D6D3B9CAD6@gmail.com>
	<94bdd2610905050515h7b215191t5d10ee81a405888c@mail.gmail.com>
	<9F98D556-CF38-4786-B9E9-A8F813D58892@gmail.com>
	<94bdd2610905050633t2731c08t9399ee2ded72584f@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
	<20090506174408.E0B903A410E@sparrow.telecommunity.com>
	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
	<gtsksk$jgh$1@ger.gmane.org>
Message-ID: <20090506191544.97C3A3A40A5@sparrow.telecommunity.com>

At 08:28 PM 5/6/2009 +0200, Hanno Schlichting wrote:
>Doug Hellmann wrote:
> > On May 6, 2009, at 1:46 PM, P.J. Eby wrote:
> >
> >> At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:
> >>
> >>> On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
> >>>
> >>>> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
> >>>>> I don't see any advantage, in the context of this discussion, to
> >>>>> having an additional, incompatible naming for full-path-to-a-class.
> >>>>
> >>>> Setuptools doesn't limit an entry point to being a class, function,
> >>>> or other top-level name within a module.  It can be a method of a
> >>>> class, or an attribute of an attribute.  The ':' removes any
> >>>> ambiguity as to which part of the name is the module, and which
> >>>> parts are attributes within that module.
> >>>
> >>> Is that level of complexity useful in practice?  I can understand how
> >>> it came to be implemented, but is it actually used by any projects?
> >>
> >> I use it; I'm not sure who else does.
> >>
> >> The particular use case I have (and that's most likely to be shared)
> >> is that the calling app or framework wants a callable or function, but
> >> the providing app or library implements that callable as a classmethod
> >> for convenience.
> >
> > That's pretty much what I expected.  It feels a little messy to have
> > plugins exposing "internals" like that but not so much so that I propose
> > we don't allow it. The ":" syntax seems like the right way to go.
>
>I'd be tempted to call this an edge-case. You should be able to expose
>the internal detail you'd need via a module scope alias for the
>particular case. That seems easier than providing a whole new notion.

Ah, now you reminded me of a forgotten detail: the specific use case 
was that I had a library with a Command class that was widely used as 
a base class; simply adding a classmethod to that base class allowed 
all the existing classes to be used as plugins without any change to 
their modules' source code: the entry points just always reference 
the classmethod.


From pje at telecommunity.com  Wed May  6 21:20:24 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 06 May 2009 15:20:24 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.co
 m>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
Message-ID: <20090506191746.BF4613A40A5@sparrow.telecommunity.com>

At 08:58 PM 5/6/2009 +0200, Tarek Ziad? wrote:
>Sorry to post again that mail, but it seems that this is the last remaining
>issue for the versionning proposal.
>
>Phillip, could you check my proposal for your development versions of
>postreleases
>use case ?

Is the code up-to-date with the below?


>thx
>
>On Thu, Apr 23, 2009 at 11:19 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> > 2009/4/22 P.J. Eby <pje at telecommunity.com>:
> >> I don't see how it can manage, e.g. a development version of a 
> postrelease,
> >> with an SVN rev or date stamp on it.  Such versions might not be found on
> >> PyPI or on RPMs, but would be needed in development.
> >
> > So, instead of having 'dev' and 'post', we would require a third case
> > for "pos+dev" version
> >
> > so  (dev|post)N+ could become, ((dev|post)N+)|(postN+devN+)
> >
> > example:
> >
> > 1.0.dev459 < 1.0 < 1.0.post456dev463 < 1.0.post456 < 1.0.post489
> >
> >
> >>
> >> (Btw, the wiki page pseudo-regex doesn't match what the code actually
> >> parses, either.)
> >>
> >
> > i'll check it thx
> >
> >>
> >
> >
> >
> > --
> > Tarek Ziad? | http://ziade.org
> >
>
>
>
>--
>Tarek Ziad? | http://ziade.org


From ianb at colorstudy.com  Wed May  6 21:51:25 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 6 May 2009 14:51:25 -0500
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <gtsl3t$jq0$1@ger.gmane.org>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com> 
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com> 
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com> 
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
	<20090506174408.E0B903A410E@sparrow.telecommunity.com> 
	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
	<gtsksk$jgh$1@ger.gmane.org> <gtsl3t$jq0$1@ger.gmane.org>
Message-ID: <b654cd2e0905061251k1cc02f8fhc909b69fc8ec7a76@mail.gmail.com>

On Wed, May 6, 2009 at 1:32 PM, Tres Seaver <tseaver at palladion.com> wrote:

>  > I'd be tempted to call this an edge-case. You should be able to expose
> > the internal detail you'd need via a module scope alias for the
> > particular case. That seems easier than providing a whole new notion.
>
> I'm actually a big fan of the ':', because it makes explicit the
> difference between the "import" and the "named thing", even for
> module-scoped names.
>

Yeah, I also like it primarily for clarity.  Also you can provide better
error messages when it fails, matching up the error against the intention.

In some contexts I also extend this syntax to do something like:

  module, path = name.split(':', 1)
  obj = eval(path, load_module(module).__dict__)

I don't propose that we do eval for plugins, but it's a nice side-effect of
the syntax that it's possible to add in other contexts.  Also I don't think
there's any strong precedence for purely dot notation, loading objects from
strings is something that's always done ad hoc, and the only widely used
library I know of that people use for this is Setuptools (indirectly through
entry points).

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090506/02942c7d/attachment-0001.htm>

From ziade.tarek at gmail.com  Wed May  6 22:11:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 May 2009 22:11:31 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090506191746.BF4613A40A5@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>

On Wed, May 6, 2009 at 9:20 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:58 PM 5/6/2009 +0200, Tarek Ziad? wrote:
>>
>> Sorry to post again that mail, but it seems that this is the last
>> remaining
>> issue for the versionning proposal.
>>
>> Phillip, could you check my proposal for your development versions of
>> postreleases
>> use case ?
>
> Is the code up-to-date with the below?

No, I'll do a branch then with that version
>
>
>> thx
>>
>> On Thu, Apr 23, 2009 at 11:19 AM, Tarek Ziad? <ziade.tarek at gmail.com>
>> wrote:
>> > 2009/4/22 P.J. Eby <pje at telecommunity.com>:
>> >> I don't see how it can manage, e.g. a development version of a
>> >> postrelease,
>> >> with an SVN rev or date stamp on it. ?Such versions might not be found
>> >> on
>> >> PyPI or on RPMs, but would be needed in development.
>> >
>> > So, instead of having 'dev' and 'post', we would require a third case
>> > for "pos+dev" version
>> >
>> > so ?(dev|post)N+ could become, ((dev|post)N+)|(postN+devN+)
>> >
>> > example:
>> >
>> > 1.0.dev459 < 1.0 < 1.0.post456dev463 < 1.0.post456 < 1.0.post489
>> >
>> >
>> >>
>> >> (Btw, the wiki page pseudo-regex doesn't match what the code actually
>> >> parses, either.)
>> >>
>> >
>> > i'll check it thx
>> >
>> >>
>> >
>> >
>> >
>> > --
>> > Tarek Ziad? | http://ziade.org
>> >
>>
>>
>>
>> --
>> Tarek Ziad? | http://ziade.org
>
>



-- 
Tarek Ziad? | http://ziade.org

From hannosch at hannosch.eu  Wed May  6 22:26:55 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Wed, 06 May 2009 22:26:55 +0200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <b654cd2e0905061251k1cc02f8fhc909b69fc8ec7a76@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>	<20090506174408.E0B903A410E@sparrow.telecommunity.com>
	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>	<gtsksk$jgh$1@ger.gmane.org>
	<gtsl3t$jq0$1@ger.gmane.org>
	<b654cd2e0905061251k1cc02f8fhc909b69fc8ec7a76@mail.gmail.com>
Message-ID: <gtsrqe$ctn$1@ger.gmane.org>

Ian Bicking wrote:
> On Wed, May 6, 2009 at 1:32 PM, Tres Seaver <tseaver at palladion.com
> <mailto:tseaver at palladion.com>> wrote:
> 
>     I'm actually a big fan of the ':', because it makes explicit the
>     difference between the "import" and the "named thing", even for
>     module-scoped names.
> 
> Yeah, I also like it primarily for clarity.  Also you can provide better
> error messages when it fails, matching up the error against the intention.

[...]

> Also I don't
> think there's any strong precedence for purely dot notation, loading
> objects from strings is something that's always done ad hoc, and the
> only widely used library I know of that people use for this is
> Setuptools (indirectly through entry points).

The one I and others use in quite many places is zope.dottedname [1]
(the actual function is at [2]). This one doesn't make any difference
between modules and attributes. It has no zope dependency but happens to
be used throughout the entire Zope / Plone stack.

Hanno

[1] http://pypi.python.org/pypi/zope.dottedname
[2]
http://svn.zope.org/zope.dottedname/trunk/src/zope/dottedname/resolve.py?view=markup


From ziade.tarek at gmail.com  Thu May  7 10:02:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 7 May 2009 10:02:50 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
Message-ID: <94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>

On Wed, May 6, 2009 at 10:11 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Wed, May 6, 2009 at 9:20 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 08:58 PM 5/6/2009 +0200, Tarek Ziad? wrote:
>>>
>>> Sorry to post again that mail, but it seems that this is the last
>>> remaining
>>> issue for the versionning proposal.
>>>
>>> Phillip, could you check my proposal for your development versions of
>>> postreleases
>>> use case ?
>>
>> Is the code up-to-date with the below?
>
> No, I'll do a branch then with that version

done (I have also fixed minor bugs in that branch

http://bitbucket.org/tarek/distutilsversion/src/3fe1afab2e1a/

Tarek

-- 
Tarek Ziad? | http://ziade.org

From wheatix at gmail.com  Thu May  7 03:38:47 2009
From: wheatix at gmail.com (Wheat)
Date: Wed, 6 May 2009 18:38:47 -0700 (PDT)
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
Message-ID: <b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>


> There are *many* benefits of adding entry points into Distutils. In
> fact, adding a plugin system in there,
> will help make the commands more extendable and therefore will help us
> in the long term to remove things
> out of Distutils.
>
> So any strong opinion against them ?
>
> If not, I'll add a section in PEP 376 - and the first use case I can
> see is having a pluggable way to extend
> the list of files contained in .egg-info.
>
> The work to be done would be to extract entry points from
> pkg_resources (the discovery work that consists of looking
> for entry_points.txt files will be covered by the APIs already
> described in that PEP) This is not hard.
>

Yes, I'd like to see entry_points added to Distutils!

I'll also mention my most common use-case for using entry_points is
installing
console_scripts using zc.recipe.egg. This use-case only needs
discovery of
entry_points. Once a distribution's entry_points are made available,
it's
entirely up to the application installer (eg. zc.recipe.egg) as to
what is
done with the entry_point information.

For example, let's say that I have a Python project which I want to
use
zest.releaser for. The entry_points for this package is:

    entry_points={
        'console_scripts': [
            'release = zest.releaser.release:main',
            'prerelease = zest.releaser.prerelease:main',
            'postrelease = zest.releaser.postrelease:main',
            'fullrelease = zest.releaser.fullrelease:main',
            'longtest = zest.releaser.longtest:main',
            'lasttagdiff = zest.releaser.lasttagdiff:main'],
    }

Then I can use buildout to install this package into a project's
deployment
with:

    [buildout]
    parts = releaser

    [releaser]
    recipe = zc.recipe.egg
    eggs = zest.releaser

Where the zc.recipe.egg recipe will turn all 'console_scripts'
entry_points
into scripts in the ./bin directory. Which I find pretty nice :)

Anyways, I'm mentioning this use-case just to re-iterate that there is
a
difference between asking a distribution what entry_points it provides
(discovery) and deciding what configuration or actions to take based
on that
information (activation or script generation). Also, as for discovery,
this
use-case only requires discovery on a per-distribution basis, it
doesn't
need a 'iterate over all distribution's entry_points installed in some
location' feature.

Also, using console_scripts for entry_points means that there is a
second
way of specifying scripts, since Distutils already has the 'scripts'
metadata field. I would say that the 'scripts' field should then be
deprecated, but perhaps there are reason's beyond breaking backwards
compatability for keeping that field around?

From ziade.tarek at gmail.com  Thu May  7 11:14:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 7 May 2009 11:14:04 +0200
Subject: [Distutils] Distutils global status
Message-ID: <94bdd2610905070214i2874b86eue489c81d722bfc7b@mail.gmail.com>

Hi,

Here's a quick summary of the ongoing work in Distutils, on the issues
we seem to reach a consensus.

The goal would be to finish the discussions on those and to push them
in some kind of "Distutils official roadmap"
then look at implementation details (for instance, the pkgutil APIs
will be linear unless we use some kind of index)

- task :
  - link
  - remaining work

---

- Introduction the new version comparison algorithm in Distutils

   - http://bitbucket.org/tarek/distutilsversion
   - Phillip needs to check the tarek-postdev branch for his use case,
then Trent and al.
   - We need to add the "bits" constructor

- Standardize the .egg-info directory, provide APIs, uninstall command

  - http://svn.python.org/view/peps/trunk/pep-0376.txt?revision=72420&view=markup
  - we need to peer-review it here again

- Change PKG-INFO content (PEP 345 changes)

  - http://svn.python.org/view/peps/branches/jim-update-345/pep-0345.txt?revision=71412&view=markup
  - Tres needs to push the latest changes
  - we need to peer-review it here again

- Finish the massive cleanup / test coverage - not much to say
  - work in progress by Tarek


Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From noah.gift at gmail.com  Thu May  7 13:18:29 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Thu, 7 May 2009 23:18:29 +1200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090506191544.97C3A3A40A5@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<90AF5462-E20C-4056-BA94-4FFA58D4FE37@gmail.com>
	<94bdd2610905050805u49f013va85d39d850089c76@mail.gmail.com>
	<87eiv3htrm.fsf@benfinney.id.au>
	<20090506024804.1FFCC3A4080@sparrow.telecommunity.com>
	<FD2D0ABC-CE93-4AC3-B03B-23B3C80611D2@gmail.com>
	<20090506174408.E0B903A410E@sparrow.telecommunity.com>
	<5AE981AE-865A-4C4F-9AC3-F38EED819FE9@gmail.com>
	<gtsksk$jgh$1@ger.gmane.org>
	<20090506191544.97C3A3A40A5@sparrow.telecommunity.com>
Message-ID: <e91cc0270905070418n70529ca0ie34c0afee0fc236f@mail.gmail.com>

On Thu, May 7, 2009 at 7:18 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:28 PM 5/6/2009 +0200, Hanno Schlichting wrote:
>>
>> Doug Hellmann wrote:
>> > On May 6, 2009, at 1:46 PM, P.J. Eby wrote:
>> >
>> >> At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote:
>> >>
>> >>> On May 5, 2009, at 10:50 PM, P.J. Eby wrote:
>> >>>
>> >>>> At 12:03 PM 5/6/2009 +1000, Ben Finney wrote:
>> >>>>> I don't see any advantage, in the context of this discussion, to
>> >>>>> having an additional, incompatible naming for full-path-to-a-class.
>> >>>>
>> >>>> Setuptools doesn't limit an entry point to being a class, function,
>> >>>> or other top-level name within a module. ?It can be a method of a
>> >>>> class, or an attribute of an attribute. ?The ':' removes any
>> >>>> ambiguity as to which part of the name is the module, and which
>> >>>> parts are attributes within that module.
>> >>>
>> >>> Is that level of complexity useful in practice? ?I can understand how
>> >>> it came to be implemented, but is it actually used by any projects?
>> >>
>> >> I use it; I'm not sure who else does.
>> >>
>> >> The particular use case I have (and that's most likely to be shared)
>> >> is that the calling app or framework wants a callable or function, but
>> >> the providing app or library implements that callable as a classmethod
>> >> for convenience.
>> >
>> > That's pretty much what I expected. ?It feels a little messy to have
>> > plugins exposing "internals" like that but not so much so that I propose
>> > we don't allow it. The ":" syntax seems like the right way to go.
>>
>> I'd be tempted to call this an edge-case. You should be able to expose
>> the internal detail you'd need via a module scope alias for the
>> particular case. That seems easier than providing a whole new notion.
>
> Ah, now you reminded me of a forgotten detail: the specific use case was
> that I had a library with a Command class that was widely used as a base
> class; simply adding a classmethod to that base class allowed all the
> existing classes to be used as plugins without any change to their modules'
> source code: the entry points just always reference the classmethod.

[slightly off topic]
It seems like registration is the canonical use case for classmethods
in Python no?  Tarek has a good example in Expert Python Programming
of this in his observer example.
[/end]


>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Cheers,

Noah

From doug.hellmann at gmail.com  Thu May  7 14:35:44 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 08:35:44 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
Message-ID: <082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>


On May 6, 2009, at 9:38 PM, Wheat wrote:

> I'll also mention my most common use-case for using entry_points is
> installing
> console_scripts using zc.recipe.egg.

I'm curious about that because I've never understood the benefit of  
using entry points for console scripts.  Why not just list the script  
as a part of the package and have it installed "normally"?

> Also, using console_scripts for entry_points means that there is a
> second
> way of specifying scripts, since Distutils already has the 'scripts'
> metadata field. I would say that the 'scripts' field should then be
> deprecated, but perhaps there are reason's beyond breaking backwards
> compatability for keeping that field around?

I would argue the other way.  Why force authors of console scripts to  
deal with entry points instead of just installing the script as-is?

Doug


From p.f.moore at gmail.com  Thu May  7 14:54:01 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 May 2009 13:54:01 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
Message-ID: <79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>

2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
> I would argue the other way. ?Why force authors of console scripts to deal
> with entry points instead of just installing the script as-is?

Please explain "as-is" with reference to ensuring that the script
works cross-platform. I think the benefit of entry points for scripts
is that it generates appropriate wrappers to allow use on all
platforms.

Having said that, I find setuptools entry points to be
over-engineered, and the Windows wrappers (in particular, the fact
that they are not version-independent) to be somewhat clumsy. But as a
concept, I like the idea of having a way of specifying that a script
is intended as an "executable", and having distutils do the job of
generating whatever platform cruft is required [1] to make that work.

Of course, for even remotely modern Python versions, I'd argue
strongly that packages shouldn't be including console scripts, but
should rather be supplying modules that can be run as scripts, via the
-m argument to python. Users can then build aliases, shell scripts, or
whatever is appropriate based on that.

Paul.

[1] And note especially that .bat files are *not* suitable wrappers on
Windows, in spite of the fact that they are commonly used. Their
biggest disadvantage is that they don't nest.

From doug.hellmann at gmail.com  Thu May  7 15:50:15 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 09:50:15 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
Message-ID: <5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>


On May 7, 2009, at 8:54 AM, Paul Moore wrote:

> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>> I would argue the other way.  Why force authors of console scripts  
>> to deal
>> with entry points instead of just installing the script as-is?
>
> Please explain "as-is" with reference to ensuring that the script
> works cross-platform. I think the benefit of entry points for scripts
> is that it generates appropriate wrappers to allow use on all
> platforms.

I write a python script call hello.py like this:

	#!/usr/bin/env python

	def main():
		print 'hello!'

	if __name__ == '__main__':
		main()

Why make me define an entry point for that?  I can just put it in /usr/ 
bin or somewhere in the path on Windows and call it as "hello.py".

Does setuptools give me something extra for Windows?  I'm not a  
regular Windows user, so it's likely that there are features I don't  
know about.

> Having said that, I find setuptools entry points to be
> over-engineered, and the Windows wrappers (in particular, the fact
> that they are not version-independent) to be somewhat clumsy. But as a
> concept, I like the idea of having a way of specifying that a script
> is intended as an "executable", and having distutils do the job of
> generating whatever platform cruft is required [1] to make that work.

That's the part I'm not clear about.  What "cruft" is needed anywhere?

> Of course, for even remotely modern Python versions, I'd argue
> strongly that packages shouldn't be including console scripts, but
> should rather be supplying modules that can be run as scripts, via the
> -m argument to python. Users can then build aliases, shell scripts, or
> whatever is appropriate based on that.
>
> Paul.
>
> [1] And note especially that .bat files are *not* suitable wrappers on
> Windows, in spite of the fact that they are commonly used. Their
> biggest disadvantage is that they don't nest.


From p.f.moore at gmail.com  Thu May  7 16:03:29 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 May 2009 15:03:29 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
Message-ID: <79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>

2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
> I write a python script call hello.py like this:
>
> ? ? ? ?#!/usr/bin/env python
>
> ? ? ? ?def main():
> ? ? ? ? ? ? ? ?print 'hello!'
>
> ? ? ? ?if __name__ == '__main__':
> ? ? ? ? ? ? ? ?main()
>
> Why make me define an entry point for that? ?I can just put it in /usr/bin
> or somewhere in the path on Windows and call it as "hello.py".

That works but a lot of Unix users have in the past objected to having
'.py' in the name. People then start trying to cater for them by
leaving off the '.py', which means it doesn't work on Windows, etc
etc.

As a Windows user, I'm all in favour of this. But can you persuade the
Unix users to agree with me, please? :-)

> Does setuptools give me something extra for Windows? ?I'm not a regular
> Windows user, so it's likely that there are features I don't know about.

I don't think so, as such. It gives Unix and Windows users who care
(== not me, and clearly not you, either) the ability to call the
command "hello" rather than "hello.py".

Paul.

From floris.bruynooghe at gmail.com  Thu May  7 16:12:15 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 7 May 2009 15:12:15 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
Message-ID: <20090507141215.GA16344@laurie.devork>

On Thu, May 07, 2009 at 03:03:29PM +0100, Paul Moore wrote:
> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
> > Why make me define an entry point for that? ?I can just put it in /usr/bin
> > or somewhere in the path on Windows and call it as "hello.py".
> 
> That works but a lot of Unix users have in the past objected to having
> '.py' in the name. People then start trying to cater for them by
> leaving off the '.py', which means it doesn't work on Windows, etc
> etc.

Why can't the install_scripts command from distutils install with a
.py extension on windonws and without on UNIX?  (Or rather: what would
be wrong with such an approach?)


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From eric at trueblade.com  Thu May  7 16:15:19 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 07 May 2009 10:15:19 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
Message-ID: <4A02ECF7.80004@trueblade.com>

Doug Hellmann wrote:
> 
> On May 7, 2009, at 8:54 AM, Paul Moore wrote:
> 
>> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>>> I would argue the other way.  Why force authors of console scripts to 
>>> deal
>>> with entry points instead of just installing the script as-is?
>>
>> Please explain "as-is" with reference to ensuring that the script
>> works cross-platform. I think the benefit of entry points for scripts
>> is that it generates appropriate wrappers to allow use on all
>> platforms.
> 
> I write a python script call hello.py like this:
> 
>     #!/usr/bin/env python
> 
>     def main():
>         print 'hello!'
> 
>     if __name__ == '__main__':
>         main()
> 
> Why make me define an entry point for that?  I can just put it in 
> /usr/bin or somewhere in the path on Windows and call it as "hello.py".
> 
> Does setuptools give me something extra for Windows?  I'm not a regular 
> Windows user, so it's likely that there are features I don't know about.

Yes. It creates a .exe wrapper [1]. By using entry points, I don't need 
to care what the target system is. Also, /usr/bin/env might invoke the 
wrong python.

>> [1] And note especially that .bat files are *not* suitable wrappers on
>> Windows, in spite of the fact that they are commonly used. Their
>> biggest disadvantage is that they don't nest.

Which is why it creates .exe wrappers, not batch files.


[1] http://peak.telecommunity.com/DevCenter/EasyInstall

From tseaver at palladion.com  Thu May  7 16:20:46 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 May 2009 10:20:46 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <4A02ECF7.80004@trueblade.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com>
Message-ID: <gtuqo0$ir5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
> Doug Hellmann wrote:
>> On May 7, 2009, at 8:54 AM, Paul Moore wrote:
>>
>>> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>>>> I would argue the other way.  Why force authors of console scripts to 
>>>> deal
>>>> with entry points instead of just installing the script as-is?
>>> Please explain "as-is" with reference to ensuring that the script
>>> works cross-platform. I think the benefit of entry points for scripts
>>> is that it generates appropriate wrappers to allow use on all
>>> platforms.
>> I write a python script call hello.py like this:
>>
>>     #!/usr/bin/env python
>>
>>     def main():
>>         print 'hello!'
>>
>>     if __name__ == '__main__':
>>         main()
>>
>> Why make me define an entry point for that?  I can just put it in 
>> /usr/bin or somewhere in the path on Windows and call it as "hello.py".
>>
>> Does setuptools give me something extra for Windows?  I'm not a regular 
>> Windows user, so it's likely that there are features I don't know about.
> 
> Yes. It creates a .exe wrapper [1]. By using entry points, I don't need 
> to care what the target system is. Also, /usr/bin/env might invoke the 
> wrong python.

Exactly:  using entry points for console scripts guarantees that the
python into which the corresponding distribution is installed is the one
used to run the script, which is *highly* desirable.  Otherwise, you end
up with the "just install everything in the system Python's
site-packages" mess.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKAu4++gerLs4ltQ4RAjW4AJ0cM5oqDL35qoBq1nnl4YD1RmX1NACghbqM
Zom9yb5WCzaXFrMz7V75zQU=
=CZLG
-----END PGP SIGNATURE-----


From doug.hellmann at gmail.com  Thu May  7 16:35:53 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 10:35:53 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
Message-ID: <7664DEE4-3B34-434E-94DB-396FEC453607@gmail.com>


On May 7, 2009, at 10:03 AM, Paul Moore wrote:

> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>>
>> Does setuptools give me something extra for Windows?  I'm not a  
>> regular
>> Windows user, so it's likely that there are features I don't know  
>> about.
>
> I don't think so, as such. It gives Unix and Windows users who care
> (== not me, and clearly not you, either) the ability to call the
> command "hello" rather than "hello.py".


Hmm, yeah, I see that as useful.  I don't work with Windows a lot  
myself, but I don't want to explicitly drop support there, either.

Doug


From p.f.moore at gmail.com  Thu May  7 16:36:52 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 May 2009 15:36:52 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <gtuqo0$ir5$1@ger.gmane.org>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
Message-ID: <79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>

2009/5/7 Tres Seaver <tseaver at palladion.com>:
> Eric Smith wrote:
>> Doug Hellmann wrote:
>>> On May 7, 2009, at 8:54 AM, Paul Moore wrote:
>>>
>>>> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>>>>> I would argue the other way. ?Why force authors of console scripts to
>>>>> deal
>>>>> with entry points instead of just installing the script as-is?
>>>> Please explain "as-is" with reference to ensuring that the script
>>>> works cross-platform. I think the benefit of entry points for scripts
>>>> is that it generates appropriate wrappers to allow use on all
>>>> platforms.
>>> I write a python script call hello.py like this:
>>>
>>> ? ? #!/usr/bin/env python
>>>
>>> ? ? def main():
>>> ? ? ? ? print 'hello!'
>>>
>>> ? ? if __name__ == '__main__':
>>> ? ? ? ? main()
>>>
>>> Why make me define an entry point for that? ?I can just put it in
>>> /usr/bin or somewhere in the path on Windows and call it as "hello.py".
>>>
>>> Does setuptools give me something extra for Windows? ?I'm not a regular
>>> Windows user, so it's likely that there are features I don't know about.
>>
>> Yes. It creates a .exe wrapper [1]. By using entry points, I don't need
>> to care what the target system is. Also, /usr/bin/env might invoke the
>> wrong python.
>
> Exactly: ?using entry points for console scripts guarantees that the
> python into which the corresponding distribution is installed is the one
> used to run the script, which is *highly* desirable. ?Otherwise, you end
> up with the "just install everything in the system Python's
> site-packages" mess.

... and somewhere around here we end up with what I described as an
over-engineered solution.

By trying to satisfy everyone's requirements, you ultimately satisfy no-one's.

Sigh. I keep meaning to avoid getting sucked back into this tar-pit,
and I keep failing :-(

Just put me down as a hearty +1 for Doug's "just deploy a script
called whatever.py" approach for standalone stuff, and using python -m
for scripts distributed as part of larger distributions.

Paul.

From doug.hellmann at gmail.com  Thu May  7 16:38:36 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 10:38:36 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <gtuqo0$ir5$1@ger.gmane.org>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
Message-ID: <B804F891-0E30-4844-9E70-A9E22A8DDD95@gmail.com>


On May 7, 2009, at 10:20 AM, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Eric Smith wrote:
>> Doug Hellmann wrote:
>>> On May 7, 2009, at 8:54 AM, Paul Moore wrote:
>>>
>>>> 2009/5/7 Doug Hellmann <doug.hellmann at gmail.com>:
>>>>> I would argue the other way.  Why force authors of console  
>>>>> scripts to
>>>>> deal
>>>>> with entry points instead of just installing the script as-is?
>>>> Please explain "as-is" with reference to ensuring that the script
>>>> works cross-platform. I think the benefit of entry points for  
>>>> scripts
>>>> is that it generates appropriate wrappers to allow use on all
>>>> platforms.
>>> I write a python script call hello.py like this:
>>>
>>>    #!/usr/bin/env python
>>>
>>>    def main():
>>>        print 'hello!'
>>>
>>>    if __name__ == '__main__':
>>>        main()
>>>
>>> Why make me define an entry point for that?  I can just put it in
>>> /usr/bin or somewhere in the path on Windows and call it as  
>>> "hello.py".
>>>
>>> Does setuptools give me something extra for Windows?  I'm not a  
>>> regular
>>> Windows user, so it's likely that there are features I don't know  
>>> about.
>>
>> Yes. It creates a .exe wrapper [1]. By using entry points, I don't  
>> need
>> to care what the target system is. Also, /usr/bin/env might invoke  
>> the
>> wrong python.
>
> Exactly:  using entry points for console scripts guarantees that the
> python into which the corresponding distribution is installed is the  
> one
> used to run the script, which is *highly* desirable.  Otherwise, you  
> end
> up with the "just install everything in the system Python's
> site-packages" mess.

pip installs my scripts into a virtualenv without any issue and  
without using entry points, AFAICT.

I guess if we move to requiring entry points and disallowing simple  
script distribution I'll need to find another way to package  
virtualenvwrapper.  Since it's a bash script, it doesn't have entry  
points.  I've been using setuptools to package it so it can be  
installed via easy_install, since it is a Python development tool.

Doug


From wheatix at gmail.com  Thu May  7 22:14:06 2009
From: wheatix at gmail.com (Wheat)
Date: Thu, 7 May 2009 13:14:06 -0700 (PDT)
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com> 
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com> 
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com> 
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
Message-ID: <d3bef572-8f04-4abc-b4b4-65f51ae9422d@d19g2000prh.googlegroups.com>


> I write a python script call hello.py like this:
>
>         #!/usr/bin/env python
>
>         def main():
>                 print 'hello!'
>
>         if __name__ == '__main__':
>                 main()
>
> Why make me define an entry point for that?  I can just put it in /usr/
> bin or somewhere in the path on Windows and call it as "hello.py".

With an entry point, hello.py could simply be:

    def main():
            print 'hello!'

But I agree, the Distutils 'scripts' is conceptually simpler.
Especially
since it let's you run a script 'as-is' without first running an
install
tool to generate a script wrapper. It's only once a script has
dependencies
on other Python libraries where relying on those libraries being
available
for import may not work. In this case, the 'scripts' field forces the
user
to install those libraries in a global location (site-packages), this
isn't always desirable though!

As for including BASH scripts in the 'scripts' field ... I dunno.
Clearly
entry_points are meant to only annotate Python code! Is including BASH
scripts in the 'scripts' field an abuse of Distutils or is it
acceptable
usage? The Distutils docs aren't explicit on this, so I'm inclined to
say
it's acceptable, so this is at least one good reason for keeping the
'scripts' field.

But then having both 'scripts' and 'entry_points/console_scripts' is
less than perfect since it introduces (mostly) unneccessary TIMTOWTDI.

From ben+python at benfinney.id.au  Thu May  7 23:53:03 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 08 May 2009 07:53:03 +1000
Subject: [Distutils] Adding entry points into Distutils ?
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
Message-ID: <87k54sfum8.fsf@benfinney.id.au>

Doug Hellmann <doug.hellmann at gmail.com> writes:

> I write a python script call hello.py like this:
> 
> 	#!/usr/bin/env python
> 
> 	def main():
> 		print 'hello!'
> 
> 	if __name__ == '__main__':
> 		main()
> 
> Why make me define an entry point for that?  I can just put it in
> /usr/ bin or somewhere in the path on Windows and call it as
> "hello.py".

To address the issue raised elsewhere in this thread:

Why should the name of a *command* include the suffix ?.py?? That just
begs for the situation down the line where one of these commands is
being used widely, by the name ?hello.py?, and then someone wants to
provide an alternative implementation in another language.

A command implemented in Ruby named ?hello.py? (for backward
compatibility) is even more annoying that one implemented in Python.

This situation is entirely predictable, so naming it *without* a suffix
is the right way to go. It also makes the command less annoying to type.


The simple rule of thumb:

If it's primarily a module to be imported, name it ?foo.py?, don't mark
it executable, and don't use the shebang line.

If it's primarily a command to be run as the main program, name it ?foo?
with no ?.py? suffix, mark it executable, and use the shebang line.

-- 
 \     ?I have had a perfectly wonderful evening, but this wasn't it.? |
  `\                                                     ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From eric at trueblade.com  Fri May  8 00:04:21 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 07 May 2009 18:04:21 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	<4A02ECF7.80004@trueblade.com>
	<gtuqo0$ir5$1@ger.gmane.org>
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
Message-ID: <4A035AE5.3040203@trueblade.com>

Paul Moore wrote:
> 2009/5/7 Tres Seaver <tseaver at palladion.com>:
>> Eric Smith wrote:
>>> Yes. It creates a .exe wrapper [1]. By using entry points, I don't need
>>> to care what the target system is. Also, /usr/bin/env might invoke the
>>> wrong python.
>> Exactly:  using entry points for console scripts guarantees that the
>> python into which the corresponding distribution is installed is the one
>> used to run the script, which is *highly* desirable.  Otherwise, you end
>> up with the "just install everything in the system Python's
>> site-packages" mess.
> 
> ... and somewhere around here we end up with what I described as an
> over-engineered solution.

Or, what I call a minimal set of required functionality.

> By trying to satisfy everyone's requirements, you ultimately satisfy no-one's.

Not sure I agree, but in any event that's certainly not a reason to try 
and make a generally useful solution.

I think entry points and scripts are two issues that shouldn't be 
conflated. Entry points are generally useful (to me and others), and 
having a way of the installer knowing about scripts and generating 
correct wrappers for them (in a cross-platform way) is also generally 
useful (to me and others). That scripts currently use entry points is a 
design detail that perhaps doesn't need to be exposed.

Eric.


From noah.gift at gmail.com  Fri May  8 00:21:14 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Fri, 8 May 2009 10:21:14 +1200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <4A035AE5.3040203@trueblade.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
	<4A035AE5.3040203@trueblade.com>
Message-ID: <e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>

On Fri, May 8, 2009 at 10:04 AM, Eric Smith <eric at trueblade.com> wrote:
> Paul Moore wrote:
>>
>> 2009/5/7 Tres Seaver <tseaver at palladion.com>:
>>>
>>> Eric Smith wrote:
>>>>
>>>> Yes. It creates a .exe wrapper [1]. By using entry points, I don't need
>>>> to care what the target system is. Also, /usr/bin/env might invoke the
>>>> wrong python.
>>>
>>> Exactly: ?using entry points for console scripts guarantees that the
>>> python into which the corresponding distribution is installed is the one
>>> used to run the script, which is *highly* desirable. ?Otherwise, you end
>>> up with the "just install everything in the system Python's
>>> site-packages" mess.
>>
>> ... and somewhere around here we end up with what I described as an
>> over-engineered solution.
>
> Or, what I call a minimal set of required functionality.
>
>> By trying to satisfy everyone's requirements, you ultimately satisfy
>> no-one's.
>
> Not sure I agree, but in any event that's certainly not a reason to try and
> make a generally useful solution.
>
> I think entry points and scripts are two issues that shouldn't be conflated.
> Entry points are generally useful (to me and others), and having a way of
> the installer knowing about scripts and generating correct wrappers for them
> (in a cross-platform way) is also generally useful (to me and others).

Two current problems I have with console scripts with setuptools are:

1.  Different versions of Python conflict with previous versions of
console scripts.  Take paste for example.
2.  The entry point mechanism IIRC recursively scans the site-packages
directory and loads up the system path with eggs.  This is too
expensive of an operation for the current environment I work in.
3.  There doesn't seem to be a clean way to inject user specific
environment details to the console script.
I often need the ability to alter the sys.path in a user specific way
for the entry point without needing to mess up the global sys.path
permanently.

> scripts currently use entry points is a design detail that perhaps doesn't
> need to be exposed.
>
> Eric.
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Cheers,

Noah

From wheatix at gmail.com  Thu May  7 22:18:35 2009
From: wheatix at gmail.com (Wheat)
Date: Thu, 7 May 2009 13:18:35 -0700 (PDT)
Subject: [Distutils] Setuptools + Subversion 1.6 - new setuptools
	release necessary
In-Reply-To: <4A013739.6020105@zopyx.com>
References: <49F69462.6050309@zopyx.com> <49F6FBC1.8080001@simplistix.co.uk> 
	<49F6FD73.10809@zopyx.com> <4A013739.6020105@zopyx.com>
Message-ID: <99d67a00-c550-4b8f-9914-a7348a587b1e@k19g2000prh.googlegroups.com>



On May 6, 12:07?am, Andreas Jung <li... at zopyx.com> wrote:
> On 28.04.09 14:58, Andreas Jung wrote:
>
> > On 28.04.2009 14:51 Uhr, Chris Withers wrote:
>
> >> Andreas Jung wrote:
>
> >>> it is known that the latest setuptools version produces broken
> >>> packages with SVN 1.6 checkouts. Could we get a fixed setuptools
> >>> version asap - fixing this issue is essential
> >>> (there is some patch floating around).
>
> There is obviously a bugreport and patches available for this issue:
>
> http://bugs.python.org/setuptools/issue64
>
> Any chance to see this patch being merged and released together
> with a new setuptools version soon?
>

*bump*, *bump*

Yeah, I spent a good amount of time yesterday banging my head against
broken sdists after recently upgrading to SVN 1.6. A new setuptools
release would be much appreciated!



From david.lyon at preisshare.net  Fri May  8 00:30:20 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 07 May 2009 18:30:20 -0400
Subject: [Distutils] =?utf-8?q?Adding_entry_points_into_Distutils_=3F?=
In-Reply-To: <20090507141215.GA16344@laurie.devork>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
	<20090507141215.GA16344@laurie.devork>
Message-ID: <a3d3025b8757ea2386205aa019de4994@preisshare.net>


On Thu, 7 May 2009 15:12:15 +0100, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> Why can't the install_scripts command from distutils install with a
> .py extension on windonws and without on UNIX?  (Or rather: what would
> be wrong with such an approach?)

Perphaps it is too much of a good idea....

or perphaps it would make things too easy for windows and unix users...

David



From doug.hellmann at gmail.com  Fri May  8 00:54:48 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 18:54:48 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <87k54sfum8.fsf@benfinney.id.au>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<87k54sfum8.fsf@benfinney.id.au>
Message-ID: <9A9BC2E7-FA3F-47A2-A968-2D9DC4DC3277@gmail.com>


On May 7, 2009, at 5:53 PM, Ben Finney wrote:

> Doug Hellmann <doug.hellmann at gmail.com> writes:
>
>> I write a python script call hello.py like this:
>>
>> 	#!/usr/bin/env python
>>
>> 	def main():
>> 		print 'hello!'
>>
>> 	if __name__ == '__main__':
>> 		main()
>>
>> Why make me define an entry point for that?  I can just put it in
>> /usr/ bin or somewhere in the path on Windows and call it as
>> "hello.py".
>
> To address the issue raised elsewhere in this thread:
>
> Why should the name of a *command* include the suffix ?.py?? That just
> begs for the situation down the line where one of these commands is
> being used widely, by the name ?hello.py?, and then someone wants to
> provide an alternative implementation in another language.

I personally don't care if it includes the .py suffix or not, most of  
the time.  I tend not to include it in my own code, and don't get bent  
out of shape if someone else's tool does use it.  Of course, as I've  
been reminded in this thread not including the .py (and not using  
entry points) means my "scripts" don't work as-is when installed on  
Windows.

Doug


From pje at telecommunity.com  Fri May  8 01:24:28 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 07 May 2009 19:24:28 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.co
 m>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
	<4A035AE5.3040203@trueblade.com>
	<e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>
Message-ID: <20090507232150.165F13A4080@sparrow.telecommunity.com>

At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
>1.  Different versions of Python conflict with previous versions of
>console scripts.  Take paste for example.

I don't understand what you mean.

>2.  The entry point mechanism IIRC recursively scans the site-packages
>directory and loads up the system path with eggs.  This is too
>expensive of an operation for the current environment I work in.

The scan is not recursive; only files that are actually *on* sys.path 
are scanned: i.e., either an .egg that is directly on sys.path, or an 
.egg-info in a directory that is directly on sys.path.

Now, it's possible that some application you are using does such a 
scan explicitly; I'm just noting that merely querying or loading 
entry points doesn't cause any recursive scans, and it most 
definitely does not add anything new to sys.path, unless the entry 
point to be loaded has declared an additional dependency that's *not* 
on sys.path yet.


>3.  There doesn't seem to be a clean way to inject user specific
>environment details to the console script.
>I often need the ability to alter the sys.path in a user specific way
>for the entry point without needing to mess up the global sys.path
>permanently.

I don't understand what you mean here, either.


From noah.gift at gmail.com  Fri May  8 01:49:38 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Fri, 8 May 2009 11:49:38 +1200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090507232150.165F13A4080@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
	<4A035AE5.3040203@trueblade.com>
	<e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>
	<20090507232150.165F13A4080@sparrow.telecommunity.com>
Message-ID: <e91cc0270905071649x5fa2c142yed25aa44b3839670@mail.gmail.com>

resending, as I accidently only sent to PJE

On Fri, May 8, 2009 at 11:48 AM, Noah Gift <noah.gift at gmail.com> wrote:
> On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
>>>
>>> 1.  Different versions of Python conflict with previous versions of
>>> console scripts.  Take paste for example.
>>
>> I don't understand what you mean.
>
> Sorry that was a bit curt.
>
> One of the problems with Pylons installation and virtualenv is that
> you might have previously installed paste script, and that could have
> been triggered to a specific version of paste like this:
>
>
> #!/vol/apps/python-2.5.1_64/bin/python
> # EASY-INSTALL-ENTRY-SCRIPT: 'PasteScript==1.7.3','console_scripts','paster'
> __requires__ = 'PasteScript==1.7.3'
> import sys
> from pkg_resources import load_entry_point
>
> sys.exit(
>   load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
>
> if you do a which paste, you will see this:
>
> CSH#ngift at naseberry# which paste
> /usr/bin/paste
>
>
> What this means is the you could really want some theoretical new
> version of the paste script in your virtualenv, or in a standard
> python site-packages directory, and or a different version of python,
> say 2.6, but the name of the script hides which actual version you
> want and it specifically chooses one version of Python.  There isn't
> that good of a solution to solve this, but easy_install itself, seems
> to "do the right thing", but appending the actual python version to
> the script name.
>
> I think the idea way is that one bootstrap could work across all
> version of python so that python versions wouldn't need to be appended
> to the script name.  Additionally, it could be useful, but I have no
> idea how this would work, to have the bootstrap dynamically pick the
> right version of itself that it needs to run in a given context.
>
>
>
>>
>>> 2.  The entry point mechanism IIRC recursively scans the site-packages
>>> directory and loads up the system path with eggs.  This is too
>>> expensive of an operation for the current environment I work in.
>>
>> The scan is not recursive; only files that are actually *on* sys.path are
>> scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info
>> in a directory that is directly on sys.path.
>
> I could be wrong, as this is from memory, but I believe that each egg
> inside of site-packages gets seperately injected in sys.path.  I
> suppose this is the only way to deal with package versions cleanly.  I
> have noticed that whole process is reasonably expensive.
>>
>> Now, it's possible that some application you are using does such a scan
>> explicitly; I'm just noting that merely querying or loading entry points
>> doesn't cause any recursive scans, and it most definitely does not add
>> anything new to sys.path, unless the entry point to be loaded has declared
>> an additional dependency that's *not* on sys.path yet.
>>
>>
>>> 3.  There doesn't seem to be a clean way to inject user specific
>>> environment details to the console script.
>>> I often need the ability to alter the sys.path in a user specific way
>>> for the entry point without needing to mess up the global sys.path
>>> permanently.
>>
>> I don't understand what you mean here, either.
>
> In film enviroments the whole way we work is by toggling between
> different enviroments.  A developer, artists, etc, will need to work
> on a specific shot in a movie, and they also need to toggle between
> films, etc.  Each of these enviroments is different because different
> code is being used. Keeping all possible combinations of python
> environments in your global environment can make the python interpeter
> grind to halt because of the way Python itself recursives over paths
> looking for files.  The idea scenario, at least for my current
> environment, is for sys.path to "inject" a custom path or N paths at
> the actual run time of the command line tool, because it limits the
> scanning Python needs to do.  Here is an example of my quick and dirty
> hack to make IPython run quicker:
>
> if __name__ == "__main__":
>    import sys
>    sys.path = [
>     '/vol/apps/python-2.5.1_64/lib/python2.5',
>     '/vol/apps/python-2.5.1_64/lib/python2.5/site-packages',
>     '/vol/apps/python-2.5.1_64/lib/python2.5/lib-dynload/',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-x86_64.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/lxml-2.1.2-py2.5-linux-x86_64.egg',
>     '/vol/filmstudio/linux64/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg',
>     ]
>
>    import IPython.Shell
>    IPython.Shell.start().mainloop()
>
>
> On one hand, IPython is now very snappy and runs in a useable fashion.
> The problem now, is that all flexibility is lost to dynamically add
> more things based on the users environment.  Perhaps by having the
> boostrap script look at some custom environmental variable to "inject"
> one more path, but just for that boostrap, in that given context, as
> we don't want to keep extra stuff around in the global PYTHONPATH and
> it is different for each user.
>
>>
>>
>
>
>
> --
> Cheers,
>
> Noah
>



-- 
Cheers,

Noah



On Fri, May 8, 2009 at 11:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
>>
>> 1. ?Different versions of Python conflict with previous versions of
>> console scripts. ?Take paste for example.
>
> I don't understand what you mean.
>
>> 2. ?The entry point mechanism IIRC recursively scans the site-packages
>> directory and loads up the system path with eggs. ?This is too
>> expensive of an operation for the current environment I work in.
>
> The scan is not recursive; only files that are actually *on* sys.path are
> scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info
> in a directory that is directly on sys.path.
>
> Now, it's possible that some application you are using does such a scan
> explicitly; I'm just noting that merely querying or loading entry points
> doesn't cause any recursive scans, and it most definitely does not add
> anything new to sys.path, unless the entry point to be loaded has declared
> an additional dependency that's *not* on sys.path yet.
>
>
>> 3. ?There doesn't seem to be a clean way to inject user specific
>> environment details to the console script.
>> I often need the ability to alter the sys.path in a user specific way
>> for the entry point without needing to mess up the global sys.path
>> permanently.
>
> I don't understand what you mean here, either.
>
>



-- 
Cheers,

Noah

From pje at telecommunity.com  Fri May  8 01:28:53 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 07 May 2009 19:28:53 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <B804F891-0E30-4844-9E70-A9E22A8DDD95@gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<B804F891-0E30-4844-9E70-A9E22A8DDD95@gmail.com>
Message-ID: <20090507232615.2E9763A4119@sparrow.telecommunity.com>

At 10:38 AM 5/7/2009 -0400, Doug Hellmann wrote:
>pip installs my scripts into a virtualenv without any issue and
>without using entry points, AFAICT.
>
>I guess if we move to requiring entry points and disallowing simple
>script distribution I'll need to find another way to package
>virtualenvwrapper.  Since it's a bash script, it doesn't have entry
>points.  I've been using setuptools to package it so it can be
>installed via easy_install, since it is a Python development tool.

Setuptools still supports "classic" scripts, and I don't see any 
reason to remove that support.  People do package non-Python scripts 
with their projects, after all.

easy_install basically examines such scripts to see if they end with 
.py, have a #! line with 'python' in it, or are valid Python source 
code.  If any of the above are true, it makes a Python script 
wrapper, otherwise it assumes the script is some other language and 
leaves it alone.


From doug.hellmann at gmail.com  Fri May  8 02:03:53 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 7 May 2009 20:03:53 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <20090507232615.2E9763A4119@sparrow.telecommunity.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<B804F891-0E30-4844-9E70-A9E22A8DDD95@gmail.com>
	<20090507232615.2E9763A4119@sparrow.telecommunity.com>
Message-ID: <7495E595-C1C9-4EB5-B4EF-22DCD57B106A@gmail.com>


On May 7, 2009, at 7:28 PM, P.J. Eby wrote:

> At 10:38 AM 5/7/2009 -0400, Doug Hellmann wrote:
>> pip installs my scripts into a virtualenv without any issue and
>> without using entry points, AFAICT.
>>
>> I guess if we move to requiring entry points and disallowing simple
>> script distribution I'll need to find another way to package
>> virtualenvwrapper.  Since it's a bash script, it doesn't have entry
>> points.  I've been using setuptools to package it so it can be
>> installed via easy_install, since it is a Python development tool.
>
> Setuptools still supports "classic" scripts, and I don't see any  
> reason to remove that support.  People do package non-Python scripts  
> with their projects, after all.
>
> easy_install basically examines such scripts to see if they end  
> with .py, have a #! line with 'python' in it, or are valid Python  
> source code.  If any of the above are true, it makes a Python script  
> wrapper, otherwise it assumes the script is some other language and  
> leaves it alone.

That would do what I want.  I only brought it up because another  
poster in this thread mentioned eliminating "scripts" an requiring  
console apps to use entry points.


From pje at telecommunity.com  Fri May  8 01:26:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 07 May 2009 19:26:51 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <d3bef572-8f04-4abc-b4b4-65f51ae9422d@d19g2000prh.googlegro
	ups.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<d3bef572-8f04-4abc-b4b4-65f51ae9422d@d19g2000prh.googlegroups.com>
Message-ID: <20090507232413.5C98B3A4080@sparrow.telecommunity.com>

At 01:14 PM 5/7/2009 -0700, Wheat wrote:
>But then having both 'scripts' and 'entry_points/console_scripts' is
>less than perfect since it introduces (mostly) unneccessary TIMTOWTDI.

I interpret 'scripts' as meaning "non-Python scripts", so there is 
still only one obvious way to do each thing.  ;-)


From greg.ewing at canterbury.ac.nz  Fri May  8 03:23:09 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 08 May 2009 13:23:09 +1200
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<79990c6b0905070703g28ae08bbw96286e7ef2b0f81a@mail.gmail.com>
Message-ID: <4A03897D.2080608@canterbury.ac.nz>

Paul Moore wrote:

> That works but a lot of Unix users have in the past objected to having
> '.py' in the name.

So install a symlink from hello -> hello.py.

-- 
Greg

From fress.eventos at gmail.com  Fri May  8 13:13:14 2009
From: fress.eventos at gmail.com (Mariana Costa)
Date: Fri, 08 May 2009 08:13:14 -0300
Subject: [Distutils] Organize e diminua riscos - NS
Message-ID: <E1M2O0w-0001YS-7W@mailt89.virtuaserver.com.br>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090508/1c08e49b/attachment.htm>

From chris at simplistix.co.uk  Sat May  9 16:21:10 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 09 May 2009 15:21:10 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <4A035AE5.3040203@trueblade.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	<4A02ECF7.80004@trueblade.com>	<gtuqo0$ir5$1@ger.gmane.org>	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
	<4A035AE5.3040203@trueblade.com>
Message-ID: <4A059156.6030401@simplistix.co.uk>

Eric Smith wrote:
> Paul Moore wrote:
>> 2009/5/7 Tres Seaver <tseaver at palladion.com>:
>>> Eric Smith wrote:
>>>> Yes. It creates a .exe wrapper [1]. By using entry points, I don't need
>>>> to care what the target system is. Also, /usr/bin/env might invoke the
>>>> wrong python.
>>> Exactly:  using entry points for console scripts guarantees that the
>>> python into which the corresponding distribution is installed is the one
>>> used to run the script, which is *highly* desirable.  Otherwise, you end
>>> up with the "just install everything in the system Python's
>>> site-packages" mess.
>>
>> ... and somewhere around here we end up with what I described as an
>> over-engineered solution.
> 
> Or, what I call a minimal set of required functionality.

+1.

<flame>Anyone who's arguing against this is either not deploying stuff 
in a repeatable fashion, and so isn't serious in my books, or is so 
serious that they're cutting vm images to roll out and so dump 
everything for the app in site packages</flame>

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Sat May  9 16:24:32 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 09 May 2009 15:24:32 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <e91cc0270905071649x5fa2c142yed25aa44b3839670@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	<4A02ECF7.80004@trueblade.com>
	<gtuqo0$ir5$1@ger.gmane.org>	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>	<4A035AE5.3040203@trueblade.com>	<e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>	<20090507232150.165F13A4080@sparrow.telecommunity.com>
	<e91cc0270905071649x5fa2c142yed25aa44b3839670@mail.gmail.com>
Message-ID: <4A059220.6090306@simplistix.co.uk>

Noah Gift wrote:
>>> I don't understand what you mean here, either.
>> In film enviroments the whole way we work is by toggling between
>> different enviroments.  A developer, artists, etc, will need to work
>> on a specific shot in a movie, and they also need to toggle between
>> films, etc.  

Have you guys looked at buildout?
I know the docs suck, but it strikes me that it might solve a lot of the 
problems you're complaining about...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Sat May  9 16:25:07 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 09 May 2009 15:25:07 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	<4A02ECF7.80004@trueblade.com>
	<gtuqo0$ir5$1@ger.gmane.org>	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>	<4A035AE5.3040203@trueblade.com>
	<e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>
Message-ID: <4A059243.5090002@simplistix.co.uk>

Noah Gift wrote:
> 3.  There doesn't seem to be a clean way to inject user specific
> environment details to the console script.

os.environ?

Either than, or I'm not following you...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From p.f.moore at gmail.com  Sat May  9 17:55:24 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 9 May 2009 16:55:24 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <4A059156.6030401@simplistix.co.uk>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>
	<4A035AE5.3040203@trueblade.com> <4A059156.6030401@simplistix.co.uk>
Message-ID: <79990c6b0905090855u567fe742q49915c44010153be@mail.gmail.com>

2009/5/9 Chris Withers <chris at simplistix.co.uk>:
> +1.
>
> <flame>Anyone who's arguing against this is either not deploying stuff in a
> repeatable fashion, and so isn't serious in my books, or is so serious that
> they're cutting vm images to roll out and so dump everything for the app in
> site packages</flame>

Hmm. I'll accept the flame to the extent that I don't deploy stuff to
a wide enough audience to be qualified to comment on that side of
things. But:

1. Can you clarify what "this" is? Part of the issue I see is that
there's never a clear enough statement of a proposal for a non-expert
in the field to follow.
2. Distutils is for distributing python modules/packages, so
*application* deployment is out of scope. Script support is for
small-scale stuff (in my view). The fact that it gets used for more
doesn't mean it's appropriate...
3. Accepting that I don't know what you mean by "this", can I point
out that as a user, I personally have problems with a significant
proportion of scripts distributed with packages - so are you saying
that those packages "aren't serious", or that there is no way of doing
what they (and I) want at present?

Concrete examples of specific packages would probably help. Right this
moment, I can't personally provide any because I recently had to
rebuild my PC and haven't yet reinstalled the vast number of
tried-once-but-never-used-again packages I used to have available...
I'll see if I can find some in due course. (Basically, the types of
things I see are scripts distributed on Windows with no filetype
extension, .bat wrappers for command line scripts which don't work
right when called from another bat file, setuptools-built exe wrappers
which result in version-specific binaries for pure-python code, to
give some examples).

Paul.

From tseaver at palladion.com  Sat May  9 20:01:29 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 09 May 2009 14:01:29 -0400
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <4A059220.6090306@simplistix.co.uk>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	<4A02ECF7.80004@trueblade.com>	<gtuqo0$ir5$1@ger.gmane.org>	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>	<4A035AE5.3040203@trueblade.com>	<e91cc0270905071521j5fdb5627y2a0b3ff34a9d216b@mail.gmail.com>	<20090507232150.165F13A4080@sparrow.telecommunity.com>	<e91cc0270905071649x5fa2c142yed25aa44b3839670@mail.gmail.com>
	<4A059220.6090306@simplistix.co.uk>
Message-ID: <gu4gdr$sfo$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Withers wrote:
> Noah Gift wrote:
>>>> I don't understand what you mean here, either.
>>> In film enviroments the whole way we work is by toggling between
>>> different enviroments.  A developer, artists, etc, will need to work
>>> on a specific shot in a movie, and they also need to toggle between
>>> films, etc.  
> 
> Have you guys looked at buildout?
> I know the docs suck, but it strikes me that it might solve a lot of the 
> problems you're complaining about...

The docs have gotten a lot better lately, thanks largely to the efforts
of Baiju M, who has done a lot of the gardening to get the project site
up and useful:

  http://www.buildout.org/


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKBcT5+gerLs4ltQ4RAljeAJ9dnfZRiKLtt+9YgRHdpA0V0FxHlwCfWO7V
5EQmUxX+t2DBGnG7aguXvnQ=
=dib9
-----END PGP SIGNATURE-----


From iwan at reahl.org  Mon May 11 13:53:34 2009
From: iwan at reahl.org (Iwan Vosloo)
Date: Mon, 11 May 2009 13:53:34 +0200
Subject: [Distutils] Possible bug in setuptools - test command?
Message-ID: <1242042814.7241.62.camel@easymoney>

Hi there,

We've been having some trouble with a script which invokes
setuptools.setup(script_args=['test']) more than once in a single
process.

During each invocation, python imports different, new instances of the
same module.

We've narrowed it down to setuptools/command/test.py, in the method
with_project_on_sys_path:

It saves sys.modules like this:

        old_modules = sys.modules.copy()

Then does its thing and restores sys.modules like this:

            sys.modules.clear()
            sys.modules.update(old_modules)

But, as the code below illustrates, this results in multiple imports of
the same modules:

import sys

def i():
    old_modules = sys.modules.copy()

    from decimal import Decimal
    
    sys.modules.clear()
    sys.modules.update(old_modules)

    return Decimal


assert i() is i(), 'Two different Decimal instances returned'  # Fails


Is this a bug?  (We are on version 0.6c8 of setuptools, python 2.5)

There is an easy solution:

Save sys.modules like before:
        old_modules = sys.modules.copy()

But restore it like this:
    sys.modules = old_modules

That seems to fix our problem (not sure of other effects though).

Regards
- Iwan Vosloo


From iwan at reahl.org  Mon May 11 18:11:43 2009
From: iwan at reahl.org (Iwan Vosloo)
Date: Mon, 11 May 2009 18:11:43 +0200
Subject: [Distutils] Possible bug in setuptools - test command?
In-Reply-To: <1242042814.7241.62.camel@easymoney>
References: <1242042814.7241.62.camel@easymoney>
Message-ID: <1242058303.7241.66.camel@easymoney>

On Mon, 2009-05-11 at 13:53 +0200, Iwan Vosloo wrote:
> During each invocation, python imports different, new instances of the
> same module.
...
> Is this a bug?  (We are on version 0.6c8 of setuptools, python 2.5)

Sorry, on second thought we realised this is obviously indended
behaviour of the test command, forcing new instances of modules to be
imported each time.

Somehow a particular package manages to get a hold of classes from an
earlier import though - that seems to be our problem, not setuptools
itself.

Regards
- Iwan


From ziade.tarek at gmail.com  Tue May 12 12:13:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 12 May 2009 12:13:44 +0200
Subject: [Distutils] Setuptools release (svn 1.6)
Message-ID: <94bdd2610905120313w1ec30f8csdd87f6a1ec08e027@mail.gmail.com>

Phillip,

What about blessing someone to review and commit the patch for svn 1.6
support (and maybe other patches)
and do a setuptools release ?

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org

From rupert.thurner at gmail.com  Tue May 12 13:08:42 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Tue, 12 May 2009 04:08:42 -0700 (PDT)
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
Message-ID: <842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>

On Apr 28, 7:39?am, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> On Tue, Apr 28, 2009 at 5:23 AM, David Cournapeau
>
> <da... at ar.media.kyoto-u.ac.jp> wrote:
> > Tarek Ziad? wrote:
>
> >> I am not sur to understand what you are trying to achieve.
>
> >> This indicates your environment variable CC was properly set and sent
> >> to the compiler.
>
> > I don't think it does: the user requires /opt/SUNWspro/bin/cc as a C
> > compiler, and easy_install uses /opt/studio/SOS11/SUNWspro/bin/cc which
> > are different compilers, unless one is a softlink to the other.
>
> Oops, correct, I didn't notice there were different.
>
> Rupert, can you try downloading that package, uncompress it and run :
>
> # CC=/opt/SUNWspro/bin/cc python setup.py install
>
> If this fails too, there's a problem in Distutils.sysconfig.customize_compiler
> we need to look at. (basically, it looks at os.environ if your
> compiler type is "unix"
> and not mingw or bcpp, or msvc etc..)
>
> If it works, the problem is on easy_install itself.
>

you are right! it works, nearly, but this might be a reportlab
problem:

# CC=/cs/SUNWspro/bin/cc python setup.py install
################################################
#Attempting install of _rl_accel, sgmlop & pyHnj
#extensions from '/tmp/ReportLab_2_3/src/rl_addons/rl_accel'
################################################
################################################
#Attempting install of _renderPM
#extensions from '/tmp/ReportLab_2_3/src/rl_addons/renderPM'
# installing with freetype version 21
################################################
running install
running build
running build_py
copying src/reportlab/lib/hyphen.mashed -> build/lib.solaris-2.10-
sun4v-2.6/reportlab/lib
running build_clib
building '_renderPM_libart' library
/cs/SUNWspro/bin/cc -DNDEBUG -O -xO3 -xarch=v8 -DLIBART_COMPILATION -
DWORDS_BIGENDIAN -I/tmp/ReportLab_2_3/src/rl_addons/renderPM -I/tmp/
ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl -c /tmp/ReportLab_2_3/
src/rl_addons/renderPM/libart_lgpl/art_uta.c -o build/
temp.solaris-2.10-sun4v-2.6/tmp/ReportLab_2_3/src/rl_addons/renderPM/
libart_lgpl/art_uta.o
...
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
line 29: warning: invalid white space character in directive
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
line 31: warning: invalid white space character in directive
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
line 33: warning: invalid white space character in directive
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
line 38: warning: invalid white space character in directive
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
line 39: syntax error before or at:
...
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
line 81: type specifier can not be used as array size expression
qualifier
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
line 82: warning: no explicit type given
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
line 83: warning: old-style declaration or incorrect type for:
art_uta_free
"/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
line 88: cannot recover from previous errors
cc: acomp failed for /tmp/ReportLab_2_3/src/rl_addons/renderPM/
libart_lgpl/art_uta.c
error: command '/opt/SUNWspro/bin/cc' failed with exit status 2




From tl at gocept.com  Tue May 12 16:53:23 2009
From: tl at gocept.com (Thomas Lotze)
Date: Tue, 12 May 2009 16:53:23 +0200
Subject: [Distutils] Buildout: adding a download API?
Message-ID: <20090512165323.02f05bc8@krusty.ws.whq.gocept.com>

Buildout as well as a number of buildout recipes need to download files
from the net. They need to take care of doing the actual download,
using the download cache and honouring the offline option. A couple of
years ago, we also wrote the gocept.download recipe that does all that
as well as MD5 checks and some other stuff.

In order to simplify this situatin, we propose adding a download API to
buildout itself. A function like this:

def download(url, use_cache=True, md5sum=None)
    ...
    return filename

might be sufficient to remove the download logic from recipes and make
optional checksum testing available without needing to depend on a
separate download recipe.

As a consequence, zc.recipe.cmmi would be able to do MD5 checks, which
would basically make gocept.cmmi redundant. Using gocept.download's
capabilities is the one feature of it which still currently keeps us
from dropping it in favour of zc.recipe.cmmi.

If there's no objection against a download API being added to buildout,
we'd like to implement one soon.

Otherwise, we'd propose implementing MD5 testing in zc.recipe.cmmi
since we consider it a good thing to reduce the zoo of cmmi-related
recipes. We do think that a reusable API would be the better choice,
though.

Viele Gr??e,
Thomas

-- 
Thomas Lotze ? tl at gocept.com
gocept gmbh & co. kg ? forsterstra?e 29 ? 06112 halle (saale) ? germany
http://gocept.com ? tel +49 345 1229889 0 ? fax +49 345 1229889 1
Zope and Plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090512/8ce42d35/attachment.pgp>

From strawman at astraw.com  Tue May 12 18:10:47 2009
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 12 May 2009 09:10:47 -0700
Subject: [Distutils] Setuptools release (svn 1.6)
In-Reply-To: <94bdd2610905120313w1ec30f8csdd87f6a1ec08e027@mail.gmail.com>
References: <94bdd2610905120313w1ec30f8csdd87f6a1ec08e027@mail.gmail.com>
Message-ID: <4A099F87.5090704@astraw.com>

Tarek Ziad? wrote:
> Phillip,
>
> What about blessing someone to review and commit the patch for svn 1.6
> support (and maybe other patches)
> and do a setuptools release ?
>
> Regards
> Tarek
>   
What about the idea that the next release of setuptools move svn support 
to a setuptools_svn plugin like the other VCSs? That way, only the 
plugin needs to be updated with new svn releases, not the core setuptools.

-Andrew


From jim at zope.com  Tue May 12 19:36:31 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 12 May 2009 13:36:31 -0400
Subject: [Distutils] Buildout: adding a download API?
In-Reply-To: <20090512165323.02f05bc8@krusty.ws.whq.gocept.com>
References: <20090512165323.02f05bc8@krusty.ws.whq.gocept.com>
Message-ID: <943EBD46-6D2D-40E4-97F2-CC18E3FD9B9C@zope.com>


On May 12, 2009, at 10:53 AM, Thomas Lotze wrote:

> Buildout as well as a number of buildout recipes need to download  
> files
> from the net. They need to take care of doing the actual download,
> using the download cache and honouring the offline option. A couple of
> years ago, we also wrote the gocept.download recipe that does all that
> as well as MD5 checks and some other stuff.
>
> In order to simplify this situatin, we propose adding a download API  
> to
> buildout itself. A function like this:
>
> def download(url, use_cache=True, md5sum=None)
>    ...
>    return filename
>
> might be sufficient to remove the download logic from recipes and make
> optional checksum testing available without needing to depend on a
> separate download recipe.
>
> As a consequence, zc.recipe.cmmi would be able to do MD5 checks, which
> would basically make gocept.cmmi redundant. Using gocept.download's
> capabilities is the one feature of it which still currently keeps us
> from dropping it in favour of zc.recipe.cmmi.
>
> If there's no objection against a download API being added to  
> buildout,
> we'd like to implement one soon.
>
> Otherwise, we'd propose implementing MD5 testing in zc.recipe.cmmi
> since we consider it a good thing to reduce the zoo of cmmi-related
> recipes. We do think that a reusable API would be the better choice,
> though.


+1

Thanks!

You should add a "namespace" option to the api.  Right now we have at  
least 2 namespaces,
dist and cmmi. (My download cache has a "minitage" directory. I wonder  
where that came from. :)

Jim

--
Jim Fulton
Zope Corporation



From david.lyon at preisshare.net  Wed May 13 03:07:16 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 May 2009 21:07:16 -0400
Subject: [Distutils] Distutils roadmap : big picture..
In-Reply-To: <94bdd2610904300046s29fd70c9s6ec84639190cf695@mail.gmail.com>
References: <94bdd2610904300046s29fd70c9s6ec84639190cf695@mail.gmail.com>
Message-ID: <397de93ae90d13ef5bbcfdc9fcdc609d@preisshare.net>


Hi All,

I must admit that my perspective of distutils is from a system
administrators perspective. Not a package developer.

Maybe I write code sometimes that I want to ship out.. like a
package mangager.. haha.. anyway..

My general comments for distutils are:

 - introduce gui tools to help developers "build"

 - introduce gui tools to help users "manage"

 - improve support of distutils on the windows platform
   so that it looks like it was developed after the
   game space invaders and not before.

 - talk to other "stars" in the python universe ie py2exe
   wxpython, pip.. installers.. ie NSIS etc etc..

 - improve package consistency

 - write more tests

After spending a few months on this list I now recognise
that many people have dedicated an enormous amount of time
effort, skill and commitment.

In summary I'm suggesting that we breath some new life
into distutils...


David

ps: my pet hate is windows installers for packages. I want
these to go away. I don't like running executables to install
a package. 




From ziade.tarek at gmail.com  Wed May 13 14:34:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 13 May 2009 14:34:49 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
Message-ID: <94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>

Any feedback on this "dev version of post release" use case or I move
forward and integrate it ?


On Thu, May 7, 2009 at 10:02 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Wed, May 6, 2009 at 10:11 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> On Wed, May 6, 2009 at 9:20 PM, P.J. Eby <pje at telecommunity.com> wrote:
>>> At 08:58 PM 5/6/2009 +0200, Tarek Ziad? wrote:
>>>>
>>>> Sorry to post again that mail, but it seems that this is the last
>>>> remaining
>>>> issue for the versionning proposal.
>>>>
>>>> Phillip, could you check my proposal for your development versions of
>>>> postreleases
>>>> use case ?
>>>
>>> Is the code up-to-date with the below?
>>
>> No, I'll do a branch then with that version
>
> done (I have also fixed minor bugs in that branch
>
> http://bitbucket.org/tarek/distutilsversion/src/3fe1afab2e1a/
>
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
>



-- 
Tarek Ziad? | http://ziade.org

From me at rpatterson.net  Wed May 13 23:36:30 2009
From: me at rpatterson.net (Ross Patterson)
Date: Wed, 13 May 2009 14:36:30 -0700
Subject: [Distutils] Eggs, zc.buildout, and Distributed Version Control,
	oh my!
Message-ID: <87d4acfzxd.fsf@transitory.lefae.org>

...and of course the elephant in the living room, svn:externals.  :)

I've been using DVC (git mostly) more and more these days and enjoying
it very much.  It works particularly well when I have a self-contained
software package.  I tend to start a project in SVN with the standard
trunk/branches/tags layout and then use git in front of the SVN repo.  I
make the egg root itself a buildout for a test environment with my
favorite but not required development tools.  (Of course I test with
"python setup.py test" before release when the egg doesn't require any
additional deployment details provided by the buildout.  I don't want to
get into that discussion.)  It all works much better than trying to work
directly with SVN far beyond the benefits of just being able to work
offline.

What I want to ask about and still haven't worked out an answer to is
how to best manage framework or deployment projects where I am
developing a number of eggs in parallel under a buildout that needs to
be versioned as well.  In this case, I'm finding moving away from
svn:externals more painful than I expected.  Having an overview of all
version control status for all code being developed is very valuable.

Now some of this comes down to eggs being overused and I've certainly
been guilty of that myself.  For example, I used to make separate eggs
for all my client specific code (policy, theme, etc.).  This is clearly
unnecessary and I've switched to using one egg that contians multiple
sub-packages for factoring.

Even with that, however, I still find the lack of a good "what changes
have I made to all sub projects" overview to be rather painful.  Maybe
this is outside the scope of the individual DVC tool and should be
handled by an independent tool that aggregates status across projects.
How do others handle this kind of situation?

Ross


From hannosch at hannosch.eu  Thu May 14 00:13:25 2009
From: hannosch at hannosch.eu (Hanno Schlichting)
Date: Thu, 14 May 2009 00:13:25 +0200
Subject: [Distutils] Eggs, zc.buildout, and Distributed Version Control,
	oh my!
In-Reply-To: <87d4acfzxd.fsf@transitory.lefae.org>
References: <87d4acfzxd.fsf@transitory.lefae.org>
Message-ID: <gufgm5$hll$1@ger.gmane.org>

Ross Patterson wrote:
> ....and of course the elephant in the living room, svn:externals.  :)

> Even with that, however, I still find the lack of a good "what changes
> have I made to all sub projects" overview to be rather painful.  Maybe
> this is outside the scope of the individual DVC tool and should be
> handled by an independent tool that aggregates status across projects.
> How do others handle this kind of situation?

Both the official Plone development and we at Jarn internally moved to
http://pypi.python.org/pypi/mr.developer for this use-case.

It has a among others a VCS agnostic "./bin/develop status" command and
gets rid of svn:externals in favor of managing them inside the buildout
configuration.

Hanno


From me at rpatterson.net  Thu May 14 04:13:06 2009
From: me at rpatterson.net (Ross Patterson)
Date: Wed, 13 May 2009 19:13:06 -0700
Subject: [Distutils] Eggs, zc.buildout, and Distributed Version Control,
	oh my!
References: <87d4acfzxd.fsf@transitory.lefae.org> <gufgm5$hll$1@ger.gmane.org>
Message-ID: <87ljp0e8jx.fsf@transitory.lefae.org>

Hanno Schlichting <hannosch at hannosch.eu> writes:

> Ross Patterson wrote:
>> ....and of course the elephant in the living room, svn:externals.  :)
>
>> Even with that, however, I still find the lack of a good "what changes
>> have I made to all sub projects" overview to be rather painful.  Maybe
>> this is outside the scope of the individual DVC tool and should be
>> handled by an independent tool that aggregates status across projects.
>> How do others handle this kind of situation?
>
> Both the official Plone development and we at Jarn internally moved to
> http://pypi.python.org/pypi/mr.developer for this use-case.
>
> It has a among others a VCS agnostic "./bin/develop status" command and
> gets rid of svn:externals in favor of managing them inside the buildout
> configuration.

Awesome!  I'd read the PyPI page when I saw mr.developer in the RSS feed
but it's page doesn't mention it will do status so I overlooked it.
Thanks.

Any other options out there?

Ross


From pje at telecommunity.com  Thu May 14 04:08:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 13 May 2009 22:08:18 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.co
 m>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
Message-ID: <20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>

At 02:34 PM 5/13/2009 +0200, Tarek Ziad? wrote:
>Any feedback on this "dev version of post release" use case

It looks like the 'suggest' function doesn't handle svn revisions 
properly for conversion from setuptools versions.  Setuptools '-r###' 
is a post-release tag, as is any alpha string alphabetically after 
'final'.  That is, '0.4a1.r10' is actually '0.4a1.post10'.  (Notice, 
for example, that setuptools' trunk version is '0.7a1dev-r66608'.)

See 
http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version 
for the full syntax; I'm a bit concerned that there may be other 
incorrect interpretations taking place in this function.  (Notice, 
for example, that a '-' in a version number indicates a post-release, 
but your code is treating '-' as identical to '.'.)


>or I move
>forward and integrate it ?

Into the stdlib? I'm a bit surprised that something with such global 
consequences is having its bleeding edge development done in the 
stdlib, rather than as an external package.

I sort of assumed at first the idea was to provide a normalized or 
canonical form for setuptools versions, but of course it's a bit more 
restrictive than that, and at the moment the only mapping between the 
two seems broken -- and in a way that suggests the designers neither 
read the setuptools version documentation nor examined what people 
were actually doing with their versioning schemes.

To be clear, I'm not opposed to blessing a normalized version scheme 
-- there are solid benefits to doing such a thing.  I'm opposed to 
putting one in the stdlib without a clear rationale for the choices, 
and an upgrade path that either lets setuptools users ignore the new 
scheme, or gives them some benefit for switching.

Meanwhile, I haven't seen any indication that this new format is 
easier for non-Python tools to parse.  Is the plan for other tools to 
use the stdlib parsing code, because I don't see how existing tools 
like RPM et al could use the proposed scheme.


From randall at tnr.cc  Thu May 14 05:56:50 2009
From: randall at tnr.cc (Randall Smith)
Date: Wed, 13 May 2009 22:56:50 -0500
Subject: [Distutils] including a third party egg without "installing" it
Message-ID: <gug4q5$r9v$1@ger.gmane.org>

I'd like to bundle some third party eggs with my application without 
installing them.  Typically, I'd read __file__ and alter sys.path at 
runtime to include the third party packages, but I understand that will 
not work with zipped eggs.

I don't know the best way to go about this, but I at least want to know 
a clean way to do it without requiring the third party eggs to be 
downloaded and installed separately.  Consider this layout.

theapp/
   docs/
   setup.py
   theapp/
     __init__.py
     app.py
     ext/
       thirdparty1.egg
       thirdparty2.egg
       thirdparty3.egg
   tests/

 From my current understanding about how setuptools works, I think I 
could treat the third party eggs as data files and and runtime 
(somehow?) insert them into the path and import them.

I'm trying to work from this example.

http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources

Is this a workable approach?  Is there a better way?

--Randall


From ziade.tarek at gmail.com  Thu May 14 10:19:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 10:19:11 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
	<20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905140119i4eca75d2wc9800ec93e8b4620@mail.gmail.com>

2009/5/14 P.J. Eby <pje at telecommunity.com>:
> At 02:34 PM 5/13/2009 +0200, Tarek Ziad? wrote:
>>
>> Any feedback on this "dev version of post release" use case
>
> It looks like the 'suggest' function doesn't handle svn revisions properly

Ok I'll check that

> To be clear, I'm not opposed to blessing a normalized version scheme --
> there are solid benefits to doing such a thing. ?I'm opposed to putting one
> in the stdlib without a clear rationale for the choices, and an upgrade path
> that either lets setuptools users ignore the new scheme, or gives them some
> benefit for switching.

PEP 345 is being rewritten with a rationale specification of versions
in mind, and the plan
is to refer to this version scheme and document it there.

see http://svn.python.org/view/peps/branches/jim-update-345/pep-0345.txt?view=markup

from a setuptools user point of view, the benefit I can see is that
they will have better
version numbers that will comply with a documented standard, that is
usable and understandable
by os packagers for example.

Now I suppose setuptools will have to propose both for some time, but
in Distutils,
no one really uses LooseVersion and StricVersion, so the idea is to
deprecated them and introduce
the new one in there.

But what I am scared of is : who will work on setuptools side ? can
you bless someone to do the
work when we agree on a common roadmap ?

>
> Meanwhile, I haven't seen any indication that this new format is easier for
> non-Python tools to parse. ?Is the plan for other tools to use the stdlib
> parsing code, because I don't see how existing tools like RPM et al could
> use the proposed scheme.

Some people from Fedora and Ubuntu worked on this (I've CC them) , and
the whole idea was to provide
a scheme that would make rpm and debian packagers happy with it.

I am not a RPM expert at all, but the output is supposed to be
RPM-friendly. Toshio ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu May 14 10:59:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 10:59:22 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
	<20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
Message-ID: <94bdd2610905140159t25e1a686k55e3fcb720cde84f@mail.gmail.com>

2009/5/14 P.J. Eby <pje at telecommunity.com>:
> At 02:34 PM 5/13/2009 +0200, Tarek Ziad? wrote:
>>
>> Any feedback on this "dev version of post release" use case
>
> It looks like the 'suggest' function doesn't handle svn revisions properly
> for conversion from setuptools versions. ?Setuptools '-r###' is a
> post-release tag, as is any alpha string alphabetically after 'final'. ?That
> is, '0.4a1.r10' is actually '0.4a1.post10'. ?(Notice, for example, that
> setuptools' trunk version is '0.7a1dev-r66608'.)
>
> See
> http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version
> for the full syntax; I'm a bit concerned that there may be other incorrect
> interpretations taking place in this function. ?(Notice, for example, that a
> '-' in a version number indicates a post-release, but your code is treating
> '-' as identical to '.'.)

Ok, I have changed the suggest function accordingly,


    def test_suggest_rational_version(self):

        self.assertEquals(suggest('1.0'), '1.0')
        self.assertEquals(suggest('1.0-alpha1'), '1.0a1')
        self.assertEquals(suggest('1.0rc2'), '1.0c2')
        self.assertEquals(suggest('walla walla washington'), None)

        # from setuptools
        self.assertEquals(suggest('0.4a1.r10'), '0.4a1.post10')
        self.assertEquals(suggest('0.7a1dev-r66608'), '0.7a1.dev66608')
        self.assertEquals(suggest('0.6a9.dev-r41475'), '0.6a9.dev41475')
        self.assertEquals(suggest('2.4preview1'), '2.4c1')
        self.assertEquals(suggest('2.4rc1'), '2.4c1')
        self.assertEquals(suggest('2.4pre1') , '2.4c1')
        self.assertEquals(suggest('2.1-rc2'), None)         # no suggestion


Tarek
-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu May 14 12:03:21 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 12:03:21 +0200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
Message-ID: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>

Hello

for PEP 376, I have one last fuzzy point.

http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup

The "get_egg_info" api is currently based on scanning the whole
sys.path. And since sys.path can be modified by people,
so the algorithm is linear and can slow down when there are a lot of paths.

I have a proposal: let's restrict the search for this API to
site-package directories only. (directories added with
site.addsitedir)

People will be able to mark add any directory (like the per-user
site-package directory - http://www.python.org/dev/peps/pep-0370)

This requires to add in site.py a registry to keep track of all
directories added through site.addsitedir

Any thoughts ?

(In the meantime, I'll propose the inclusion of a sitedir list in
site, in Python-ideas)

++
Tarek
-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu May 14 14:38:33 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 14:38:33 +0200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
Message-ID: <94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>

On Thu, May 14, 2009 at 12:33 PM, Noah Gift <noah.gift at gmail.com> wrote:
> One thing that isn't clear to me is whether this could then lead to a
> potentially new third party packaging system, or library, that will just do
> the same thing again, and create a linear scanning algorithm. ?Is there a
> way to architect things in a way that will never increase the number of
> stats to the filesystem or limit them?

I'd like to make a difference between what is "importable" and what is
"installed"
in Python.

what's "installed" for me is what is in a site-packages directory that
contains egg-info
directories.

At startup, you have to visit at least once every site-packages
directory, that's linear
and that won't go away.  But it's OK eventually, with a cache or a
memory index that
takes care of making the lookup code non-linear after Python has started
*as long as sys.path is not involved*

So, by restricting this API to site-packages (e.g. "installed"),
rather than sys.path (eg "importable"),
is just a way of limiting the scope to some specific places where you
are supposed to install
packages (and where egg-info files are located).

When a program is launched, the site-packages places should be limited to:

- the python site-packages (wether is central, wether it's local,
using virtualenv)
- the per user site-packages (PEP 370)
- maybe a third place added manually using site.addsitedir()


> One other thing that hopefully isn't off topic is the idea of scanning for
> imports in general. ?Does it at all seem logical to have an option for
> python to have zero scanning to load a module? ?After all if you really know
> the path to a module, why do you need to scan anything first to find it?

well if it's not loaded already, you have to find it don't you ? And
you don't know
where it is, you have to locate it  (see imporlib and pep 302)

But for me that's another story, than locating the metadata info of an installed
package.

Tarek

From setuptools at bugs.python.org  Thu May 14 15:37:06 2009
From: setuptools at bugs.python.org (mkesper)
Date: Thu, 14 May 2009 13:37:06 +0000
Subject: [Distutils] [issue69] No 2.6 install setup
In-Reply-To: <1242308226.35.0.0176269422822.issue69@psf.upfronthosting.co.za>
Message-ID: <1242308226.35.0.0176269422822.issue69@psf.upfronthosting.co.za>


New submission from mkesper <mkesper at schokokeks.org>:

If you want to install 0.6c9 on windows, you face a hen-and-egg-Problem:
An .eeg exists, but no installer.

----------
messages: 287
nosy: mkesper
priority: feature
status: unread
title: No 2.6 install setup

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue69>
_______________________________________________

From p.f.moore at gmail.com  Thu May 14 15:38:07 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 May 2009 14:38:07 +0100
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
Message-ID: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>

2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
> I'd like to make a difference between what is "importable" and what is
> "installed" in Python.
>
> what's "installed" for me is what is in a site-packages directory that
> contains egg-info directories.

I'm not sure how well-defined that is. Can you precisely define what a
"site-packages directory" is? Specifically, what about the Windows
registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
There may be other "special" locations, I can't recall for certain.

Paul.

From p.f.moore at gmail.com  Thu May 14 15:38:07 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 May 2009 14:38:07 +0100
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
Message-ID: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>

2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
> I'd like to make a difference between what is "importable" and what is
> "installed" in Python.
>
> what's "installed" for me is what is in a site-packages directory that
> contains egg-info directories.

I'm not sure how well-defined that is. Can you precisely define what a
"site-packages directory" is? Specifically, what about the Windows
registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
There may be other "special" locations, I can't recall for certain.

Paul.

From david.lyon at preisshare.net  Thu May 14 16:34:13 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 14 May 2009 10:34:13 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
Message-ID: <91619979339c5dddeba9940e28f24d03@preisshare.net>

On Thu, 14 May 2009 12:03:21 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:

> I have a proposal: let's restrict the search for this API to
> site-package directories only. (directories added with
> site.addsitedir)
> 
> People will be able to mark add any directory (like the per-user
> site-package directory - http://www.python.org/dev/peps/pep-0370)
> 

To me... it sounds like a good idea... but needs a little refining.

Let me clarify the three type of packages in any python system:

 1) Project Packages (needed by projects - ie subdirectories/eggs)

 2) User Packages ( introduced python 2.6 windows version)

 3) Site-Packages ( /lib/site-packages)

Anything else.... is really lost or just misplaced.

If we think of all packages in these three divisions, it will give
us more clarity in providing what users are inevitably after.

> This requires to add in site.py a registry to keep track of all
> directories added through site.addsitedir

There's no need for this... given that all the mechanisms for tracking
packages already exist. But only knights of the old code know where
it all is and how it works....

David




From david.lyon at preisshare.net  Thu May 14 16:41:23 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 14 May 2009 10:41:23 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
Message-ID: <5c3493d2869ce41a8c8758b8e267e949@preisshare.net>

On Thu, 14 May 2009 14:38:07 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
> 
> I'm not sure how well-defined that is. Can you precisely define what a
> "site-packages directory" is? 

Site-packages is referenced from the "original" site.py code.

It is a place from my reading where all packages common for the python
installation are stored. It is the site repository for all extra packages.

It's specifically hardcoded with an 
os.path.join(python-path,"lib","site-packages") command.

Specifically, what about the Windows
> registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
> There may be other "special" locations, I can't recall for certain.

There's a user-packages location under python 2.6

But it's use hasn't been picked up. If it was under linux then the
equivalent would possibly be ~/.user-packages. 


David


From david.lyon at preisshare.net  Thu May 14 16:54:30 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 14 May 2009 10:54:30 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
Message-ID: <92f4fc6b3b3d319e27cb68b0cbdef817@preisshare.net>

On Thu, 14 May 2009 14:38:33 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> When a program is launched, the site-packages places should be limited
to:
> 
> - the python site-packages (wether is central, wether it's local,
> using virtualenv)
> - the per user site-packages (PEP 370)
> - maybe a third place added manually using site.addsitedir()

Yes...

To make it even simpler again....

 1) project-packages (the current directory)
 
 2) user-packages

 3) site-packages

This would make things simpler and a lot faster.

btw, this is how I am building package installation into
my package manager gui.

Regards

David






From p.f.moore at gmail.com  Thu May 14 17:01:55 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 May 2009 16:01:55 +0100
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <5c3493d2869ce41a8c8758b8e267e949@preisshare.net>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<5c3493d2869ce41a8c8758b8e267e949@preisshare.net>
Message-ID: <79990c6b0905140801i41927679ta05c71855b3d1598@mail.gmail.com>

2009/5/14 David Lyon <david.lyon at preisshare.net>:
> On Thu, 14 May 2009 14:38:07 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
>>
>> I'm not sure how well-defined that is. Can you precisely define what a
>> "site-packages directory" is?
>
> Site-packages is referenced from the "original" site.py code.
>
> It is a place from my reading where all packages common for the python
> installation are stored. It is the site repository for all extra packages.
>
> It's specifically hardcoded with an
> os.path.join(python-path,"lib","site-packages") command.
>
> Specifically, what about the Windows
>> registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
>> There may be other "special" locations, I can't recall for certain.
>
> There's a user-packages location under python 2.6
>
> But it's use hasn't been picked up. If it was under linux then the
> equivalent would possibly be ~/.user-packages.

No, the registry stuff is completely different - it has been in
Windows Python for a long time (probably back as far as 1.5 or
earlier). It's not often used, but it's in addition to PYTHONPATH. I
think older versions of pywin32 used it but newer versions don't seem
to. I suspect it's not used much - but it's bound to be used in some
installations...

The main point here is to emphasize that there are some fairly obscure
ways for directories to end up in sys.path, even if we exclude user
code. The proposal needs to make a statement on how all such cases are
handled - I don't have any particular opinion on *what* should happen,
just that it gets documented.

Paul.

From david.lyon at preisshare.net  Thu May 14 17:10:30 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 14 May 2009 11:10:30 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <79990c6b0905140801i41927679ta05c71855b3d1598@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<5c3493d2869ce41a8c8758b8e267e949@preisshare.net>
	<79990c6b0905140801i41927679ta05c71855b3d1598@mail.gmail.com>
Message-ID: <daf7e147afe44a6ba5c04ec0585332ef@preisshare.net>

On Thu, 14 May 2009 16:01:55 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
 
> The main point here is to emphasize that there are some fairly obscure
> ways for directories to end up in sys.path, even if we exclude user
> code. The proposal needs to make a statement on how all such cases are
> handled - I don't have any particular opinion on *what* should happen,
> just that it gets documented.

My take on it is that sys.path get's messed with too easily.

Any item added in sys.path causes a full directory search to happen
which is fairly time consuming.

The best solution in performance is one where sys.path is limited to
a handful of entries (say ./, user-packages, site-packages) and
any packages that are stored are referenced in .PTH files along those
three paths.

Of course, I'm being way too simplistic here... and in the real
world many systems aren't like that.... yet.... lol

but who knows what the future can bring....

David


From pje at telecommunity.com  Thu May 14 18:07:45 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 12:07:45 -0400
Subject: [Distutils] including a third party egg without "installing" it
In-Reply-To: <gug4q5$r9v$1@ger.gmane.org>
References: <gug4q5$r9v$1@ger.gmane.org>
Message-ID: <20090514160507.A66843A4061@sparrow.telecommunity.com>

At 10:56 PM 5/13/2009 -0500, Randall Smith wrote:
>I'd like to bundle some third party eggs with my application without 
>installing them.  Typically, I'd read __file__ and alter sys.path at 
>runtime to include the third party packages, but I understand that 
>will not work with zipped eggs.
>
>I don't know the best way to go about this, but I at least want to 
>know a clean way to do it without requiring the third party eggs to 
>be downloaded and installed separately.  Consider this layout.
>
>theapp/
>   docs/
>   setup.py
>   theapp/
>     __init__.py
>     app.py
>     ext/
>       thirdparty1.egg
>       thirdparty2.egg
>       thirdparty3.egg
>   tests/
>
> From my current understanding about how setuptools works, I think I 
> could treat the third party eggs as data files and and runtime 
> (somehow?) insert them into the path and import them.
>
>I'm trying to work from this example.
>
>http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources
>
>Is this a workable approach?

No.  You need the eggs to be unzipped, and in the *root* of your 
distributed egg, alongside your package. i.e.:

theapp/
    ...
tp1.egg/
    ...contents here...
tp1.egg/
    ...contents here...

You can then simply use require() at runtime to access the bundled eggs.


>   Is there a better way?

Yes.  Simply install your entire application as eggs and scripts in a 
single directory (using "easy_install -maxd somedir myapp") and then 
tarball and ship "somedir".  Less muss, less fuss, and no 
require().  (I.e., just declare your dependencies in setup.py)

If you want to, you can even ship without needing setuptools 
installed on the target machine - just copy the pkg_resources module 
to "somedir" before you tarball it up.


From pje at telecommunity.com  Thu May 14 18:11:47 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 12:11:47 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905140159t25e1a686k55e3fcb720cde84f@mail.gmail.co
 m>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
	<20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
	<94bdd2610905140159t25e1a686k55e3fcb720cde84f@mail.gmail.com>
Message-ID: <20090514160906.13F993A4061@sparrow.telecommunity.com>

At 10:59 AM 5/14/2009 +0200, Tarek Ziad? wrote:
>2009/5/14 P.J. Eby <pje at telecommunity.com>:
> > At 02:34 PM 5/13/2009 +0200, Tarek Ziad? wrote:
> >>
> >> Any feedback on this "dev version of post release" use case
> >
> > It looks like the 'suggest' function doesn't handle svn revisions properly
> > for conversion from setuptools versions.  Setuptools '-r###' is a
> > post-release tag, as is any alpha string alphabetically after 
> 'final'.  That
> > is, '0.4a1.r10' is actually '0.4a1.post10'.  (Notice, for example, that
> > setuptools' trunk version is '0.7a1dev-r66608'.)
> >
> > See
> > 
> http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version
> > for the full syntax; I'm a bit concerned that there may be other incorrect
> > interpretations taking place in this function.  (Notice, for 
> example, that a
> > '-' in a version number indicates a post-release, but your code is treating
> > '-' as identical to '.'.)
>
>Ok, I have changed the suggest function accordingly,
>
>
>     def test_suggest_rational_version(self):
>
>         self.assertEquals(suggest('1.0'), '1.0')
>         self.assertEquals(suggest('1.0-alpha1'), '1.0a1')
>         self.assertEquals(suggest('1.0rc2'), '1.0c2')
>         self.assertEquals(suggest('walla walla washington'), None)
>
>         # from setuptools
>         self.assertEquals(suggest('0.4a1.r10'), '0.4a1.post10')
>         self.assertEquals(suggest('0.7a1dev-r66608'), '0.7a1.dev66608')
>         self.assertEquals(suggest('0.6a9.dev-r41475'), '0.6a9.dev41475')
>         self.assertEquals(suggest('2.4preview1'), '2.4c1')
>         self.assertEquals(suggest('2.4rc1'), '2.4c1')
>         self.assertEquals(suggest('2.4pre1') , '2.4c1')

All looking pretty good, except:

>         self.assertEquals(suggest('2.1-rc2'), None)         # no suggestion

This should be equivalent to '2.1c2':

 >>> from pkg_resources import parse_version as pv
 >>> pv('2.1-rc2')==pv('2.1c2')
True


From pje at telecommunity.com  Thu May 14 18:22:30 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 12:22:30 -0400
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <94bdd2610905140119i4eca75d2wc9800ec93e8b4620@mail.gmail.co
 m>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<20090422142628.C05483A4070@sparrow.telecommunity.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
	<20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
	<94bdd2610905140119i4eca75d2wc9800ec93e8b4620@mail.gmail.com>
Message-ID: <20090514161949.19E363A4061@sparrow.telecommunity.com>

At 10:19 AM 5/14/2009 +0200, Tarek Ziad? wrote:
>from a setuptools user point of view, the benefit I can see is that
>they will have better
>version numbers

Better how?  That was my question.  I personally find the common 
version patterns in use (e.g.  '-' and '-r' for post-releases) less 
clunky and easier to read than 'post'.  I also don't care for '.' in 
front of dev and post.  So, I don't see the changes as all "better" 
from a readability POV.

That's not to say it's a bad thing, if it gives other benefits.  But 
it has not been stated what the benefits are supposed to be.


>that will comply with a documented standard, that is

This isn't actually a change; setuptools also has a documented standard.


>usable and understandable by os packagers for example.

I agree that a canonical form is useful.  However, if that canonical 
form can be generated from their existing version numbers, then why 
should they bother changing?

That's the open question, IOW.


>Now I suppose setuptools will have to propose both for some time,

I don't see why.  AFAICT, RationalVersion is a strict subset of the 
syntax supported by setuptools, and most setuptools versions in use 
should be convertible.



>But what I am scared of is : who will work on setuptools side ? can
>you bless someone to do the
>work when we agree on a common roadmap ?

I don't see that there is any work *to* do in setuptools core, since 
RationalVersion is a strict subset, and we have setup-argument 
validation providers already.  I.e., anyone can make a plugin that 
validates or converts a setup(version="...") string, put it on 
sys.path, and force canonical versions.


From floris.bruynooghe at gmail.com  Thu May 14 18:13:43 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 14 May 2009 17:13:43 +0100
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
Message-ID: <20090514161343.GA8889@laurie.devork>

On Thu, May 14, 2009 at 02:38:07PM +0100, Paul Moore wrote:
> 2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
> > I'd like to make a difference between what is "importable" and what is
> > "installed" in Python.
> >
> > what's "installed" for me is what is in a site-packages directory that
> > contains egg-info directories.
> 
> I'm not sure how well-defined that is. Can you precisely define what a
> "site-packages directory" is? Specifically, what about the Windows
> registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
> There may be other "special" locations, I can't recall for certain.

I've treated those as an equivalent of the PYTHONPATH environment
variable (but one that doesn't require rebooting if you add to it
during a system wide installation).  So they are not site-packages
directories.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Thu May 14 18:33:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 12:33:10 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com
 >
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
Message-ID: <20090514163029.E48A03A4061@sparrow.telecommunity.com>

At 12:03 PM 5/14/2009 +0200, Tarek Ziad? wrote:
>Hello
>
>for PEP 376, I have one last fuzzy point.
>
>http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup
>
>The "get_egg_info" api is currently based on scanning the whole
>sys.path. And since sys.path can be modified by people,
>so the algorithm is linear and can slow down when there are a lot of paths.
>
>I have a proposal: let's restrict the search for this API to
>site-package directories only. (directories added with
>site.addsitedir)
>
>People will be able to mark add any directory (like the per-user
>site-package directory - http://www.python.org/dev/peps/pep-0370)
>
>This requires to add in site.py a registry to keep track of all
>directories added through site.addsitedir
>
>Any thoughts ?

What tradeoffs are you optimizing for?  Note that a single scan of 
every directory on sys.path is exactly what happens when an import 
doesn't find its target until the *last* directory on sys.path.  So 
this is not really a big deal if you're only doing it *once*.

If you want to optimize for repeated searches, the best way to do 
this is with a structure like pkg_resources' WorkingSet object - it 
simply reads the directories once and makes an object for each 
installed package.  These objects don't do any further I/O, so really 
we're just talking about caching a list of .egg-info filenames.

Each object in the set can be queried for its metadata -- in which 
case it reads it exactly once, and caches it.

With this setup, the full directory scan is only ever done once -- 
and it's basically equivalent to adding an extra import at the time 
you first import the metadata management module.

Yes, it does mean a global, unless you want to hand off cache 
management to the application.  But the way pkg_resources does it, 
with WorkingSet and Distribution objects, allows an app with special 
needs to do its own path management and search operations.

IOW, this approach keeps simple things simple, and leaves complex 
things possible.  It also does less I/O than what you're proposing, 
since in the normal case the directories are only ever searched once, 
and the actual metadata reads are both lazy and cached.

Note, too, that site-packages dirs are likely to have more packages 
on them than other directories, which means you're not necessarily 
saving much I/O to start with, and even that small savings evaporates 
as soon as you do more than one lookup for plugins.


From ziade.tarek at gmail.com  Thu May 14 23:04:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 23:04:26 +0200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
Message-ID: <94bdd2610905141404i1d598998o1a1901419ccef213@mail.gmail.com>

On Thu, May 14, 2009 at 3:38 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/5/14 Tarek Ziad? <ziade.tarek at gmail.com>:
>> I'd like to make a difference between what is "importable" and what is
>> "installed" in Python.
>>
>> what's "installed" for me is what is in a site-packages directory that
>> contains egg-info directories.
>
> I'm not sure how well-defined that is. Can you precisely define what a
> "site-packages directory" is? Specifically, what about the Windows
> registry items under HKLM\Software\Python\PythonCore\x.y\PythonPath?
> There may be other "special" locations, I can't recall for certain.

I need to check that, I wasn't aware of it



>
> Paul.
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu May 14 22:43:25 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 22:43:25 +0200
Subject: [Distutils] RFC : Version comparison
In-Reply-To: <20090514161949.19E363A4061@sparrow.telecommunity.com>
References: <94bdd2610904220212x29c83a7etd612dc5cbac9c10a@mail.gmail.com>
	<94bdd2610904230219v17d9aeak3cce3946c0ba3d2a@mail.gmail.com>
	<94bdd2610905061158q65e8e371jf957d33b4031c5fa@mail.gmail.com>
	<20090506191746.BF4613A40A5@sparrow.telecommunity.com>
	<94bdd2610905061311i63f03155w2ec4b18255a6a933@mail.gmail.com>
	<94bdd2610905070102q9e6f222xc66d0c82912e22a0@mail.gmail.com>
	<94bdd2610905130534q7fe09bccybec575b9a8df2540@mail.gmail.com>
	<20090514020540.7F1AF3A40A5@sparrow.telecommunity.com>
	<94bdd2610905140119i4eca75d2wc9800ec93e8b4620@mail.gmail.com>
	<20090514161949.19E363A4061@sparrow.telecommunity.com>
Message-ID: <94bdd2610905141343t11dc48dxecee087dfbc41372@mail.gmail.com>

2009/5/14 P.J. Eby <pje at telecommunity.com>:
> At 10:19 AM 5/14/2009 +0200, Tarek Ziad? wrote:
>>
>> from a setuptools user point of view, the benefit I can see is that
>> they will have better
>> version numbers
>
> Better how? ?That was my question. ?I personally find the common version
> patterns in use (e.g. ?'-' and '-r' for post-releases) less clunky and
> easier to read than 'post'.

Removing the usage of "-" is important for debian packagers for example.

>?I also don't care for '.' in front of dev and
> post. ?So, I don't see the changes as all "better" from a readability POV.
>
> That's not to say it's a bad thing, if it gives other benefits. ?But it has
> not been stated what the benefits are supposed to be.

True, we didn't post that, we just talked about it during Pycon,

Maybe Matthias and Toshio can write down a detailed list of the issues
with setuptools version system, to make them clearer.


>> Now I suppose setuptools will have to propose both for some time,
>
> I don't see why. ?AFAICT, RationalVersion is a strict subset of the syntax
> supported by setuptools, and most setuptools versions in use should be
> convertible.

e.g. switch to scrict=True at some point.

>> But what I am scared of is : who will work on setuptools side ? can
>> you bless someone to do the
>> work when we agree on a common roadmap ?
>
> I don't see that there is any work *to* do in setuptools core, since
> RationalVersion is a strict subset, and we have setup-argument validation
> providers already. ?I.e., anyone can make a plugin that validates or
> converts a setup(version="...") string, put it on sys.path, and force
> canonical versions.

Ok that's great then. (the svn support would need the same extendability)

Regards
Tarek

From ziade.tarek at gmail.com  Thu May 14 23:10:14 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 23:10:14 +0200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <20090514163029.E48A03A4061@sparrow.telecommunity.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<20090514163029.E48A03A4061@sparrow.telecommunity.com>
Message-ID: <94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.com>

2009/5/14 P.J. Eby <pje at telecommunity.com>:
> IOW, this approach keeps simple things simple, and leaves complex things
> possible. ?It also does less I/O than what you're proposing, since in the
> normal case the directories are only ever searched once, and the actual
> metadata reads are both lazy and cached.

Makes a lot of sense yes. I think I'll just start a prototype for that
code,  propose 376 on python-dev and see.

From pje at telecommunity.com  Thu May 14 23:27:20 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 17:27:20 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.co
 m>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<20090514163029.E48A03A4061@sparrow.telecommunity.com>
	<94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.com>
Message-ID: <20090514212441.E54D33A4061@sparrow.telecommunity.com>

At 11:10 PM 5/14/2009 +0200, Tarek Ziad? wrote:
>2009/5/14 P.J. Eby <pje at telecommunity.com>:
> > IOW, this approach keeps simple things simple, and leaves complex things
> > possible.  It also does less I/O than what you're proposing, since in the
> > normal case the directories are only ever searched once, and the actual
> > metadata reads are both lazy and cached.
>
>Makes a lot of sense yes. I think I'll just start a prototype for that
>code,  propose 376 on python-dev and see.

Notice, by the way, that this is a good example of pkg_resources' 
code only looking complex if you haven't gone through all the 
requirements and design issues yourself.  The further you go with 
this, the more likely you're going to end up with something very much 
resembling pkg_resources itself.

The bad news is that you'll be reinventing a lot of wheels.  The good 
news is, you should also be able to understand and maintain 
pkg_resources internals by the time you're done.  ;-)

The relevant classes in pkg_resources, btw, are WorkingSet, 
Distribution, and EntryPoint.  You might want to just copy them and 
strip out the parts you don't need.  (i.e. sys.path manipulation and 
requirements-resolving stuff)


From ziade.tarek at gmail.com  Thu May 14 23:38:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 May 2009 23:38:34 +0200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <20090514212441.E54D33A4061@sparrow.telecommunity.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<20090514163029.E48A03A4061@sparrow.telecommunity.com>
	<94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.com>
	<20090514212441.E54D33A4061@sparrow.telecommunity.com>
Message-ID: <94bdd2610905141438i5ead789csb48b9bb7cad04c86@mail.gmail.com>

On Thu, May 14, 2009 at 11:27 PM, P.J. Eby <pje at telecommunity.com> wrote:
>
> The relevant classes in pkg_resources, btw, are WorkingSet, Distribution,
> and EntryPoint. ?You might want to just copy them and strip out the parts
> you don't need. ?(i.e. sys.path manipulation and requirements-resolving
> stuff)

I tried to reuse it already, but I ended up removing lots of stuff and
creating a simpler loop to scan sys.path elements,
and work with the different sorts (zip dir, dir, etc)

That's basically what I have done here:
http://bitbucket.org/tarek/extensions/src/tip/extensions/reader.py)

That happened because I got a bit lost in the way finders were working
in setuptools, which seemed
a bit over-engineered at that time to me (but I might end up changing
my mind at sime point when I understand it better).

Although, I think I understand why WorkingSet is required now.





-- 
Tarek Ziad? | http://ziade.org

From randall at tnr.cc  Fri May 15 00:23:30 2009
From: randall at tnr.cc (Randall Smith)
Date: Thu, 14 May 2009 17:23:30 -0500
Subject: [Distutils] including a third party egg without "installing" it
In-Reply-To: <20090514160507.A66843A4061@sparrow.telecommunity.com>
References: <gug4q5$r9v$1@ger.gmane.org>
	<20090514160507.A66843A4061@sparrow.telecommunity.com>
Message-ID: <gui5l6$57e$1@ger.gmane.org>

P.J. Eby wrote:
> Yes.  Simply install your entire application as eggs and scripts in a 
> single directory (using "easy_install -maxd somedir myapp") and then 
> tarball and ship "somedir".  Less muss, less fuss, and no require().  
> (I.e., just declare your dependencies in setup.py)

So far, I like this.  I installed the script in the same directory (I 
passing "mad" to easy_install).

cd myapp/trunk
easy_install -mad ~/tmp/appdir --script-dir=~/tmp/appdir .

Thanks.

--Randall


From pje at telecommunity.com  Fri May 15 00:48:11 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 14 May 2009 18:48:11 -0400
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <94bdd2610905141438i5ead789csb48b9bb7cad04c86@mail.gmail.co
 m>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<20090514163029.E48A03A4061@sparrow.telecommunity.com>
	<94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.com>
	<20090514212441.E54D33A4061@sparrow.telecommunity.com>
	<94bdd2610905141438i5ead789csb48b9bb7cad04c86@mail.gmail.com>
Message-ID: <20090514224532.162153A4061@sparrow.telecommunity.com>

At 11:38 PM 5/14/2009 +0200, Tarek Ziad? wrote:
>That happened because I got a bit lost in the way finders were working
>in setuptools, which seemed
>a bit over-engineered at that time to me (but I might end up changing
>my mind at sime point when I understand it better).

The finders mechanism is there to allow support for arbitrary PEP 302 
importers -- i.e., sys.path strings that don't go to the filesystem.

The main reason why there are two finders currently (find_on_path and 
find_in_zip) is that find_in_zip supports egg directory 
bundling.  That is, an .egg zipfile may contain arbitrarily nested 
.egg directories, and an .egg directory may contain arbitrarily 
nested .egg directories or .egg zipfiles.

However, if you're not implementing dependency resolution, you don't 
need that support.  If such a nested item were in use, it'd be on 
sys.path already.  (In other words, the nesting lookups are only 
useful for alternate version discovery, not simple metadata lookups.)


From noah.gift at gmail.com  Fri May 15 05:11:49 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Fri, 15 May 2009 15:11:49 +1200
Subject: [Distutils] PEP 376 - site-directories and site.addsitedir
In-Reply-To: <20090514224532.162153A4061@sparrow.telecommunity.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<20090514163029.E48A03A4061@sparrow.telecommunity.com>
	<94bdd2610905141410s13d2c34fp69c8208fd35111b2@mail.gmail.com>
	<20090514212441.E54D33A4061@sparrow.telecommunity.com>
	<94bdd2610905141438i5ead789csb48b9bb7cad04c86@mail.gmail.com>
	<20090514224532.162153A4061@sparrow.telecommunity.com>
Message-ID: <e91cc0270905142011g70a1305du855e4b54694490fc@mail.gmail.com>

On Fri, May 15, 2009 at 10:48 AM, P.J. Eby <pje at telecommunity.com> wrote:

> At 11:38 PM 5/14/2009 +0200, Tarek Ziad? wrote:
>
>> That happened because I got a bit lost in the way finders were working
>> in setuptools, which seemed
>> a bit over-engineered at that time to me (but I might end up changing
>> my mind at sime point when I understand it better).
>>
>
> The finders mechanism is there to allow support for arbitrary PEP 302
> importers -- i.e., sys.path strings that don't go to the filesystem.
>
> The main reason why there are two finders currently (find_on_path and
> find_in_zip) is that find_in_zip supports egg directory bundling.  That is,
> an .egg zipfile may contain arbitrarily nested .egg directories, and an .egg
> directory may contain arbitrarily nested .egg directories or .egg zipfiles.


The idea of nested eggs reminds me again of the concept of a "toolset",
where something like virtualenv contains a discreet set of libraries for a
particular package, like Pylons 0.9.7.  You could download this qualified
toolset without needing to worry about runtime configuration issues, and
then it would also be quite fast to import if it was isolated from the other
toolsets.  This means that an entry point could just call right to a toolset
or a set of toolsets.

The downside of having isolated toolsets/virtualenvs is more diskspace, and
the need to explicately add new libraries to your toolset.  The upside is a
streamlined sys.path and guaranteed compatability by the toolset authors.

>
>
> However, if you're not implementing dependency resolution, you don't need
> that support.  If such a nested item were in use, it'd be on sys.path
> already.  (In other words, the nesting lookups are only useful for alternate
> version discovery, not simple metadata lookups.)
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Cheers,

Noah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090515/3fd3c100/attachment.htm>

From brian at vanguardistas.net  Fri May 15 13:00:41 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Fri, 15 May 2009 13:00:41 +0200
Subject: [Distutils] Utility to mirror an egg index from a Debian
	distribution
Message-ID: <20090515110041.GC66338@Boo.lan>

Hi,

It may seem like a backwards way of doing things, but I have a need for
a utility that can maintain a python package index mirror of a Debian
repository.

The basic idea is to extract the tarballs of Python packages from the
Debian repository and rename them to the original setuptools name. It
should also create a buildout-compatible versions file of the versions
in the repository.

My current implementation idea is to unpack the tarball and use the
egg-metadata to figure out what the "egg" name of the tarball should be.

Does such a tool exists? If not, I'll probably start working on one on
svn.zope.org Real Soon Now (TM). Comments, suggestions much
appreciated!

-- 
Brian Sutherland

From pje at telecommunity.com  Fri May 15 18:25:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 15 May 2009 12:25:31 -0400
Subject: [Distutils] Utility to mirror an egg index from a Debian
 distribution
In-Reply-To: <20090515110041.GC66338@Boo.lan>
References: <20090515110041.GC66338@Boo.lan>
Message-ID: <20090515162252.81B003A4061@sparrow.telecommunity.com>

At 01:00 PM 5/15/2009 +0200, Brian Sutherland wrote:
>Hi,
>
>It may seem like a backwards way of doing things, but I have a need for
>a utility that can maintain a python package index mirror of a Debian
>repository.
>
>The basic idea is to extract the tarballs of Python packages from the
>Debian repository and rename them to the original setuptools name. It
>should also create a buildout-compatible versions file of the versions
>in the repository.
>
>My current implementation idea is to unpack the tarball and use the
>egg-metadata to figure out what the "egg" name of the tarball should be.

Running "setup.py --name --version" will dump out the name and 
version, whether you use distutils or setuptools.  If you want a 
setuptools-compatible name and version, you'll need to postprocess 
those strings with pkg_resources.safe_name() and safe_version(), then 
escape them with to_filename() if you're using them as components of 
a sdist or egg filename.


From david at ar.media.kyoto-u.ac.jp  Sat May 16 05:44:43 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 16 May 2009 12:44:43 +0900
Subject: [Distutils] Strict/Loose Version and toolchains checks
Message-ID: <4A0E36AB.5060704@ar.media.kyoto-u.ac.jp>

Hi,
   
    Since we are talking about versioning, here is one more complaint
which may fall into the bugreport category or not, I am not sure. There
are some version checks in some codepaths which do not make much sense
IMHO, in particular:
    - distutils checks the versions of the toolchain. This is broken. If
I want to use gcc built from svn to build my extensions, distutils
should just do what I ask it to do, and not barf as it currently does.
    - bdist_msi also fails to run when the project version is not a
released version. I can't see the rationale for that behavior, and it is
not consistent with others bdist_ anyway.

thanks,

David

From ziade.tarek at gmail.com  Sat May 16 09:36:13 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 16 May 2009 09:36:13 +0200
Subject: [Distutils] Strict/Loose Version and toolchains checks
In-Reply-To: <4A0E36AB.5060704@ar.media.kyoto-u.ac.jp>
References: <4A0E36AB.5060704@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610905160036n22cf9938wf3817123dfad0a37@mail.gmail.com>

Hi David,

can you fill an issue about that ?

Thanks

On Sat, May 16, 2009 at 5:44 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
> ? ?Since we are talking about versioning, here is one more complaint
> which may fall into the bugreport category or not, I am not sure. There
> are some version checks in some codepaths which do not make much sense
> IMHO, in particular:
> ? ?- distutils checks the versions of the toolchain. This is broken. If
> I want to use gcc built from svn to build my extensions, distutils
> should just do what I ask it to do, and not barf as it currently does.
> ? ?- bdist_msi also fails to run when the project version is not a
> released version. I can't see the rationale for that behavior, and it is
> not consistent with others bdist_ anyway.
>
> thanks,
>
> David
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From chris at simplistix.co.uk  Fri May 15 17:20:53 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 15 May 2009 16:20:53 +0100
Subject: [Distutils] Adding entry points into Distutils ?
In-Reply-To: <79990c6b0905090855u567fe742q49915c44010153be@mail.gmail.com>
References: <94bdd2610905041646g317ddd91s5180fe67b3c6bec5@mail.gmail.com>	
	<b10441b8-0063-46bd-9e2c-2095f1ccb979@d25g2000prn.googlegroups.com>	
	<082A4DCD-2F04-4A34-8A1D-00F09F68483E@gmail.com>	
	<79990c6b0905070554y7eaf407fobfcaf97d0ebe5890@mail.gmail.com>	
	<5832D785-392B-4460-B34D-1D6792FBD3D3@gmail.com>	
	<4A02ECF7.80004@trueblade.com> <gtuqo0$ir5$1@ger.gmane.org>	
	<79990c6b0905070736y7217725ai7185b171cf0a101b@mail.gmail.com>	
	<4A035AE5.3040203@trueblade.com>
	<4A059156.6030401@simplistix.co.uk>
	<79990c6b0905090855u567fe742q49915c44010153be@mail.gmail.com>
Message-ID: <4A0D8855.5030604@simplistix.co.uk>

Paul Moore wrote:
> 2009/5/9 Chris Withers <chris at simplistix.co.uk>:
>> +1.
>>
>> <flame>Anyone who's arguing against this is either not deploying stuff in a
>> repeatable fashion, and so isn't serious in my books, or is so serious that
>> they're cutting vm images to roll out and so dump everything for the app in
>> site packages</flame>
> 
> Hmm. I'll accept the flame to the extent that I don't deploy stuff to
> a wide enough audience to be qualified to comment on that side of
> things. But:
> 
> 1. Can you clarify what "this" is?

Given that I'm writing this offline on a train, the answer is "no", I'm 
afraid ;-)

> Part of the issue I see is that
> there's never a clear enough statement of a proposal for a non-expert
> in the field to follow.

Which proposal do you think isn't clear enough?

> 2. Distutils is for distributing python modules/packages, so
> *application* deployment is out of scope.

Python applications rely on packages and modules. Application in the 
scope I'm using it might mean something like "YouTube".

> Script support is for
> small-scale stuff (in my view).

Wrong. I use it for creating start/stop scripts for services along with 
scripts to stick in crontabs. I don't need to bother with that for 
small-scales stuff.

> 3. Accepting that I don't know what you mean by "this", can I point
> out that as a user, I personally have problems with a significant
> proportion of scripts distributed with packages - so are you saying
> that those packages "aren't serious",

Probably, or certainly that their authors don't really care about the 
scripts they generate...

> or that there is no way of doing
> what they (and I) want at present?

I'm afraid I don't remember/understand what you want...

> I'll see if I can find some in due course. (Basically, the types of
> things I see are scripts distributed on Windows with no filetype
> extension, .bat wrappers for command line scripts which don't work
> right when called from another bat file, setuptools-built exe wrappers
> which result in version-specific binaries for pure-python code, to
> give some examples).

The last is the "right" thing. These scripts work cross-platform and are 
tied to the correct version of python for the setup as "originally 
intended", ie: the version the package was installed with.

cheers,

Chris



From david at ar.media.kyoto-u.ac.jp  Sat May 16 12:50:46 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 16 May 2009 19:50:46 +0900
Subject: [Distutils] Strict/Loose Version and toolchains checks
In-Reply-To: <94bdd2610905160036n22cf9938wf3817123dfad0a37@mail.gmail.com>
References: <4A0E36AB.5060704@ar.media.kyoto-u.ac.jp>
	<94bdd2610905160036n22cf9938wf3817123dfad0a37@mail.gmail.com>
Message-ID: <4A0E9A86.3090105@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> Hi David,
>
> can you fill an issue about that ?
>
>   

Done, that's 6039 and 6040 on the bug tracker,

David

From david.lyon at preisshare.net  Sat May 16 13:13:43 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 16 May 2009 07:13:43 -0400
Subject: [Distutils] Utility to mirror an egg index from a Debian
	distribution
In-Reply-To: <20090515110041.GC66338@Boo.lan>
References: <20090515110041.GC66338@Boo.lan>
Message-ID: <89868eb56914934abcc2151ee554449e@preisshare.net>


Hi Brian,

It sounds interesting. I might be interested in helping.

What I would like to do is make a test script to download all the 
packages off pypi and build them under multiple platforms.

Basically, I want to make some tests that will try to install
"everything" and then deinstall "everything" and see what happens.

I'm not sure if this has ever been done.. but it might provide
some interesting test results..

Maybe it's similar...


On Fri, 15 May 2009 13:00:41 +0200, Brian Sutherland
<brian at vanguardistas.net> wrote:
> Hi,
> 
> It may seem like a backwards way of doing things, but I have a need for
> a utility that can maintain a python package index mirror of a Debian
> repository.
> 
> The basic idea is to extract the tarballs of Python packages from the
> Debian repository and rename them to the original setuptools name. It
> should also create a buildout-compatible versions file of the versions
> in the repository.
> 
> My current implementation idea is to unpack the tarball and use the
> egg-metadata to figure out what the "egg" name of the tarball should be.
> 
> Does such a tool exists? If not, I'll probably start working on one on
> svn.zope.org Real Soon Now (TM). Comments, suggestions much
> appreciated!

From david.lyon at preisshare.net  Sun May 17 04:24:46 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 16 May 2009 22:24:46 -0400
Subject: [Distutils] =?utf-8?q?Clearing_WorkingSet_in_pkg=5Fresources=2E?=
	=?utf-8?b?Li4gPw==?=
In-Reply-To: <20090515162252.81B003A4061@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
Message-ID: <820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>


Hi,

I'm using working set and I have come into an issue.

In my package manager I can now install packages and deinstall fine.

The problem that I now face is want to refresh WorkingSet after
a package has been deinstalled.

Or, I need to do the refresh when swapping to a different version
of python that is installed on the system.

Code looks something like this:

    def installed_packages(self):

        result = []
        
        if not self.interpreted:
            site.addsitedir(self.python_sitepackages_path)

        import pkg_resources

        ws = pkg_resources.WorkingSet()

        for i in ws:
            s = str(i)
            result.append(s.split(' '))

        return result



From pje at telecommunity.com  Sun May 17 04:36:13 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 16 May 2009 22:36:13 -0400
Subject: [Distutils] Clearing WorkingSet in pkg_resources. .. ?
In-Reply-To: <820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
Message-ID: <20090517023459.5AE663A4061@sparrow.telecommunity.com>

At 10:24 PM 5/16/2009 -0400, David Lyon wrote:

>Hi,
>
>I'm using working set and I have come into an issue.
>
>In my package manager I can now install packages and deinstall fine.
>
>The problem that I now face is want to refresh WorkingSet after
>a package has been deinstalled.
>
>Or, I need to do the refresh when swapping to a different version
>of python that is installed on the system.

Just create a fresh WorkingSet object, and use that one.  You 
probably want to be using explicitly-created WorkingSet instances for 
this anyway.  (easy_install certainly does.)


From david.lyon at preisshare.net  Sun May 17 04:45:02 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 16 May 2009 22:45:02 -0400
Subject: [Distutils]
 =?utf-8?q?Clearing_WorkingSet_in_pkg=5Fresources=2E_?=
 =?utf-8?b?Li4gPw==?=
In-Reply-To: <20090517023459.5AE663A4061@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
	<20090517023459.5AE663A4061@sparrow.telecommunity.com>
Message-ID: <ad4957cfd385718213642ff1052fb243@preisshare.net>

On Sat, 16 May 2009 22:36:13 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> Just create a fresh WorkingSet object, and use that one.  You 
> probably want to be using explicitly-created WorkingSet instances for 
> this anyway.  (easy_install certainly does.)

I am creating a fresh one every time...

So after I deinstall, the package is still shown, even though it's
no longer in the system. It's still in the interpretors memory.

When I look at pkg_resources.. I can't find "the real code".. it
seems to be all just 'jumps'..

I fully understand this refreshing of packages is outside of the
original code but I am at a loss to find where in the system I
can find the code that implements workingset...

If you have any idea where that code is... I would be very
greatful...

David

From pje at telecommunity.com  Sun May 17 06:43:20 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 17 May 2009 00:43:20 -0400
Subject: [Distutils] Clearing WorkingSet in pkg_resources. . . ?
In-Reply-To: <ad4957cfd385718213642ff1052fb243@preisshare.net>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
	<20090517023459.5AE663A4061@sparrow.telecommunity.com>
	<ad4957cfd385718213642ff1052fb243@preisshare.net>
Message-ID: <20090517044039.87EA73A4061@sparrow.telecommunity.com>

At 10:45 PM 5/16/2009 -0400, David Lyon wrote:
>On Sat, 16 May 2009 22:36:13 -0400, "P.J. Eby" <pje at telecommunity.com>
>wrote:
> > Just create a fresh WorkingSet object, and use that one.  You
> > probably want to be using explicitly-created WorkingSet instances for
> > this anyway.  (easy_install certainly does.)
>
>I am creating a fresh one every time...
>
>So after I deinstall, the package is still shown, even though it's
>no longer in the system. It's still in the interpretors memory.

Do you mean on sys.path?  If it's an .egg file or directory on 
sys.path it will probably still be listed as a distribution when you 
create a new WorkingSet.  Either that, or it's an .egg-info egg whose 
.egg-info you didn't delete.


From david.lyon at preisshare.net  Sun May 17 06:55:38 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sun, 17 May 2009 00:55:38 -0400
Subject: [Distutils]
 =?utf-8?q?Clearing_WorkingSet_in_pkg=5Fresources=2E_?=
 =?utf-8?b?LiAuID8=?=
In-Reply-To: <20090517044039.87EA73A4061@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
	<20090517023459.5AE663A4061@sparrow.telecommunity.com>
	<ad4957cfd385718213642ff1052fb243@preisshare.net>
	<20090517044039.87EA73A4061@sparrow.telecommunity.com>
Message-ID: <e97725fe97befb34865407301aae33b2@preisshare.net>

On Sun, 17 May 2009 00:43:20 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:

>>So after I deinstall, the package is still shown, even though it's
>>no longer in the system. It's still in the interpretors memory.
> 
> Do you mean on sys.path?  If it's an .egg file or directory on 
> sys.path it will probably still be listed as a distribution when you 
> create a new WorkingSet.  Either that, or it's an .egg-info egg whose 
> .egg-info you didn't delete.

I have to restart the program for the refresh to work...

Ideally, I would like to see the package list refreshed immediately
after a package is installed or removed. I can make a work around
and hide packages that are deinstalled, but i was hoping to find
an easier way.

  Checking for C:\Python25\scripts\easy_install.exe
  Running installer ... C:\Python25\scripts\easy_install.exe demset
  Searching for demset
  Reading http://pypi.python.org/simple/demset/
  Reading http://deron.meranda.us/python/demset/
  Best match: demset 1.0
  Downloading http://deron.meranda.us/python/demset/dist/demset-1.0.tar.gz
  Processing demset-1.0.tar.gz
  Running demset-1.0\setup.py -q bdist_egg --dist-dir
c:\docume~1\david\locals~1\temp\easy_install-ozryy7\demset-
1.0\egg-dist-tmp-jiy7q0
  zip_safe flag not set; analyzing archive contents...
  Adding demset 1.0 to easy-install.pth file
  Installed c:\python25\lib\site-packages\demset-1.0-py2.5.egg
  Processing dependencies for demset
  Finished processing dependencies for demset

Preparing to remove demset 
 - package mentioned in PTH c:\python25\lib\site-packages\easy-install.pth
 - Updating PTH c:\python25\lib\site-packages\easy-install.pth
 - Removing Egg c:\python25\lib\site-packages\demset-1.0-py2.5.egg
 - deinst completed  demset



From pje at telecommunity.com  Sun May 17 16:29:33 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 17 May 2009 10:29:33 -0400
Subject: [Distutils] Clearing WorkingSet in pkg_resources. . . ?
In-Reply-To: <e97725fe97befb34865407301aae33b2@preisshare.net>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
	<20090517023459.5AE663A4061@sparrow.telecommunity.com>
	<ad4957cfd385718213642ff1052fb243@preisshare.net>
	<20090517044039.87EA73A4061@sparrow.telecommunity.com>
	<e97725fe97befb34865407301aae33b2@preisshare.net>
Message-ID: <20090517142657.BCCA43A40D9@sparrow.telecommunity.com>

At 12:55 AM 5/17/2009 -0400, David Lyon wrote:
>On Sun, 17 May 2009 00:43:20 -0400, "P.J. Eby" <pje at telecommunity.com>
>wrote:
>
> >>So after I deinstall, the package is still shown, even though it's
> >>no longer in the system. It's still in the interpretors memory.
> >
> > Do you mean on sys.path?  If it's an .egg file or directory on
> > sys.path it will probably still be listed as a distribution when you
> > create a new WorkingSet.  Either that, or it's an .egg-info egg whose
> > .egg-info you didn't delete.
>
>I have to restart the program for the refresh to work...
>
>Ideally, I would like to see the package list refreshed immediately
>after a package is installed or removed. I can make a work around
>and hide packages that are deinstalled, but i was hoping to find
>an easier way.

You didn't answer my question.


From david.lyon at preisshare.net  Mon May 18 00:11:40 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sun, 17 May 2009 18:11:40 -0400
Subject: [Distutils]
 =?utf-8?q?Clearing_WorkingSet_in_pkg=5Fresources=2E_?=
 =?utf-8?b?LiAuID8=?=
In-Reply-To: <20090517142657.BCCA43A40D9@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<820dffa18f04e3013b36d80fe5c4b42e@preisshare.net>
	<20090517023459.5AE663A4061@sparrow.telecommunity.com>
	<ad4957cfd385718213642ff1052fb243@preisshare.net>
	<20090517044039.87EA73A4061@sparrow.telecommunity.com>
	<e97725fe97befb34865407301aae33b2@preisshare.net>
	<20090517142657.BCCA43A40D9@sparrow.telecommunity.com>
Message-ID: <5f23b12b9716209775d545ef4411eb9d@preisshare.net>

On Sun, 17 May 2009 10:29:33 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
>> > Do you mean on sys.path?  If it's an .egg file or directory on
>> > sys.path it will probably still be listed as a distribution when you
>> > create a new WorkingSet.  Either that, or it's an .egg-info egg whose
>> > .egg-info you didn't delete.

Ok, you're right.

I didn't update the sys.path... I will go do that.

As for the .egg file/directory - they are deleted.

Yes - I didn't check for .egg-info's.

Thanks - I'll try that and see if it fixes it.

David

From brian at vanguardistas.net  Mon May 18 09:58:17 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Mon, 18 May 2009 09:58:17 +0200
Subject: [Distutils] Utility to mirror an egg index from a
	Debian	distribution
In-Reply-To: <89868eb56914934abcc2151ee554449e@preisshare.net>
References: <20090515110041.GC66338@Boo.lan>
	<89868eb56914934abcc2151ee554449e@preisshare.net>
Message-ID: <20090518075817.GA81882@Boo.lan>

On Sat, May 16, 2009 at 07:13:43AM -0400, David Lyon wrote:
> 
> Hi Brian,
> 

Hi David,

> It sounds interesting. I might be interested in helping.

Great :)

> What I would like to do is make a test script to download all the 
> packages off pypi and build them under multiple platforms.
> 
> Basically, I want to make some tests that will try to install
> "everything" and then deinstall "everything" and see what happens.
> 
> I'm not sure if this has ever been done.. but it might provide
> some interesting test results..

It think that could give some good information, especially about
dependency issues, and perhaps file conflicts.

> Maybe it's similar...

Unfortunately I don't think it's similar enough to fold these 2 projects
into one. I'm basically trying to re-make pypi (or a list of links to
pypi) from a Debian repository. I.e something that looks like this:

    http://ftp.us.debian.org/debian/

Also, while I will probably be downloading most of the tarballs, I
definitely won't be installing anything.

> 
> 
> On Fri, 15 May 2009 13:00:41 +0200, Brian Sutherland
> <brian at vanguardistas.net> wrote:
> > Hi,
> > 
> > It may seem like a backwards way of doing things, but I have a need for
> > a utility that can maintain a python package index mirror of a Debian
> > repository.
> > 
> > The basic idea is to extract the tarballs of Python packages from the
> > Debian repository and rename them to the original setuptools name. It
> > should also create a buildout-compatible versions file of the versions
> > in the repository.
> > 
> > My current implementation idea is to unpack the tarball and use the
> > egg-metadata to figure out what the "egg" name of the tarball should be.
> > 
> > Does such a tool exists? If not, I'll probably start working on one on
> > svn.zope.org Real Soon Now (TM). Comments, suggestions much
> > appreciated!
> 

-- 
Brian Sutherland

From exarkun at divmod.com  Mon May 18 22:00:25 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 18 May 2009 16:00:25 -0400
Subject: [Distutils] Version string normalization
In-Reply-To: 0
Message-ID: <20090518200025.21531.859099492.divmod.quotient.22846@henry.divmod.com>

Hello,

I've just noticed that when using setuptools, the name of the file created
by the sdist command changes.  The version passed to setup is "0.9.33+r17283"
but the file written is "Nevow-0.9.33-r17283.tar.gz".

This change causes various problems for me.  How can I avoid it?

Thanks,
Jean-Paul

From pje at telecommunity.com  Mon May 18 22:22:43 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 18 May 2009 16:22:43 -0400
Subject: [Distutils] Version string normalization
In-Reply-To: <20090518200025.21531.859099492.divmod.quotient.22846@henry
	.divmod.com>
References: <20090518200025.21531.859099492.divmod.quotient.22846@henry.divmod.com>
Message-ID: <20090518202000.CE69A3A40D7@sparrow.telecommunity.com>

At 04:00 PM 5/18/2009 -0400, Jean-Paul Calderone wrote:
>Hello,
>
>I've just noticed that when using setuptools, the name of the file created
>by the sdist command changes.  The version passed to setup is "0.9.33+r17283"
>but the file written is "Nevow-0.9.33-r17283.tar.gz".
>
>This change causes various problems for me.  How can I avoid it?

You would need to subclass the sdist command, or use a 
post-processing step to rename it.


From a.cavallo at cavallinux.eu  Mon May 18 23:00:51 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Mon, 18 May 2009 22:00:51 +0100
Subject: [Distutils] setup tools maximum recursion depth exceeded
Message-ID: <200905182200.52650.a.cavallo@cavallinux.eu>

There's been a previous post (Tarek, "Fixing the mess in sdist/egg_info") 
about the fact that setuptools fails due to a circular dependency:

distutils.sdist.run() -> setuptools.build_py.data_files ->
setuptools.egg_info.run() -> distutils.sdist.add_defaults() ->
setuptools.build_py.data_files -> etc

Actually I'm working packaging sphinx and it fails for this reason. Has 
anything moved to fix this problem? 

Regards,
Antonio

From david.lyon at preisshare.net  Tue May 19 02:34:38 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 May 2009 20:34:38 -0400
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
Message-ID: <76c6b4835a0b890745af3c0fb4049214@preisshare.net>


Hi all,

I've just had a moment to read PEP-376..

I seriously question the need for a new .EGG_INFO directory...

Given that a typical site-packages directory doesn't have many
files in it anyway.

Typically, it will contain 20+ project/package subdirectories
or .EGG files/directories.

Why not keep the .EGG_INFO files in the site-packages directory?

It hardly has many files in it anyway.

It seems so much simpler to use site-packages and doesn't add 
any extra o/s overhead (yes - more directories does slow a 
system down - slowly - but surely).

>From a pure python perspective.. another directory isn't needed.

I think we could do a lot more to manage what we already 
have... before making new wholes for mess to go in.....

David



From pje at telecommunity.com  Tue May 19 02:59:22 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 18 May 2009 20:59:22 -0400
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <76c6b4835a0b890745af3c0fb4049214@preisshare.net>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
Message-ID: <20090519005641.709A53A40D7@sparrow.telecommunity.com>

At 08:34 PM 5/18/2009 -0400, David Lyon wrote:
>Why not keep the .EGG_INFO files in the site-packages directory?

That's where they go.  Each installed project has its own .egg-info 
subdirectory containing the listed files.  See the EggFormats 
documentation for details.  PEP 376 is just adding stdlib support for 
the .egg-info format defined by setuptools, and adding a new RECORD file to it.


From david.lyon at preisshare.net  Tue May 19 02:55:53 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 May 2009 20:55:53 -0400
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <20090519005641.709A53A40D7@sparrow.telecommunity.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
	<20090519005641.709A53A40D7@sparrow.telecommunity.com>
Message-ID: <4fa98a4a5bed0edcca71770cafd0fd60@preisshare.net>



On Mon, 18 May 2009 20:59:22 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> At 08:34 PM 5/18/2009 -0400, David Lyon wrote:
>>Why not keep the .EGG_INFO files in the site-packages directory?
> 
> That's where they go.  Each installed project has its own .egg-info 
> subdirectory containing the listed files.  See the EggFormats 
> documentation for details.  PEP 376 is just adding stdlib support for 
> the .egg-info format defined by setuptools, and adding a new RECORD file
to
> it.

Let me quote....

.egg-info becomes a directory
=============================

The first change would be to make `.egg-info` a directory and let it
hold the `PKG-INFO` file built by the `write_pkg_file` method of 
the `Distribution` class in Distutils.

This change will not impact Python itself, because `egg-info` files are not
used anywhere yet in the standard library besides Distutils. 


From noah.gift at gmail.com  Tue May 19 02:59:24 2009
From: noah.gift at gmail.com (Noah Gift)
Date: Tue, 19 May 2009 12:59:24 +1200
Subject: [Distutils] PEP 376 - critique - why more sub-directories
	within python aren't needed...
In-Reply-To: <20090519005641.709A53A40D7@sparrow.telecommunity.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
	<20090519005641.709A53A40D7@sparrow.telecommunity.com>
Message-ID: <e91cc0270905181759h21972b38we4771800fca533f@mail.gmail.com>

On Tue, May 19, 2009 at 12:59 PM, P.J. Eby <pje at telecommunity.com> wrote:

> At 08:34 PM 5/18/2009 -0400, David Lyon wrote:
>
>> Why not keep the .EGG_INFO files in the site-packages directory?
>>
>
> That's where they go.  Each installed project has its own .egg-info
> subdirectory containing the listed files.  See the EggFormats documentation
> for details.  PEP 376 is just adding stdlib support for the .egg-info format
> defined by setuptools, and adding a new RECORD file to it.


But if this implementation is the same as eggs, then each egg directory is
then scanned and imported into sys.path, unlike normal packages which are
called via simple namespaces.

>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Cheers,

Noah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090519/e2eaf1cc/attachment.htm>

From jeff at taupro.com  Tue May 19 04:22:38 2009
From: jeff at taupro.com (Jeff Rush)
Date: Mon, 18 May 2009 21:22:38 -0500
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <76c6b4835a0b890745af3c0fb4049214@preisshare.net>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
Message-ID: <4A1217EE.9060506@taupro.com>

David Lyon wrote:
> 
> I seriously question the need for a new .EGG_INFO directory...
> 
> Given that a typical site-packages directory doesn't have many
> files in it anyway.
> 
> Typically, it will contain 20+ project/package subdirectories
> or .EGG files/directories.

;-) Mine has 213 files/directories, and I use virtualenv and buildout to keep
the number down.  I also use Gentoo Linux which has distro-packaged lots of
Python-based apps, leading to more entries in site-packages than might
otherwise be expected in a virtualenv/buildout environment.

-Jeff


From david.lyon at preisshare.net  Tue May 19 04:34:24 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 May 2009 22:34:24 -0400
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <4A1217EE.9060506@taupro.com>
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
	<4A1217EE.9060506@taupro.com>
Message-ID: <527a4fe52b68470f8e0e6614c5712640@preisshare.net>

On Mon, 18 May 2009 21:22:38 -0500, Jeff Rush <jeff at taupro.com> wrote:
> ;-) Mine has 213 files/directories, and I use virtualenv and buildout to
> keep
> the number down.

Ok.. that sounds like it is "in the normal range"

How many subdirectories in site-packages is not so important if they
are packages. How many regular files?

213 directories and 213 .EGG_INFO files doesn't sound like too much
to me.

Somebody should actually run some performance tests... to examine
the performance impact on any of this.....

David



From pje at telecommunity.com  Tue May 19 04:51:51 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 18 May 2009 22:51:51 -0400
Subject: [Distutils] PEP 376 - critique - why more sub-directories
 within python aren't needed...
In-Reply-To: <e91cc0270905181759h21972b38we4771800fca533f@mail.gmail.com
 >
References: <94bdd2610905140303r75f5612x6bf96ba3b6ff52cc@mail.gmail.com>
	<e91cc0270905140333u5bb229c5x5ef9ccd5ed7627e6@mail.gmail.com>
	<94bdd2610905140538s72200ef1t4ed237a70878c5b5@mail.gmail.com>
	<79990c6b0905140638s38de38abicedfeae19c7a8d62@mail.gmail.com>
	<76c6b4835a0b890745af3c0fb4049214@preisshare.net>
	<20090519005641.709A53A40D7@sparrow.telecommunity.com>
	<e91cc0270905181759h21972b38we4771800fca533f@mail.gmail.com>
Message-ID: <20090519024911.34EFF3A40D7@sparrow.telecommunity.com>

At 12:59 PM 5/19/2009 +1200, Noah Gift wrote:


>On Tue, May 19, 2009 at 12:59 PM, P.J. Eby 
><<mailto:pje at telecommunity.com>pje at telecommunity.com> wrote:
>At 08:34 PM 5/18/2009 -0400, David Lyon wrote:
>Why not keep the .EGG_INFO files in the site-packages directory?
>
>
>That's where they go.  Each installed project has its own .egg-info 
>subdirectory containing the listed files.  See the EggFormats 
>documentation for details.  PEP 376 is just adding stdlib support 
>for the .egg-info format defined by setuptools, and adding a new 
>RECORD file to it.
>
>
>But if this implementation is the same as eggs, then each egg 
>directory is then scanned and imported into sys.path, unlike normal 
>packages which are called via simple namespaces.

You are confused.  Please refer to the EggFormats documentation for 
the layout difference between .egg and .egg-info:

    http://peak.telecommunity.com/DevCenter/EggFormats

.egg-info eggs look just like "normal" distutils installs, except for 
the addition of a "project-version-pyversion.egg-info" subdirectory 
alongside the code.  In Python 2.5 and up, distutils also installs 
packages this way, except the "project-version-pyversion.egg-info" is 
a file instead of a directory - as documented in the above documentation.

PEP 376 is simply proposing stdlib support for querying the existing 
formats, and adding a new standard RECORD file to go in that directory.


From mail at timgolden.me.uk  Tue May 19 16:00:14 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 19 May 2009 15:00:14 +0100
Subject: [Distutils] Changes in r72585
Message-ID: <4A12BB6E.2050202@timgolden.me.uk>

[coming from a thread on python-win32:
http://mail.python.org/pipermail/python-win32/2009-May/009130.html
]

In short: changes to build_ext.py in r72585 caused the pywin32
packages to stop building correctly. The changes in question
are those which call get_ext_filename with the short form of
the ext.name (without the package prefix). The pywin32 setup.py
expects the fully-qualified name in the get_ext_filename in
its subclass.

Now I don't know whether this is an incautious change to build_ext
or a false -- altho' historically valid -- assumption on the part
of the pywin32 developers. You can see their code here at line 1119:

http://pywin32.cvs.sourceforge.net/viewvc/pywin32/pywin32/setup.py?revision=1.107&view=markup

I'm afraid I'm fairly ignorant of the design rationale behind
distutils; I'm hoping someone here has a better idea :)

Opinions?

TJG

From ziade.tarek at gmail.com  Tue May 19 16:54:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 16:54:18 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <4A12BB6E.2050202@timgolden.me.uk>
References: <4A12BB6E.2050202@timgolden.me.uk>
Message-ID: <94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>

Hi Tim, thanks for the feedback,

this has been fixed in r72758

http://svn.python.org/view/python/trunk/Lib/distutils/command/build_ext.py?r1=72593&r2=72758

Regards
Tarek

On Tue, May 19, 2009 at 4:00 PM, Tim Golden <mail at timgolden.me.uk> wrote:
> [coming from a thread on python-win32:
> http://mail.python.org/pipermail/python-win32/2009-May/009130.html
> ]
>
> In short: changes to build_ext.py in r72585 caused the pywin32
> packages to stop building correctly. The changes in question
> are those which call get_ext_filename with the short form of
> the ext.name (without the package prefix). The pywin32 setup.py
> expects the fully-qualified name in the get_ext_filename in
> its subclass.
>
> Now I don't know whether this is an incautious change to build_ext
> or a false -- altho' historically valid -- assumption on the part
> of the pywin32 developers. You can see their code here at line 1119:
>
> http://pywin32.cvs.sourceforge.net/viewvc/pywin32/pywin32/setup.py?revision=1.107&view=markup
>
> I'm afraid I'm fairly ignorant of the design rationale behind
> distutils; I'm hoping someone here has a better idea :)
>
> Opinions?
>
> TJG
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From mail at timgolden.me.uk  Tue May 19 17:01:02 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 19 May 2009 16:01:02 +0100
Subject: [Distutils] Changes in r72585
In-Reply-To: <94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
Message-ID: <4A12C9AE.7010502@timgolden.me.uk>

Tarek Ziad? wrote:
> Hi Tim, thanks for the feedback,
> 
> this has been fixed in r72758
> 
> http://svn.python.org/view/python/trunk/Lib/distutils/command/build_ext.py?r1=72593&r2=72758

'fraid not :)

It was running against r72758 which showed the bug up:
I had to back-track through a few revisions to find what
introduced it.

The attached patch against r72780 "fixes" the problem
(ie makes pywin32 build correctly) but since I've no
real idea what the intention of the code is, I've likewise
no idea of what I might be breaking by doing this :(

TJG
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: build_ext.patch
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090519/d631a7de/attachment.txt>

From ziade.tarek at gmail.com  Tue May 19 17:00:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 17:00:50 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
Message-ID: <94bdd2610905190800u708557deu54a8802acb8563e3@mail.gmail.com>

By the way,

Do you happen to have a pywin32 buildbot that could run on the python
trunk ? (where I can get emails when it's broken)

That would be useful because I am doing a lot of work in the trunk.

Tarek


On Tue, May 19, 2009 at 4:54 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi Tim, thanks for the feedback,
>
> this has been fixed in r72758
>
> http://svn.python.org/view/python/trunk/Lib/distutils/command/build_ext.py?r1=72593&r2=72758
>
> Regards
> Tarek
>
> On Tue, May 19, 2009 at 4:00 PM, Tim Golden <mail at timgolden.me.uk> wrote:
>> [coming from a thread on python-win32:
>> http://mail.python.org/pipermail/python-win32/2009-May/009130.html
>> ]
>>
>> In short: changes to build_ext.py in r72585 caused the pywin32
>> packages to stop building correctly. The changes in question
>> are those which call get_ext_filename with the short form of
>> the ext.name (without the package prefix). The pywin32 setup.py
>> expects the fully-qualified name in the get_ext_filename in
>> its subclass.
>>
>> Now I don't know whether this is an incautious change to build_ext
>> or a false -- altho' historically valid -- assumption on the part
>> of the pywin32 developers. You can see their code here at line 1119:
>>
>> http://pywin32.cvs.sourceforge.net/viewvc/pywin32/pywin32/setup.py?revision=1.107&view=markup
>>
>> I'm afraid I'm fairly ignorant of the design rationale behind
>> distutils; I'm hoping someone here has a better idea :)
>>
>> Opinions?
>>
>> TJG
>> _______________________________________________
>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
> --
> Tarek Ziad? | http://ziade.org
>



-- 
Tarek Ziad? | http://ziade.org

From mail at timgolden.me.uk  Tue May 19 17:08:43 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 19 May 2009 16:08:43 +0100
Subject: [Distutils] Changes in r72585
In-Reply-To: <94bdd2610905190800u708557deu54a8802acb8563e3@mail.gmail.com>
References: <4A12BB6E.2050202@timgolden.me.uk>	
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
	<94bdd2610905190800u708557deu54a8802acb8563e3@mail.gmail.com>
Message-ID: <4A12CB7B.2020104@timgolden.me.uk>

Tarek Ziad? wrote:
> By the way,
> 
> Do you happen to have a pywin32 buildbot that could run on the python
> trunk ? (where I can get emails when it's broken)

I have what you might call a manual buildbot. (A buildman?)
I have set up an environment where I can create
installers for python trunk & pywin32 for the trunk
and principal branches (release-maint26 / py3k).

Once I've ironed out the issues with those, I hope
to be able to do the same for other simple projects
such as pyodbc which don't require large amounts of
library configuration.

Ultimately, I hope to be able to link into something
like snakebite.org to produce overnight build MSIs
of Windows projects. (Long-term plan, tho'!)

TJG

> 
> That would be useful because I am doing a lot of work in the trunk.
> 
> Tarek
> 
> 
> On Tue, May 19, 2009 at 4:54 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> Hi Tim, thanks for the feedback,
>>
>> this has been fixed in r72758
>>
>> http://svn.python.org/view/python/trunk/Lib/distutils/command/build_ext.py?r1=72593&r2=72758
>>
>> Regards
>> Tarek
>>
>> On Tue, May 19, 2009 at 4:00 PM, Tim Golden <mail at timgolden.me.uk> wrote:
>>> [coming from a thread on python-win32:
>>> http://mail.python.org/pipermail/python-win32/2009-May/009130.html
>>> ]
>>>
>>> In short: changes to build_ext.py in r72585 caused the pywin32
>>> packages to stop building correctly. The changes in question
>>> are those which call get_ext_filename with the short form of
>>> the ext.name (without the package prefix). The pywin32 setup.py
>>> expects the fully-qualified name in the get_ext_filename in
>>> its subclass.
>>>
>>> Now I don't know whether this is an incautious change to build_ext
>>> or a false -- altho' historically valid -- assumption on the part
>>> of the pywin32 developers. You can see their code here at line 1119:
>>>
>>> http://pywin32.cvs.sourceforge.net/viewvc/pywin32/pywin32/setup.py?revision=1.107&view=markup
>>>
>>> I'm afraid I'm fairly ignorant of the design rationale behind
>>> distutils; I'm hoping someone here has a better idea :)
>>>
>>> Opinions?
>>>
>>> TJG
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>>
>> --
>> Tarek Ziad? | http://ziade.org
>>
> 
> 
> 


From ziade.tarek at gmail.com  Tue May 19 17:12:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 17:12:56 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <4A12C9AE.7010502@timgolden.me.uk>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
	<4A12C9AE.7010502@timgolden.me.uk>
Message-ID: <94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>

On Tue, May 19, 2009 at 5:01 PM, Tim Golden <mail at timgolden.me.uk> wrote:

> Index: Lib/distutils/command/build_ext.py
> ===================================================================
> --- Lib/distutils/command/build_ext.py ?(revision 72780)
> +++ Lib/distutils/command/build_ext.py ?(working copy)
> @@ -652,7 +652,8 @@
> ? ? ? ? ? ? filename = self.get_ext_filename(ext_name)
> ? ? ? ? ? ? return os.path.join(package_dir, filename)
> ? ? ? ? else:
> - ? ? ? ? ? ?filename = self.get_ext_filename(ext_name)
> + ? ? ? ? ? ?fullname = self.get_ext_fullname(ext_name)
> + ? ? ? ? ? ?filename = self.get_ext_filename(fullname)
> ? ? ? ? ? ? return os.path.join(self.build_lib, filename)
>
> ? ? def get_ext_fullname(self, ext_name):
>

Ok I see what's wrong. I am writing a test + fix right away.

build_ext and all the compiling code is still really undertested in
Distutils. I've kinda opened a can of worms
since I am working on the compilers part of the package.

I'll let you know when it's fixed.

++
Tarek

-- 
Tarek Ziad? | http://ziade.org

From mail at timgolden.me.uk  Tue May 19 17:15:41 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 19 May 2009 16:15:41 +0100
Subject: [Distutils] Changes in r72585
In-Reply-To: <94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>
References: <4A12BB6E.2050202@timgolden.me.uk>	
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>	
	<4A12C9AE.7010502@timgolden.me.uk>
	<94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>
Message-ID: <4A12CD1D.9090103@timgolden.me.uk>

Tarek Ziad? wrote:
> On Tue, May 19, 2009 at 5:01 PM, Tim Golden <mail at timgolden.me.uk> wrote:
> 
>> Index: Lib/distutils/command/build_ext.py
>> ===================================================================
>> --- Lib/distutils/command/build_ext.py  (revision 72780)
>> +++ Lib/distutils/command/build_ext.py  (working copy)
>> @@ -652,7 +652,8 @@
>>             filename = self.get_ext_filename(ext_name)
>>             return os.path.join(package_dir, filename)
>>         else:
>> -            filename = self.get_ext_filename(ext_name)
>> +            fullname = self.get_ext_fullname(ext_name)
>> +            filename = self.get_ext_filename(fullname)
>>             return os.path.join(self.build_lib, filename)
>>
>>     def get_ext_fullname(self, ext_name):
>>
> 
> Ok I see what's wrong. I am writing a test + fix right away.
> 
> build_ext and all the compiling code is still really undertested in
> Distutils. I've kinda opened a can of worms
> since I am working on the compilers part of the package.
> 
> I'll let you know when it's fixed.

Thanks. Happy to run it through my "buildman" to check it
covers that case.

TJG

From ziade.tarek at gmail.com  Tue May 19 17:16:25 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 17:16:25 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <4A12CD1D.9090103@timgolden.me.uk>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
	<4A12C9AE.7010502@timgolden.me.uk>
	<94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>
	<4A12CD1D.9090103@timgolden.me.uk>
Message-ID: <94bdd2610905190816s40e33643n6c577a60a09dcd4@mail.gmail.com>

On Tue, May 19, 2009 at 5:15 PM, Tim Golden <mail at timgolden.me.uk> wrote:
>
> Thanks. Happy to run it through my "buildman" to check it
> covers that case.


>>> from tim.golden import buildman
>>> buildman.blame_list.register('tarek at ziade.org')

From a.cavallo at cavallinux.eu  Tue May 19 17:17:40 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Tue, 19 May 2009 16:17:40 +0100
Subject: [Distutils] Changes in r72585
In-Reply-To: <4A12CB7B.2020104@timgolden.me.uk>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190800u708557deu54a8802acb8563e3@mail.gmail.com>
	<4A12CB7B.2020104@timgolden.me.uk>
Message-ID: <200905191617.41361.a.cavallo@cavallinux.eu>

On Tuesday 19 May 2009 16:08:43 Tim Golden wrote:
> Tarek Ziad? wrote:
> > By the way,
> >
> > Do you happen to have a pywin32 buildbot that could run on the python
> > trunk ? (where I can get emails when it's broken)
>
> I have what you might call a manual buildbot. (A buildman?)
> I have set up an environment where I can create
> installers for python trunk & pywin32 for the trunk
> and principal branches (release-maint26 / py3k).

It looks like a continuous integration is in need. 
As you might remember from a previous thread, I am doing something for the 
python intepreter.

It is based on the suse build server so it covers several rpm linux distro 
already (fedora, centos and suse).

As part of it I wrote also few smoke tests to verify that things work well and 
they're integrated in the rpmbuild process.

Regards,
Antonio



From ziade.tarek at gmail.com  Tue May 19 17:23:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 17:23:48 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <200905191617.41361.a.cavallo@cavallinux.eu>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190800u708557deu54a8802acb8563e3@mail.gmail.com>
	<4A12CB7B.2020104@timgolden.me.uk>
	<200905191617.41361.a.cavallo@cavallinux.eu>
Message-ID: <94bdd2610905190823g6168cc98v70468287b487f0ed@mail.gmail.com>

On Tue, May 19, 2009 at 5:17 PM, A. Cavallo <a.cavallo at cavallinux.eu> wrote:
> On Tuesday 19 May 2009 16:08:43 Tim Golden wrote:
>> Tarek Ziad? wrote:
>> > By the way,
>> >
>> > Do you happen to have a pywin32 buildbot that could run on the python
>> > trunk ? (where I can get emails when it's broken)
>>
>> I have what you might call a manual buildbot. (A buildman?)
>> I have set up an environment where I can create
>> installers for python trunk & pywin32 for the trunk
>> and principal branches (release-maint26 / py3k).
>
> It looks like a continuous integration is in need.
> As you might remember from a previous thread, I am doing something for the
> python intepreter.
>
> It is based on the suse build server so it covers several rpm linux distro
> already (fedora, centos and suse).
>
> As part of it I wrote also few smoke tests to verify that things work well and
> they're integrated in the rpmbuild process.
>

Yes I remember that. Are you able to automate this when distutils
trunk is changed ?

If not, I'd love to help. If it's already the case, having mails sent
here on failures would be a blast

Tarek


-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue May 19 18:21:54 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 May 2009 18:21:54 +0200
Subject: [Distutils] Changes in r72585
In-Reply-To: <4A12CD1D.9090103@timgolden.me.uk>
References: <4A12BB6E.2050202@timgolden.me.uk>
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>
	<4A12C9AE.7010502@timgolden.me.uk>
	<94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>
	<4A12CD1D.9090103@timgolden.me.uk>
Message-ID: <94bdd2610905190921qe5f1444t3c3802177ce69f52@mail.gmail.com>

> Thanks. Happy to run it through my "buildman" to check it
> covers that case.

Done in r72781, and currently propagating it to the branches

Thanks for the feedback Tim !

From mail at timgolden.me.uk  Tue May 19 18:25:01 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 19 May 2009 17:25:01 +0100
Subject: [Distutils] Changes in r72585
In-Reply-To: <94bdd2610905190921qe5f1444t3c3802177ce69f52@mail.gmail.com>
References: <4A12BB6E.2050202@timgolden.me.uk>	
	<94bdd2610905190754q3da0da24me891ffdc51f57e1@mail.gmail.com>	
	<4A12C9AE.7010502@timgolden.me.uk>	
	<94bdd2610905190812x4c105166w22a806b82aa6532a@mail.gmail.com>	
	<4A12CD1D.9090103@timgolden.me.uk>
	<94bdd2610905190921qe5f1444t3c3802177ce69f52@mail.gmail.com>
Message-ID: <4A12DD5D.6040104@timgolden.me.uk>

Tarek Ziad? wrote:
>> Thanks. Happy to run it through my "buildman" to check it
>> covers that case.
> 
> Done in r72781, and currently propagating it to the branches
> 
> Thanks for the feedback Tim !

Thanks for the swift response. I'll try a rebuild against
trunk and release26 at least.

TJG

From exarkun at divmod.com  Tue May 19 20:21:13 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 19 May 2009 14:21:13 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: 0
Message-ID: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>

Hello,

For a long time, the idiom I've used to specify version information in my
distutils-based packages has been this simple one: put version information
into the Python package somewhere, structure the source layout so that the
package can be imported directly from an unpacked source tarball (or vcs wc),
and import the version into setup.py.  This worked great.  And I only had to
define my version once.

Increasingly, it seems that new distutils-related tools don't support this
idiom and break in various ways.  (eg

  http://divmod.org/trac/ticket/2629
  http://divmod.org/trac/ticket/2831
)

What is the recommendation for specifying version information in a way which
is compatible with all these new tools?

Jean-Paul

From zooko at zooko.com  Tue May 19 21:12:52 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Tue, 19 May 2009 13:12:52 -0600
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
References: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
Message-ID: <42DAFEB6-4E52-4B87-AB40-6D188247BA19@zooko.com>

On May 19, 2009, at 12:21 PM, Jean-Paul Calderone wrote:

> What is the recommendation for specifying version information in a  
> way which is compatible with all these new tools?

My personal recommendation is to put the version information in a  
flat text file and have your setup.py read that flat text file  
instead of importing a Python package, e.g.:

http://allmydata.org/trac/zfec/browser/zfec/setup.py?rev=285#L88
http://allmydata.org/trac/pyutil/browser/pyutil/setup.py?rev=141#L44

Another approach is to have the revision control be the single store  
of versioning information, and read it out of revision control  
whenever your setup.py runs.  I believe I've already submitted one  
patch for each of these techniques for Nevow:

http://divmod.org/trac/ticket/2699

A third approach is require setuptools and rely on setuptools's  
implementation of the second approach -- reading versioning  
information from your revision control tool.  That one has obvious  
disadvantages (e.g. The Big Two: http://bugs.python.org/setuptools/ 
issue54 (be more like distutils with regard to --prefix=) and http:// 
bugs.python.org/setuptools/issue53 (respect the PYTHONPATH)), but it  
has the advantage that the implementation of this feature is not in  
your setup.py but is in setuptools or in a setuptools plugin such as  
setuptools_hg or setuptools_bzr.

:-)

Regards,

Zooko

From pje at telecommunity.com  Tue May 19 22:23:16 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 19 May 2009 16:23:16 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090519182113.21531.1518435284.divmod.quotient.23432@henr
	y.divmod.com>
References: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
Message-ID: <20090519202035.E10123A40D7@sparrow.telecommunity.com>

At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>Hello,
>
>For a long time, the idiom I've used to specify version information in my
>distutils-based packages has been this simple one: put version information
>into the Python package somewhere, structure the source layout so that the
>package can be imported directly from an unpacked source tarball (or vcs wc),
>and import the version into setup.py.  This worked great.  And I only had to
>define my version once.
>
>Increasingly, it seems that new distutils-related tools don't support this
>idiom and break in various ways.  (eg
>
>  http://divmod.org/trac/ticket/2629
>  http://divmod.org/trac/ticket/2831
>)
>
>What is the recommendation for specifying version information in a way which
>is compatible with all these new tools?

Don't import the file with your version info; execfile() it instead.

That is, you can have a somepackage.version module for importing, but 
from within your setup.py, do execfile('somepackage/version.py') 
instead, so as to avoid actually importing your package (and anything 
it depends on).


From exarkun at divmod.com  Tue May 19 22:28:14 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 19 May 2009 16:28:14 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090519202035.E10123A40D7@sparrow.telecommunity.com>
Message-ID: <20090519202814.21531.317720794.divmod.quotient.23496@henry.divmod.com>

On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>Hello,
>>
>>For a long time, the idiom I've used to specify version information in my
>>distutils-based packages has been this simple one: put version information
>>into the Python package somewhere, structure the source layout so that the
>>package can be imported directly from an unpacked source tarball (or vcs 
>>wc),
>>and import the version into setup.py.  This worked great.  And I only had 
>>to
>>define my version once.
>>
>>Increasingly, it seems that new distutils-related tools don't support this
>>idiom and break in various ways.  (eg
>>
>>  http://divmod.org/trac/ticket/2629
>>  http://divmod.org/trac/ticket/2831
>>)
>>
>>What is the recommendation for specifying version information in a way 
>>which
>>is compatible with all these new tools?
>
>Don't import the file with your version info; execfile() it instead.
>
>That is, you can have a somepackage.version module for importing, but from 
>within your setup.py, do execfile('somepackage/version.py') instead, so as 
>to avoid actually importing your package (and anything it depends on).
>

execfile() doesn't set __name__ properly; setting __name__ in a namespace
passed to execfile() produces a CPython SystemError.  At the moment my
version code depends on __name__ being set.  I'm sure I could work around
this, but it'd be better if I didn't have to worry about code in the
version module being executed in two unrelated contexts.  Any other
suggestions?

Jean-Paul

From ben+python at benfinney.id.au  Wed May 20 02:17:31 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 May 2009 10:17:31 +1000
Subject: [Distutils] Specifying version information once
References: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
Message-ID: <87ljos631g.fsf@benfinney.id.au>

Jean-Paul Calderone <exarkun at divmod.com> writes:

> For a long time, the idiom I've used to specify version information in my
> distutils-based packages has been this simple one: put version information
> into the Python package somewhere, structure the source layout so that the
> package can be imported directly from an unpacked source tarball (or vcs wc),
> and import the version into setup.py.  This worked great.  And I only had to
> define my version once.
> 
> Increasingly, it seems that new distutils-related tools don't support this
> idiom and break in various ways.  (eg
> 
>  http://divmod.org/trac/ticket/2629
>  http://divmod.org/trac/ticket/2831
> )

Those breakages seem to be caused in part because the module containing
version information is doing too much.

> What is the recommendation for specifying version information in a way
> which is compatible with all these new tools?

I would recommend having a module in your package whose *only*
responsibility is version information for the package, so that importing
it won't break in different environments.

That module can the be imported both by the package itself when it needs
to know its version string, and by the ?setup.py? to get that same
version string for package building.

-- 
 \          ?The only tyrant I accept in this world is the still voice |
  `\                                          within.? ?Mahatma Gandhi |
_o__)                                                                  |
Ben Finney


From pje at telecommunity.com  Wed May 20 02:49:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 19 May 2009 20:49:37 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090519202814.21531.317720794.divmod.quotient.23496@henry
	.divmod.com>
References: <20090519202035.E10123A40D7@sparrow.telecommunity.com>
	<20090519202814.21531.317720794.divmod.quotient.23496@henry.divmod.com>
Message-ID: <20090520004655.2F0483A40D7@sparrow.telecommunity.com>

At 04:28 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>>What is the recommendation for specifying version information in a way which
>>>is compatible with all these new tools?
>>
>>Don't import the file with your version info; execfile() it instead.
>>
>>That is, you can have a somepackage.version module for importing, 
>>but from within your setup.py, do 
>>execfile('somepackage/version.py') instead, so as to avoid actually 
>>importing your package (and anything it depends on).
>
>execfile() doesn't set __name__ properly; setting __name__ in a namespace
>passed to execfile() produces a CPython SystemError.  At the moment my
>version code depends on __name__ being set.  I'm sure I could work around
>this, but it'd be better if I didn't have to worry about code in the
>version module being executed in two unrelated contexts.  Any other
>suggestions?

Well, if you're okay with depending on setuptools, you could have 
your version code do this:

   version = pkg_resources.require('ThisProject')[0].version

And *only* define the version in setup.py, but don't import or run that code.

Of, if you can't depend on setuptools (no pun intended), the only 
other thing I can think of would be to put the version in a text file 
by itself, and have both the version code and setup.py read that file.


From exarkun at divmod.com  Wed May 20 03:05:07 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 19 May 2009 21:05:07 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090520004655.2F0483A40D7@sparrow.telecommunity.com>
Message-ID: <20090520010507.21531.1260147250.divmod.quotient.23602@henry.divmod.com>

On Tue, 19 May 2009 20:49:37 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>At 04:28 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" <pje at telecommunity.com> 
>>wrote:
>>>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>>>What is the recommendation for specifying version information in a way 
>>>>which
>>>>is compatible with all these new tools?
>>>
>>>Don't import the file with your version info; execfile() it instead.
>>>
>>>That is, you can have a somepackage.version module for importing, but from 
>>>within your setup.py, do execfile('somepackage/version.py') instead, so as 
>>>to avoid actually importing your package (and anything it depends on).
>>
>>execfile() doesn't set __name__ properly; setting __name__ in a namespace
>>passed to execfile() produces a CPython SystemError.  At the moment my
>>version code depends on __name__ being set.  I'm sure I could work around
>>this, but it'd be better if I didn't have to worry about code in the
>>version module being executed in two unrelated contexts.  Any other
>>suggestions?
>
>Well, if you're okay with depending on setuptools, you could have your 
>version code do this:
>
>   version = pkg_resources.require('ThisProject')[0].version
>
>And *only* define the version in setup.py, but don't import or run that 
>code.

I don't think I'm ready to make setuptools a mandatory dependency yet.

>Of, if you can't depend on setuptools (no pun intended), the only other 
>thing I can think of would be to put the version in a text file by itself, 
>and have both the version code and setup.py read that file.
>

That's an idea that's come up before.  I'm curious to here if anyone has
tried it and what their experiences have been.

Jean-Paul

From exarkun at divmod.com  Wed May 20 05:05:30 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 19 May 2009 23:05:30 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <87ljos631g.fsf@benfinney.id.au>
Message-ID: <20090520030530.21531.1602043378.divmod.quotient.23651@henry.divmod.com>

On Wed, 20 May 2009 10:17:31 +1000, Ben Finney <ben+python at benfinney.id.au> wrote:
>Jean-Paul Calderone <exarkun at divmod.com> writes:
>
>> For a long time, the idiom I've used to specify version information in my
>> distutils-based packages has been this simple one: put version information
>> into the Python package somewhere, structure the source layout so that the
>> package can be imported directly from an unpacked source tarball (or vcs wc),
>> and import the version into setup.py.  This worked great.  And I only had to
>> define my version once.
>>
>> Increasingly, it seems that new distutils-related tools don't support this
>> idiom and break in various ways.  (eg
>>
>>  http://divmod.org/trac/ticket/2629
>>  http://divmod.org/trac/ticket/2831
>> )
>
>Those breakages seem to be caused in part because the module containing
>version information is doing too much.

The only thing the version module does is define the version.  For example,

  $ cat _version.py
  # This is an auto-generated file. Use Epsilon/bin/release-divmod to update.
  from twisted.python import versions
  version = versions.Version(__name__[:__name__.rfind('.')], 0, 9, 33)
  $

Of course, the "too much" here is the use of a structured version object.
Doing anything simpler would be much like the approach Phillip's pointed
out, to have a simple text file and parse its contents without getting
Python involved at all.

>
>> What is the recommendation for specifying version information in a way
>> which is compatible with all these new tools?
>
>I would recommend having a module in your package whose *only*
>responsibility is version information for the package, so that importing
>it won't break in different environments.
>
>That module can the be imported both by the package itself when it needs
>to know its version string, and by the ?setup.py? to get that same
>version string for package building.

This has issues.  For examples, sometimes a system-wide installation of the
package will get used instead (generally not when running setup.py directly,
but it seems to happen when other tools are used).  Otherwise, the existing
_version.py from above works fine.

Jean-Paul

From pje at telecommunity.com  Wed May 20 05:22:22 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 19 May 2009 23:22:22 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090520010507.21531.1260147250.divmod.quotient.23602@henr
	y.divmod.com>
References: <20090520004655.2F0483A40D7@sparrow.telecommunity.com>
	<20090520010507.21531.1260147250.divmod.quotient.23602@henry.divmod.com>
Message-ID: <20090520031939.1929B3A40D7@sparrow.telecommunity.com>

At 09:05 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>On Tue, 19 May 2009 20:49:37 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>>At 04:28 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>>On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" 
>>><pje at telecommunity.com> wrote:
>>>>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote:
>>>>>What is the recommendation for specifying version information in 
>>>>>a way which
>>>>>is compatible with all these new tools?
>>>>
>>>>Don't import the file with your version info; execfile() it instead.
>>>>
>>>>That is, you can have a somepackage.version module for importing, 
>>>>but from within your setup.py, do 
>>>>execfile('somepackage/version.py') instead, so as to avoid 
>>>>actually importing your package (and anything it depends on).
>>>
>>>execfile() doesn't set __name__ properly; setting __name__ in a namespace
>>>passed to execfile() produces a CPython SystemError.  At the moment my
>>>version code depends on __name__ being set.  I'm sure I could work around
>>>this, but it'd be better if I didn't have to worry about code in the
>>>version module being executed in two unrelated contexts.  Any other
>>>suggestions?
>>
>>Well, if you're okay with depending on setuptools, you could have 
>>your version code do this:
>>
>>   version = pkg_resources.require('ThisProject')[0].version
>>
>>And *only* define the version in setup.py, but don't import or run that code.
>
>I don't think I'm ready to make setuptools a mandatory dependency yet.
>
>>Of, if you can't depend on setuptools (no pun intended), the only 
>>other thing I can think of would be to put the version in a text 
>>file by itself, and have both the version code and setup.py read that file.
>
>That's an idea that's come up before.  I'm curious to here if anyone has
>tried it and what their experiences have been.

Oh, now that I think of it, there is one other thing that I've 
done.  For some of my packages (including setuptools itself), I use a 
script that automatically searches and replaces version numbers, and 
stores the current version number in a configuration file.  Another 
configuration file specifies what files to edit and what format to 
insert the numbers in.  So, I just do a quick command line like 
'./version incr build' to increment the most-minor version number in 
the docs, the setup.py, and even to edit my release script.

And I suppose one last idea that you could use would be to define 
your version string in the module in such a way that setup.py could 
read it with a regular expression match, so it wouldn't be actually 
executing the code. 


From ben+python at benfinney.id.au  Wed May 20 05:57:09 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 May 2009 13:57:09 +1000
Subject: [Distutils] Specifying version information once
References: <87ljos631g.fsf@benfinney.id.au>
	<20090520030530.21531.1602043378.divmod.quotient.23651@henry.divmod.com>
Message-ID: <874ovg5sve.fsf@benfinney.id.au>

Jean-Paul Calderone <exarkun at divmod.com> writes:

> On Wed, 20 May 2009 10:17:31 +1000, Ben Finney <ben+python at benfinney.id.au> wrote:
> >Those breakages seem to be caused in part because the module
> >containing version information is doing too much.
> 
> The only thing the version module does is define the version.  For example,
> 
>  $ cat _version.py
>  # This is an auto-generated file. Use Epsilon/bin/release-divmod to update.
>  from twisted.python import versions
>  version = versions.Version(__name__[:__name__.rfind('.')], 0, 9, 33)
>  $
> 
> Of course, the "too much" here is the use of a structured version
> object.

No, I would say ?too much? here is importing a module not guaranteed to
be installed, for creating something as simple as a version. I'd prefer
the above to be::

  $ cat _version.py
  # This is an auto-generated file. Use magicbean to update.
  version = "0.9.33"
  $

Anything that wants to have that structured into something more complex
than a delimited string can use the funky Version type; but this
particular module should import without any dependencies, for exactly
the reasons that started this thread.

> Doing anything simpler would be much like the approach Phillip's
> pointed out, to have a simple text file and parse its contents without
> getting Python involved at all.

I still like the approach of having it in a module that can be imported
to automatically have the name bound to an appropriate value.

-- 
 \     ?If you pick up a starving dog and make him prosperous, he will |
  `\      not bite you. This is the principal difference between a dog |
_o__)                    and a man.? ?Mark Twain, _Pudd'n'head Wilson_ |
Ben Finney


From a.cavallo at cavallinux.eu  Wed May 20 08:17:09 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Wed, 20 May 2009 07:17:09 +0100
Subject: [Distutils] Specifying version information once
In-Reply-To: <874ovg5sve.fsf@benfinney.id.au>
References: <87ljos631g.fsf@benfinney.id.au>
	<20090520030530.21531.1602043378.divmod.quotient.23651@henry.divmod.com>
	<874ovg5sve.fsf@benfinney.id.au>
Message-ID: <200905200717.10004.a.cavallo@cavallinux.eu>

> No, I would say ?too much? here is importing a module not guaranteed to
> be installed, for creating something as simple as a version. I'd prefer
> the above to be::
>
>   $ cat _version.py
>   # This is an auto-generated file. Use magicbean to update.
>   version = "0.9.33"
>   $

Or my preferred way:
 $cat foobar/__init__.py
__version__ = "0.9.33"

and import foobar should not trigger code execution anyway IMHO so
$ python -c 'import foobar; print foobar.__version__'
0.9.33

> I still like the approach of having it in a module that can be imported
> to automatically have the name bound to an appropriate value.

Yes, agree 100% the KISS approach is always the best.

Regards,
Antonio

From chris at simplistix.co.uk  Wed May 20 10:07:32 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 20 May 2009 09:07:32 +0100
Subject: [Distutils] buildout/setuptools setup_requires and test_requires
In-Reply-To: <20090519175348.GA20860@wiggy.net>
References: <4A129B69.3020303@simplistix.co.uk>
	<20090519163341.GB12013@wiggy.net>
	<4A12E665.9020204@simplistix.co.uk>
	<20090519175348.GA20860@wiggy.net>
Message-ID: <4A13BA44.4010609@simplistix.co.uk>

Wichert Akkerman wrote:
> Previously Chris Withers wrote:
>> Wichert Akkerman wrote:
>>>> BUT, one thing I have noticed so far is that after running buildout, I 
>>>> end up with eggs for Paste, PasteDeploy and PasteScript in 
>>>> pylons_buildout/myproject, which is wrong.
>>> Everything in setup_requires and test_requires is installed there, 
>> Why?
>>
>>> and
>>> buildout can't prevent that. 
>> Again, why?
> 
> setuptools internals do that, and buildout has no influence over that as
> far as I know. If you want to discuss that distutils-sig is the right
> list.

Sorry, still not sure I follow you... Are you saying that any packages 
specified in setup_requires and test_requires will be installed in the 
path of the package that specified them rather than the same location as 
all other eggs?

If so, that would seem to be a bug, no?

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From jeff at taupro.com  Wed May 20 11:34:33 2009
From: jeff at taupro.com (Jeff Rush)
Date: Wed, 20 May 2009 04:34:33 -0500
Subject: [Distutils] Specifying version information once
In-Reply-To: <200905200717.10004.a.cavallo@cavallinux.eu>
References: <87ljos631g.fsf@benfinney.id.au>	<20090520030530.21531.1602043378.divmod.quotient.23651@henry.divmod.com>	<874ovg5sve.fsf@benfinney.id.au>
	<200905200717.10004.a.cavallo@cavallinux.eu>
Message-ID: <4A13CEA9.8020105@taupro.com>

A. Cavallo wrote:
>> No, I would say ?too much? here is importing a module not guaranteed to
>> be installed, for creating something as simple as a version. I'd prefer
>> the above to be::
>>
>>   $ cat _version.py
>>   # This is an auto-generated file. Use magicbean to update.
>>   version = "0.9.33"
>>   $
> 
> Or my preferred way:
>  $cat foobar/__init__.py
> __version__ = "0.9.33"
> 
> and import foobar should not trigger code execution anyway IMHO so
> $ python -c 'import foobar; print foobar.__version__'
> 0.9.33

That doesn't work in all cases.  Your example is of an external query of the
version of an installed module.  You also need to query the version -before-
it is installed (during the packaging phase) on sys.path, and also from
-within- the foobar module itself.  Your code does not handle those cases.  An
attempt to 'import foobar; print foobar.__version__' from a setup.py inside
foobar won't find foobar.

-Jeff


From jeff at taupro.com  Wed May 20 11:37:11 2009
From: jeff at taupro.com (Jeff Rush)
Date: Wed, 20 May 2009 04:37:11 -0500
Subject: [Distutils] buildout/setuptools setup_requires and test_requires
In-Reply-To: <4A13BA44.4010609@simplistix.co.uk>
References: <4A129B69.3020303@simplistix.co.uk>	<20090519163341.GB12013@wiggy.net>	<4A12E665.9020204@simplistix.co.uk>	<20090519175348.GA20860@wiggy.net>
	<4A13BA44.4010609@simplistix.co.uk>
Message-ID: <4A13CF47.2060100@taupro.com>

Chris Withers wrote:
> 
> Sorry, still not sure I follow you... Are you saying that any packages
> specified in setup_requires and test_requires will be installed in the
> path of the package that specified them rather than the same location as
> all other eggs?
> 
> If so, that would seem to be a bug, no?

Nope, a feature.

Packages needed -only- for packaging (setup_requires) and testing
(test_requires) should not be generally installed.  They are installed in a
special place for use just during those particular phases of operation.  No
need to have them around in production.

-Jeff


From david at ar.media.kyoto-u.ac.jp  Wed May 20 11:28:41 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 20 May 2009 18:28:41 +0900
Subject: [Distutils] Specifying version information once
In-Reply-To: <4A13CEA9.8020105@taupro.com>
References: <87ljos631g.fsf@benfinney.id.au>	<20090520030530.21531.1602043378.divmod.quotient.23651@henry.divmod.com>	<874ovg5sve.fsf@benfinney.id.au>	<200905200717.10004.a.cavallo@cavallinux.eu>
	<4A13CEA9.8020105@taupro.com>
Message-ID: <4A13CD49.8050306@ar.media.kyoto-u.ac.jp>

Jeff Rush wrote:
> A. Cavallo wrote:
>   
>>> No, I would say ?too much? here is importing a module not guaranteed to
>>> be installed, for creating something as simple as a version. I'd prefer
>>> the above to be::
>>>
>>>   $ cat _version.py
>>>   # This is an auto-generated file. Use magicbean to update.
>>>   version = "0.9.33"
>>>   $
>>>       
>> Or my preferred way:
>>  $cat foobar/__init__.py
>> __version__ = "0.9.33"
>>
>> and import foobar should not trigger code execution anyway IMHO so
>> $ python -c 'import foobar; print foobar.__version__'
>> 0.9.33
>>     
>
> That doesn't work in all cases.  Your example is of an external query of the
> version of an installed module.  You also need to query the version -before-
> it is installed (during the packaging phase) on sys.path, and also from
> -within- the foobar module itself.  Your code does not handle those cases.  An
> attempt to 'import foobar; print foobar.__version__' from a setup.py inside
> foobar won't find foobar.
>   

That's my experience as well. I am a bit delighted to see that other
people struggled on this, I felt very stupid the first time I tried to
setup this consistently without success.

My current approach is to set up the version in setup.py, but to always
generated a trivial version.py within the package (for the installed
version). The setup.py version is used by paver, the installed version
is used by the sphinx build inside a bootstrapped environment. I think
the situation where you want to build documentation at the same time as
the software happens relatively often, I would be glad to hear of a
better alternative.

cheers,

David

From chris at simplistix.co.uk  Wed May 20 11:50:29 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 20 May 2009 10:50:29 +0100
Subject: [Distutils] buildout/setuptools setup_requires and test_requires
In-Reply-To: <4A13CF47.2060100@taupro.com>
References: <4A129B69.3020303@simplistix.co.uk>	<20090519163341.GB12013@wiggy.net>	<4A12E665.9020204@simplistix.co.uk>	<20090519175348.GA20860@wiggy.net>
	<4A13BA44.4010609@simplistix.co.uk> <4A13CF47.2060100@taupro.com>
Message-ID: <4A13D265.7030903@simplistix.co.uk>

Jeff Rush wrote:
> Packages needed -only- for packaging (setup_requires) and testing
> (test_requires) should not be generally installed.  They are installed in a
> special place for use just during those particular phases of operation.  No
> need to have them around in production.

Hmm, now I know why no-one uses setup_requires or test_requires...

This is not a clear cut issue. I can see why people might feel as you 
do, but I belong to the other camp of people who believe that all eggs 
belong in one place...

I guess I'll move those packages either to install_requires or 
extras_require['test'] and be done with it.

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From a.cavallo at cavallinux.eu  Wed May 20 13:37:19 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Wed, 20 May 2009 12:37:19 +0100
Subject: [Distutils] Specifying version information once
In-Reply-To: <4A13CEA9.8020105@taupro.com>
References: <87ljos631g.fsf@benfinney.id.au>
	<200905200717.10004.a.cavallo@cavallinux.eu>
	<4A13CEA9.8020105@taupro.com>
Message-ID: <200905201237.19786.a.cavallo@cavallinux.eu>

> >
> > Or my preferred way:
> >  $cat foobar/__init__.py
> > __version__ = "0.9.33"
> >
> > and import foobar should not trigger code execution anyway IMHO so
> > $ python -c 'import foobar; print foobar.__version__'
> > 0.9.33
>
> That doesn't work in all cases. 

No it is not that the case and I'm going to bore you probably (sorry).

The "source" dir where the module foobar is located:

~someuser/foobar-1.0/setup.py
~someuser/foobar-1.0/foobar/__init__.py
$> cat ~someuser/foobar-1.0/foobar/__init__.py
__version__ = "1.0"

Assume the previous module is installed already under:

/usr/lib/python2.5/site-lib/foobar/
/usr/lib/python2.5/site-lib/foobar/__init__.py
$> cat /usr/lib/python2.5/site-lib/foobar/__init__.py
__version__ = "0.9"

(please note the site-lib subidr, where all the module not belonging to the
python standard library are located).

$~someuser/foobar-1.0/> python -c 'import foobar; print foobar.__version__'
This return (or it does on my python interpreter): "1.0" not "0.9"

The case is different for the standard libraries, in fact:
$~someuser> touch os.py
$~someuser> python -c 'import os; print os.__file__'
this will return: /usr/lib/python2.5/os.py(c)

There's no need to have foobar "installed" to reflect the correct __version__.

Then we need to agree on what do we mean for install and packaging....
"python setup.py install", "python setup.py bdist_rpm/wininst" or during a 
complete deb/rpm package build?
In the latter the version the "version" is the one available in the spec/deb 
files and it cannot be reflected from the sources anyway, no matter how hard 
one try to do it.


> Your example is of an external query of 
> the version of an installed module.  You also need to query the version
> -before- it is installed (during the packaging phase) on sys.path, and also
> from -within- the foobar module itself.  Your code does not handle those
> cases.  An attempt to 'import foobar; print foobar.__version__' from a
> setup.py inside foobar won't find foobar.

Finding modules can always be forced using PYTHONPATH (like in case of
foobar-1.0/src/foobar layout) or from setup.py inserting into sys.path the 
subdir: that's the easiest and in my opinion the best way to do it during 
build/install stages.

Regards,
Antonio

From exarkun at divmod.com  Wed May 20 14:35:30 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 20 May 2009 08:35:30 -0400
Subject: [Distutils] Specifying version information once
In-Reply-To: <200905201237.19786.a.cavallo@cavallinux.eu>
Message-ID: <20090520123530.21531.1146565576.divmod.quotient.23911@henry.divmod.com>

On Wed, 20 May 2009 12:37:19 +0100, "A. Cavallo" <a.cavallo at cavallinux.eu> wrote:
>> >
>> > Or my preferred way:
>> >  $cat foobar/__init__.py
>> > __version__ = "0.9.33"
>> >
>> > and import foobar should not trigger code execution anyway IMHO so
>> > $ python -c 'import foobar; print foobar.__version__'
>> > 0.9.33
>>
>> That doesn't work in all cases.
>
>No it is not that the case and I'm going to bore you probably (sorry).
>
> [snip - explanation of how you can import from the package you're about
>  to install]

This is true, but it is not the complete story.  You explained the case
where setup.py is being run directly to install the package.  My first
post in the thread explained that this case works fine.  It's other cases,
such as easy_install and buildout which do not work with this strategy.

(Someone who has experience with these tools might be able to explain why
this is; I haven't investigated the details yet, I only know that people
are reporting bugs against my packages.)

> [snip]
>
>Then we need to agree on what do we mean for install and packaging....
>"python setup.py install", "python setup.py bdist_rpm/wininst" or during a
>complete deb/rpm package build?
>In the latter the version the "version" is the one available in the spec/deb
>files and it cannot be reflected from the sources anyway, no matter how hard
>one try to do it.
>

The only version I'm talking about here is the one I assign to my package.
If someone else is assigning a different version, then it becomes their
problem to make things work.

> [snip]
>
>Finding modules can always be forced using PYTHONPATH (like in case of
>foobar-1.0/src/foobar layout) or from setup.py inserting into sys.path the
>subdir: that's the easiest and in my opinion the best way to do it during
>build/install stages.
>

Maybe so, but it can be made compatible with all (or most, or any?) of the
tools people are using to install distutils-based packages these days?

Jean-Paul

From wheatix at gmail.com  Thu May 21 02:34:30 2009
From: wheatix at gmail.com (Wheat)
Date: Wed, 20 May 2009 17:34:30 -0700 (PDT)
Subject: [Distutils] Specifying version information once
In-Reply-To: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
References: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
Message-ID: <7a6e2740-3ce4-4129-b5a4-446b3dd2e741@i28g2000prd.googlegroups.com>

To chime in here with my two cents,

The version information for a Python distribution should be specified
in setup.py (or setup.py can read this information from somewhere in
the project's files). It should *not* be specified inside a Python
package or module that is part of the distribution itself.

One area where embedding the version inside the module is problematic
is a distribution which provides multiple modules or pacakges.
Remember, one package (or module) does not have to equal one
distribution! A distribution can provide many modules and packages.
Let's say you have the Python project 'foobar'. A 'foobar'
distribution provides the 'foo' package and the 'bar' package. If you
are putting the distribution version in the package, should it be in
foo.__version__ or bar.__version__? Does foo.__version__ and
bar.__version__ both get updated to reflect the version of the foobar
disribution, or are they for more fine grained versioning where they
each specify additional version's beyond the distribution version?
(whew!)

Also it's worth mentioning that PEP 376 (http://www.python.org/dev/
peps/pep-0376/) proposes to solve some of this problem, by providing
an API for reading the version of an installed distribution - and does
so without needing to import any part of that installed distribution.
For the 'foobar' distribution this would be:

import pkgutil
foobar_metadata = pkgutil.get_metadata('foobar')
print foobar_metadata.version

With this API one will be able to consistently query for the version
of an installed *distribution*, and you don't run into the nasty
chicken-and-egg problem of needing to import a package before you
install it to determine it's version, or other weirdness or corner
cases that this approach can cause.

From ben+python at benfinney.id.au  Thu May 21 04:59:37 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 21 May 2009 12:59:37 +1000
Subject: [Distutils] Specifying version information once
References: <20090519182113.21531.1518435284.divmod.quotient.23432@henry.divmod.com>
	<7a6e2740-3ce4-4129-b5a4-446b3dd2e741@i28g2000prd.googlegroups.com>
Message-ID: <87r5yj2mau.fsf@benfinney.id.au>

Wheat <wheatix at gmail.com> writes:

> The version information for a Python distribution should be specified
> in setup.py (or setup.py can read this information from somewhere in
> the project's files).

Yes, agreed completely.

> It should *not* be specified inside a Python package or module that is
> part of the distribution itself.

Why not? It certainly makes sense for the programs to be able to
discover their own version. I hope you don't advocate duplicating the
version in another place for this purpose?

> One area where embedding the version inside the module is problematic
> is a distribution which provides multiple modules or pacakges.
> Remember, one package (or module) does not have to equal one
> distribution! A distribution can provide many modules and packages.

In this case, I'm of the opinion that each Python package should know
its own version (via a simple module that declares or discovers it, with
no dependency on anything but standard Python), and the package
distribution tools should choose one of those Python packages to be the
one which determines the distribution version.

> Let's say you have the Python project 'foobar'. A 'foobar'
> distribution provides the 'foo' package and the 'bar' package. If you
> are putting the distribution version in the package, should it be in
> foo.__version__ or bar.__version__?

I'd say the distribution version should be identical to one of the
packages, as chosen by whoever's maintaining the distribution
configuration.

> Also it's worth mentioning that PEP 376 (http://www.python.org/dev/
> peps/pep-0376/) proposes to solve some of this problem, by providing
> an API for reading the version of an installed distribution - and does
> so without needing to import any part of that installed distribution.
> For the 'foobar' distribution this would be:
> 
> import pkgutil
> foobar_metadata = pkgutil.get_metadata('foobar')
> print foobar_metadata.version

Sure. This fits my constraint that the version should be discoverable by
importing a Python module without any dependencies on anything except
standard Python. It just means that ?standard Python? would need to
include that ?pkgutil? module.

-- 
 \       ?I don't care to belong to a club that accepts people like me |
  `\                                        as members.? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Thu May 21 09:34:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 21 May 2009 09:34:26 +0200
Subject: [Distutils] PEP 376 APIs
Message-ID: <94bdd2610905210034m68234f23g67f2e28d796c1d85@mail.gmail.com>

Hello

As discussed in python-dev, I have started a prototype for pep 376 to
define the APIs. I have changed the APIs with Phillip feedback (eg
using holders)

It's located here : http://bitbucket.org/tarek/pep376

The idea would be to refine it until everyone finds it good, then
update PEP 376 accordingly, then move forward with it

It's on bitbucket, so feel free to create a branch to demonstrate a change.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From setuptools at bugs.python.org  Thu May 21 11:29:41 2009
From: setuptools at bugs.python.org (Ronald Oussoren)
Date: Thu, 21 May 2009 09:29:41 +0000
Subject: [Distutils] [issue70] Patch to remove call to /usr/bin/sw_vers on
	OSX
In-Reply-To: <1242898181.47.0.206261703097.issue70@psf.upfronthosting.co.za>
Message-ID: <1242898181.47.0.206261703097.issue70@psf.upfronthosting.co.za>


New submission from Ronald Oussoren <ronaldoussoren at mac.com>:

The attached patch for setuptools 0.6c9 replaces a shell callout to 
/usr/bin/sw_vers to using platform.mac_ver on OSX.

The primary reason for writing the patch is that gdb tends to cause havoc with 
calls to os.popen (the debugged process gets an signal and debugging gets 
aborted). 

This patch makes it a lot more convenient to run "setup.py test" in a gdb 
session, without this patch I regularly have to re-run the tests several times 
until the callout to sw_vers succeeds.

----------
files: setuptools-no-sw_vers.patch
messages: 289
nosy: ronaldoussoren
priority: feature
status: unread
title: Patch to remove call to /usr/bin/sw_vers on OSX
Added file: http://bugs.python.org/setuptools/file52/setuptools-no-sw_vers.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue70>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-no-sw_vers.patch
Type: application/octet-stream
Size: 455 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090521/24d3ea33/attachment-0001.obj>

From exarkun at divmod.com  Thu May 21 19:49:55 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 21 May 2009 13:49:55 -0400
Subject: [Distutils] easy_install picks windows egg on Linux
In-Reply-To: 0
Message-ID: <20090521174956.21531.1058837091.divmod.quotient.24290@henry.divmod.com>

Hi all,

I just noticed that easy_install selects a Windows binary egg when trying to
install pyOpenSSL on Linux.  Predictably, this fails:

$ PYTHONPATH=/tmp/junk6 easy_install --install-dir /tmp/junk6 --dry-run pyOpenSSL
Creating /tmp/junk6/site.py
Searching for pyOpenSSL
Reading http://pypi.python.org/simple/pyOpenSSL/
Reading http://pyopenssl.sourceforge.net/
Reading http://launchpad.net/pyopenssl
Best match: pyOpenSSL 0.9.py2.5
Downloading http://pypi.python.org/packages/2.5/p/pyOpenSSL/pyOpenSSL-0.9.py2.5-winxp32.egg#md5=38d273a65bae20f527ff8d21c225d10e
Processing pyOpenSSL-0.9.py2.5-winxp32.egg
Moving pyOpenSSL-0.9.py2.5-winxp32.egg to /tmp/junk6
Traceback (most recent call last):
  File "/usr/bin/easy_install", line 8, in <module>
    load_entry_point('setuptools==0.6c8', 'console_scripts', 'easy_install')()
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1671, in main
    with_ei_usage(lambda:
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1659, in with_ei_usage
    return f()
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1675, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 211, in run
    self.easy_install(spec, not self.no_deps)
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 446, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 476, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 619, in install_eggs
    return [self.install_egg(dist_filename, tmpdir)]
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 696, in install_egg
    return self.egg_distribution(destination)
  File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 661, in egg_distribution
    metadata = EggMetadata(zipimport.zipimporter(egg_path))
zipimport.ZipImportError: not a Zip file
exarkun at charm:/tmp$ 

Is this a known issue?

Jean-Paul

From pje at telecommunity.com  Thu May 21 22:52:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 21 May 2009 16:52:04 -0400
Subject: [Distutils] easy_install picks windows egg on Linux
In-Reply-To: <20090521174956.21531.1058837091.divmod.quotient.24290@henr
	y.divmod.com>
References: <20090521174956.21531.1058837091.divmod.quotient.24290@henry.divmod.com>
Message-ID: <20090521205033.954263A40D9@sparrow.telecommunity.com>

At 01:49 PM 5/21/2009 -0400, Jean-Paul Calderone wrote:
>Downloading 
>http://pypi.python.org/packages/2.5/p/pyOpenSSL/pyOpenSSL-0.9.py2.5-winxp32.egg#md5=38d273a65bae20f527ff8d21c225d10e

What's bizarre is that that's not a valid egg filename; they must 
have renamed it at some point.

The filename is invalid in two ways: first, it has a '.' between the 
version and the Python version (it should be a '-') and second, 
'winxp32' is not a valid platform name.

I would suggest contacting the PyOpenSSL maintainers to find out why 
they have renamed it in this fashion; .egg files should never be 
renamed under any circumstances.

(I'm actually surprised that easy_install is recognizing it at all, 
but basically it appears to be treating it as if it's a 
non-platform-specific egg.  That's arguably a bug.)


>zipimport.ZipImportError: not a Zip file

This part is an expected failure with a --dry-run.  It perhaps could 
be fixed, but the problem is that with --dry-run on, the egg is not 
moved to the target directory, so when it tries to create a 
Distribution object for the installed egg, it fails.  This has 
nothing to do with the version problem - you will probably see this 
occur at the end of any --dry-run install, unless an 
identically-named egg was was already present.


From setuptools at bugs.python.org  Thu May 21 23:44:40 2009
From: setuptools at bugs.python.org (Jean-Paul Calderone)
Date: Thu, 21 May 2009 21:44:40 +0000
Subject: [Distutils] [issue71] type of renamed eggs misdetected by
	easy_install
In-Reply-To: <1242942280.79.0.648600868397.issue71@psf.upfronthosting.co.za>
Message-ID: <1242942280.79.0.648600868397.issue71@psf.upfronthosting.co.za>


New submission from Jean-Paul Calderone <exarkun at twistedmatrix.com>:

Due to user error, an egg which should have been a Python 2.5 Windows binary egg
got renamed to "pyOpenSSL-0.9.py2.5-winxp32.egg" and uploaded to PyPI. 
easy_install will select this egg for installation on Linux, though it is not
usable on that platform.

----------
messages: 290
nosy: exarkun
priority: bug
status: unread
title: type of renamed eggs misdetected by easy_install

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue71>
_______________________________________________

From exarkun at divmod.com  Thu May 21 23:45:51 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 21 May 2009 17:45:51 -0400
Subject: [Distutils] easy_install picks windows egg on Linux
In-Reply-To: <20090521205033.954263A40D9@sparrow.telecommunity.com>
Message-ID: <20090521214551.21531.546386996.divmod.quotient.24325@henry.divmod.com>

On Thu, 21 May 2009 16:52:04 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>At 01:49 PM 5/21/2009 -0400, Jean-Paul Calderone wrote:
>>Downloading 
>>http://pypi.python.org/packages/2.5/p/pyOpenSSL/pyOpenSSL-0.9.py2.5-winxp32.egg#md5=38d273a65bae20f527ff8d21c225d10e
>
>What's bizarre is that that's not a valid egg filename; they must have 
>renamed it at some point.
>
>The filename is invalid in two ways: first, it has a '.' between the version 
>and the Python version (it should be a '-') and second, 'winxp32' is not a 
>valid platform name.
>
>I would suggest contacting the PyOpenSSL maintainers to find out why they 
>have renamed it in this fashion; .egg files should never be renamed under 
>any circumstances.

"they" would be me.  It seems the renaming was a result of copy/pasting
some other piece of build automation combined with me not really thinking
about whether the name might be important.

>(I'm actually surprised that easy_install is recognizing it at all, but 
>basically it appears to be treating it as if it's a non-platform-specific 
>egg.  That's arguably a bug.)

Filed, issue 71.

>>zipimport.ZipImportError: not a Zip file
>
>This part is an expected failure with a --dry-run.

Makes sense.

Thanks,

Jean-Paul

From rupert.thurner at gmail.com  Fri May 22 15:11:11 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 06:11:11 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
Message-ID: <cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com>

we now face the same issue with pil-1.1.6. it compiles correctly, but
at the end it somehow tries again to call the default compiler the
creator of the easy_install package used. is this set somewhere, or
where is this set?

we have it here:      /opt/SUNWspro/bin/cc
while the default is: /opt/studio/SOS11/SUNWspro/bin/cc


# CC=/opt/SUNWspro/bin/cc python setup.py build
running build
running build_py
running build_ext
building '_imaging' extension
/opt/studio/SOS11/SUNWspro/bin/cc -G build/temp.solaris-2.10-sun4v-2.6/
_imaging.o build/temp.solaris-2.10-sun4v-2.6/decode.o build/
temp.solaris-2.10-sun4v-2.6/encode.o build/temp.solaris-2.10-sun4v-2.6/
map.o build/temp.solaris-2.10-sun4v-2.6/display.o build/
temp.solaris-2.10-sun4v-2.6/outline.o build/temp.solaris-2.10-
sun4v-2.6/path.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Access.o
build/temp.solaris-2.10-sun4v-2.6/libImaging/Antialias.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Bands.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/BitDecode.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/Blend.o build/temp.solaris-2.10-sun4v-2.6/libImaging/
Chops.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Convert.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/ConvertYCbCr.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Copy.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/Crc32.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/Crop.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Dib.o
build/temp.solaris-2.10-sun4v-2.6/libImaging/Draw.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Effects.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/EpsEncode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/File.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/Fill.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/Filter.o build/temp.solaris-2.10-sun4v-2.6/libImaging/
FliDecode.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Geometry.o
build/temp.solaris-2.10-sun4v-2.6/libImaging/GetBBox.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/GifDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/GifEncode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/HexDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Histo.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/JpegDecode.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/JpegEncode.o build/temp.solaris-2.10-sun4v-2.6/libImaging/
LzwDecode.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Matrix.o
build/temp.solaris-2.10-sun4v-2.6/libImaging/ModeFilter.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/MspDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Negative.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Offset.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Pack.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/PackDecode.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/Palette.o build/temp.solaris-2.10-sun4v-2.6/libImaging/
Paste.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Quant.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/QuantHash.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/QuantHeap.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/PcdDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/PcxDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/PcxEncode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Point.o build/temp.solaris-2.10-
sun4v-2.6/libImaging/RankFilter.o build/temp.solaris-2.10-sun4v-2.6/
libImaging/RawDecode.o build/temp.solaris-2.10-sun4v-2.6/libImaging/
RawEncode.o build/temp.solaris-2.10-sun4v-2.6/libImaging/Storage.o
build/temp.solaris-2.10-sun4v-2.6/libImaging/SunRleDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/TgaRleDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/Unpack.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/UnpackYCC.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/XbmDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/XbmEncode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/ZipDecode.o build/
temp.solaris-2.10-sun4v-2.6/libImaging/ZipEncode.o -L/usr/local/lib -L/
opt/csw/lib -L/usr/lib -ljpeg -lz -lpython2.6 -o build/
lib.solaris-2.10-sun4v-2.6/_imaging.so
unable to execute /opt/studio/SOS11/SUNWspro/bin/cc: No such file or
directory
error: command '/opt/studio/SOS11/SUNWspro/bin/cc' failed with exit
status 1



On May 12, 1:08?pm, "rupert.thurner" <rupert.thur... at gmail.com> wrote:
> On Apr 28, 7:39?am, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
>
>
>
> > On Tue, Apr 28, 2009 at 5:23 AM, David Cournapeau
>
> > <da... at ar.media.kyoto-u.ac.jp> wrote:
> > > Tarek Ziad? wrote:
>
> > >> I am not sur to understand what you are trying to achieve.
>
> > >> This indicates your environment variable CC was properly set and sent
> > >> to the compiler.
>
> > > I don't think it does: the user requires /opt/SUNWspro/bin/cc as a C
> > > compiler, and easy_install uses /opt/studio/SOS11/SUNWspro/bin/cc which
> > > are different compilers, unless one is a softlink to the other.
>
> > Oops, correct, I didn't notice there were different.
>
> > Rupert, can you try downloading that package, uncompress it and run :
>
> > # CC=/opt/SUNWspro/bin/cc python setup.py install
>
> > If this fails too, there's a problem in Distutils.sysconfig.customize_compiler
> > we need to look at. (basically, it looks at os.environ if your
> > compiler type is "unix"
> > and not mingw or bcpp, or msvc etc..)
>
> > If it works, the problem is on easy_install itself.
>
> you are right! it works, nearly, but this might be a reportlab
> problem:
>
> # CC=/cs/SUNWspro/bin/cc python setup.py install
> ################################################
> #Attempting install of _rl_accel, sgmlop & pyHnj
> #extensions from '/tmp/ReportLab_2_3/src/rl_addons/rl_accel'
> ################################################
> ################################################
> #Attempting install of _renderPM
> #extensions from '/tmp/ReportLab_2_3/src/rl_addons/renderPM'
> # installing with freetype version 21
> ################################################
> running install
> running build
> running build_py
> copying src/reportlab/lib/hyphen.mashed -> build/lib.solaris-2.10-
> sun4v-2.6/reportlab/lib
> running build_clib
> building '_renderPM_libart' library
> /cs/SUNWspro/bin/cc -DNDEBUG -O -xO3 -xarch=v8 -DLIBART_COMPILATION -
> DWORDS_BIGENDIAN -I/tmp/ReportLab_2_3/src/rl_addons/renderPM -I/tmp/
> ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl -c /tmp/ReportLab_2_3/
> src/rl_addons/renderPM/libart_lgpl/art_uta.c -o build/
> temp.solaris-2.10-sun4v-2.6/tmp/ReportLab_2_3/src/rl_addons/renderPM/
> libart_lgpl/art_uta.o
> ...
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 29: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 31: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 33: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 38: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 39: syntax error before or at:
> ...
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 81: type specifier can not be used as array size expression
> qualifier
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 82: warning: no explicit type given
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 83: warning: old-style declaration or incorrect type for:
> art_uta_free
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 88: cannot recover from previous errors
> cc: acomp failed for /tmp/ReportLab_2_3/src/rl_addons/renderPM/
> libart_lgpl/art_uta.c
> error: command '/opt/SUNWspro/bin/cc' failed with exit status 2
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From rupert.thurner at gmail.com  Fri May 22 15:05:22 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 06:05:22 -0700 (PDT)
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
Message-ID: <75bc306e-aa41-44fc-b4f5-4718665ff3ed@u8g2000yqn.googlegroups.com>

could we patch setuptools somehow? where would be the source code?

On May 12, 1:08?pm, "rupert.thurner" <rupert.thur... at gmail.com> wrote:
> On Apr 28, 7:39?am, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
>
>
>
> > On Tue, Apr 28, 2009 at 5:23 AM, David Cournapeau
>
> > <da... at ar.media.kyoto-u.ac.jp> wrote:
> > > Tarek Ziad? wrote:
>
> > >> I am not sur to understand what you are trying to achieve.
>
> > >> This indicates your environment variable CC was properly set and sent
> > >> to the compiler.
>
> > > I don't think it does: the user requires /opt/SUNWspro/bin/cc as a C
> > > compiler, and easy_install uses /opt/studio/SOS11/SUNWspro/bin/cc which
> > > are different compilers, unless one is a softlink to the other.
>
> > Oops, correct, I didn't notice there were different.
>
> > Rupert, can you try downloading that package, uncompress it and run :
>
> > # CC=/opt/SUNWspro/bin/cc python setup.py install
>
> > If this fails too, there's a problem in Distutils.sysconfig.customize_compiler
> > we need to look at. (basically, it looks at os.environ if your
> > compiler type is "unix"
> > and not mingw or bcpp, or msvc etc..)
>
> > If it works, the problem is on easy_install itself.
>
> you are right! it works, nearly, but this might be a reportlab
> problem:
>
> # CC=/cs/SUNWspro/bin/cc python setup.py install
> ################################################
> #Attempting install of _rl_accel, sgmlop & pyHnj
> #extensions from '/tmp/ReportLab_2_3/src/rl_addons/rl_accel'
> ################################################
> ################################################
> #Attempting install of _renderPM
> #extensions from '/tmp/ReportLab_2_3/src/rl_addons/renderPM'
> # installing with freetype version 21
> ################################################
> running install
> running build
> running build_py
> copying src/reportlab/lib/hyphen.mashed -> build/lib.solaris-2.10-
> sun4v-2.6/reportlab/lib
> running build_clib
> building '_renderPM_libart' library
> /cs/SUNWspro/bin/cc -DNDEBUG -O -xO3 -xarch=v8 -DLIBART_COMPILATION -
> DWORDS_BIGENDIAN -I/tmp/ReportLab_2_3/src/rl_addons/renderPM -I/tmp/
> ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl -c /tmp/ReportLab_2_3/
> src/rl_addons/renderPM/libart_lgpl/art_uta.c -o build/
> temp.solaris-2.10-sun4v-2.6/tmp/ReportLab_2_3/src/rl_addons/renderPM/
> libart_lgpl/art_uta.o
> ...
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 29: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 31: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 33: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 38: warning: invalid white space character in directive
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.h",
> line 39: syntax error before or at:
> ...
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 81: type specifier can not be used as array size expression
> qualifier
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 82: warning: no explicit type given
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 83: warning: old-style declaration or incorrect type for:
> art_uta_free
> "/tmp/ReportLab_2_3/src/rl_addons/renderPM/libart_lgpl/art_uta.c",
> line 88: cannot recover from previous errors
> cc: acomp failed for /tmp/ReportLab_2_3/src/rl_addons/renderPM/
> libart_lgpl/art_uta.c
> error: command '/opt/SUNWspro/bin/cc' failed with exit status 2
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From ziade.tarek at gmail.com  Fri May 22 15:19:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 22 May 2009 15:19:58 +0200
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com>
Message-ID: <94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>

On Fri, May 22, 2009 at 3:11 PM, rupert.thurner
<rupert.thurner at gmail.com> wrote:
> we now face the same issue with pil-1.1.6. it compiles correctly, but
> at the end it somehow tries again to call the default compiler the
> creator of the easy_install package used. is this set somewhere, or
> where is this set?
>
> we have it here: ? ? ?/opt/SUNWspro/bin/cc
> while the default is: /opt/studio/SOS11/SUNWspro/bin/cc
>
>
> # CC=/opt/SUNWspro/bin/cc python setup.py build
> running build
> running build_py
> running build_ext

The build_ext command creates a new compiler instance, then call
customize_compiler
on it, that is supposed to use the CC set in the environment when you
are under a unix-like system.

What you can do to find out what is going on is to set a pdb in
distutils.command.build_ext
in the run command, to see how customize_compiler acts in your environment.

If you don't know how to use pdb, check http://docs.python.org/library/pdb.html

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Fri May 22 15:22:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 22 May 2009 15:22:59 +0200
Subject: [Distutils] how to set cc correctly?
In-Reply-To: <75bc306e-aa41-44fc-b4f5-4718665ff3ed@u8g2000yqn.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
	<75bc306e-aa41-44fc-b4f5-4718665ff3ed@u8g2000yqn.googlegroups.com>
Message-ID: <94bdd2610905220622l72a440d0k7de9eea7ace1e0dc@mail.gmail.com>

On Fri, May 22, 2009 at 3:05 PM, rupert.thurner
<rupert.thurner at gmail.com> wrote:
> could we patch setuptools somehow? where would be the source code?

- the code is here : http://svn.python.org/view/sandbox/trunk/setuptools/
- the bug tracker is here: http://bugs.python.org/setuptools/
- the maintainer is : Phillip Eby

From rupert.thurner at gmail.com  Fri May 22 15:36:02 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 06:36:02 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com> 
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com> 
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
Message-ID: <9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com>

are you sure you are not violating something here? the documentation
on http://docs.python.org/distutils/apiref.html says:

"
distutils.sysconfig.customize_compiler(compiler)?

    Do any platform-specific customization of a
distutils.ccompiler.CCompiler instance.

    This function is only needed on Unix at this time, but should be
called consistently to support forward-compatibility. It inserts the
information that varies across Unix flavors and is stored in Python?s
Makefile. This information includes the selected compiler, compiler
and linker options, and the extension used by the linker for shared
objects.

This function is even more special-purpose, and should only be used
from Python?s own build procedures.
"

the packager of our operating systems python had the compiler in /opt/
studio/... but we do not.

rupert.


On May 22, 3:19?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> On Fri, May 22, 2009 at 3:11 PM, rupert.thurner
>
> <rupert.thur... at gmail.com> wrote:
> > we now face the same issue with pil-1.1.6. it compiles correctly, but
> > at the end it somehow tries again to call the default compiler the
> > creator of the easy_install package used. is this set somewhere, or
> > where is this set?
>
> > we have it here: ? ? ?/opt/SUNWspro/bin/cc
> > while the default is: /opt/studio/SOS11/SUNWspro/bin/cc
>
> > # CC=/opt/SUNWspro/bin/cc python setup.py build
> > running build
> > running build_py
> > running build_ext
>
> The build_ext command creates a new compiler instance, then call
> customize_compiler
> on it, that is supposed to use the CC set in the environment when you
> are under a unix-like system.
>
> What you can do to find out what is going on is to set a pdb in
> distutils.command.build_ext
> in the run command, to see how customize_compiler acts in your environment.
>
> If you don't know how to use pdb, checkhttp://docs.python.org/library/pdb.html
>
> Regards
> Tarek
>
> --
> Tarek Ziad? |http://ziade.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From ziade.tarek at gmail.com  Fri May 22 15:41:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 22 May 2009 15:41:45 +0200
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com>
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
	<9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com>
Message-ID: <94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com>

On Fri, May 22, 2009 at 3:36 PM, rupert.thurner
<rupert.thurner at gmail.com> wrote:
> are you sure you are not violating something here?

What do you mean ?

> the packager of our operating systems python had the compiler in /opt/
> studio/... but we do not.

build_ext will pick the CC it finds in the Makefile but overrides it if one
is set in os.environ['CC']

I am unable to reproduce your problem here, so you should check as I
previously said,
what is happening when you call build_ext, to understand why your CC
isn't picked.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org

From rupert.thurner at gmail.com  Fri May 22 15:44:11 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 06:44:11 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com> 
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com> 
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
Message-ID: <f20a33ba-0930-4540-9f66-4ef1dee21dab@z5g2000yqn.googlegroups.com>

i also filed a bug in opencsw to use a "standard compiler location" to
avoid such issues in the future: http://opencsw.org/mantis/view.php?id=3682.

rupert.


On May 22, 3:19?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> On Fri, May 22, 2009 at 3:11 PM, rupert.thurner
>
> <rupert.thur... at gmail.com> wrote:
> > we now face the same issue with pil-1.1.6. it compiles correctly, but
> > at the end it somehow tries again to call the default compiler the
> > creator of the easy_install package used. is this set somewhere, or
> > where is this set?
>
> > we have it here: ? ? ?/opt/SUNWspro/bin/cc
> > while the default is: /opt/studio/SOS11/SUNWspro/bin/cc
>
> > # CC=/opt/SUNWspro/bin/cc python setup.py build
> > running build
> > running build_py
> > running build_ext
>
> The build_ext command creates a new compiler instance, then call
> customize_compiler
> on it, that is supposed to use the CC set in the environment when you
> are under a unix-like system.
>
> What you can do to find out what is going on is to set a pdb in
> distutils.command.build_ext
> in the run command, to see how customize_compiler acts in your environment.
>
> If you don't know how to use pdb, checkhttp://docs.python.org/library/pdb.html
>
> Regards
> Tarek
>
> --
> Tarek Ziad? |http://ziade.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From exarkun at divmod.com  Fri May 22 15:45:39 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 22 May 2009 09:45:39 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: 0
Message-ID: <20090522134539.21531.831596775.divmod.quotient.24492@henry.divmod.com>

Hello all,

Prior to eggs, if I wanted to have debug and non-debug versions of an
extension module available, I could build and install the extension
twice and the modules would happily co-exist, for example:


  $ python setup.py build install
  ...
  $ python-dbg setup.py build install
  ...
  $ .../site-packages/twisted/python$ ls *.so
  _epoll_d.so  _epoll.so
  $ .../site-packages/twisted/python$ 

If I then ran a debug build of Python, the debug extension would be loaded.
When I ran a normal build of Python, the non-debug extension would be loaded.
However, with eggs, each new egg completely replaces the last egg.  There
seems to be no possibility for side-by-side builds.  Am I overlooking a
feature of eggs or of setuptools?

Thanks,
Jean-Paul

From rupert.thurner at gmail.com  Fri May 22 16:07:05 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 07:07:05 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com> 
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com> 
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com> 
	<9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com> 
	<94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com>
Message-ID: <deca9543-d5a0-4194-af7e-643a6c95466e@l28g2000vba.googlegroups.com>

On May 22, 3:41?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> On Fri, May 22, 2009 at 3:36 PM, rupert.thurner
>
> <rupert.thur... at gmail.com> wrote:
> > are you sure you are not violating something here?
>
> What do you mean ?
the "This function is even more special-purpose, and should only be
used
from Python?s own build procedures. "


> > the packager of our operating systems python had the compiler in /opt/
> > studio/... but we do not.
>
> build_ext will pick the CC it finds in the Makefile but overrides it if one
> is set in os.environ['CC']

i cannot find it in the code ... but it seems to work on the top level
but is not passed on, so i wonder where in the code this is ?

# ggrep -R os.environ /opt/csw/lib/python2.6/site-packages/setuptools/
*
/opt/csw/lib/python2.6/site-packages/setuptools/command/
easy_install.py:        PYTHONPATH = os.environ.get
('PYTHONPATH','').split(os.pathsep)
/opt/csw/lib/python2.6/site-packages/setuptools/command/
easy_install.py:        self.install_dir, os.environ.get
('PYTHONPATH','')
/opt/csw/lib/python2.6/site-packages/setuptools/command/
easy_install.py:    sitedirs = filter(None,os.environ.get
('PYTHONPATH','').split(os.pathsep))
/opt/csw/lib/python2.6/site-packages/setuptools/command/
easy_install.py:                    home = os.environ.get('HOME')
/opt/csw/lib/python2.6/site-packages/setuptools/command/
upload.py:        if os.environ.has_key('HOME'):
/opt/csw/lib/python2.6/site-packages/setuptools/command/
upload.py:            rc = os.path.join(os.environ['HOME'], '.pypirc')

>
> I am unable to reproduce your problem here, so you should check as I
> previously said,
> what is happening when you call build_ext, to understand why your CC
> isn't picked.

me and python programming :) i did

python -m pdb setup.py build
# python -m pdb setup.py build
> /tmp/Imaging-1.1.6/setup.py(9)<module>()
-> import glob, os, re, struct, string, sys
(Pdb) os.environ['CC']
*** NameError: name 'os' is not defined
(Pdb) import os
(Pdb) os.environ['CC']
'/opt/SUNWspro/bin/cc'
(Pdb) b distutils.command.build_ext
*** The specified object 'distutils.command.build_ext' is not a
function
or was not found along sys.path.


rupert.

From ziade.tarek at gmail.com  Fri May 22 16:13:23 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 22 May 2009 16:13:23 +0200
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <deca9543-d5a0-4194-af7e-643a6c95466e@l28g2000vba.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com>
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com>
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com>
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com>
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com>
	<9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com>
	<94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com>
	<deca9543-d5a0-4194-af7e-643a6c95466e@l28g2000vba.googlegroups.com>
Message-ID: <94bdd2610905220713h4545ea5ft574b8b423bb230cd@mail.gmail.com>

On Fri, May 22, 2009 at 4:07 PM, rupert.thurner
<rupert.thurner at gmail.com> wrote:
> On May 22, 3:41?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
>> On Fri, May 22, 2009 at 3:36 PM, rupert.thurner
>>
>> <rupert.thur... at gmail.com> wrote:
>> > are you sure you are not violating something here?
>>
>> What do you mean ?
> the "This function is even more special-purpose, and should only be
> used
> from Python?s own build procedures. "

As a matter of fact, Python uses build_ext when it's built.  What the
documentation means
here is that you should not use customize_compiler in your code, but
through build_ext
and a few other commands in Python's distutils.

I agree it's not very clear though, and that it should be made private
with a "_"


>
>
>> > the packager of our operating systems python had the compiler in /opt/
>> > studio/... but we do not.
>>
>> build_ext will pick the CC it finds in the Makefile but overrides it if one
>> is set in os.environ['CC']
>
> i cannot find it in the code ... but it seems to work on the top level
> but is not passed on, so i wonder where in the code this is ?

It's in distutils code, not setuptools. Setuptools is a layer on the
top of distutils.

From rupert.thurner at gmail.com  Fri May 22 17:26:02 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 08:26:02 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <94bdd2610905220713h4545ea5ft574b8b423bb230cd@mail.gmail.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com> 
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com> 
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com> 
	<9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com> 
	<94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com> 
	<deca9543-d5a0-4194-af7e-643a6c95466e@l28g2000vba.googlegroups.com> 
	<94bdd2610905220713h4545ea5ft574b8b423bb230cd@mail.gmail.com>
Message-ID: <d6be019c-4fe0-42b4-a4d1-2127bfa7445f@m17g2000vbi.googlegroups.com>

ah ... found it. but i do not make any progress with debugging.
somehow it seems to use the correct compiler while compiling but not
when linking.

/cs/SUNWspro/bin/cc -DNDEBUG -O -xO3 -xarch=v8 -DHAVE_LIBJPEG -
DHAVE_LIBZ -DWORDS_BIGENDIAN -I/opt/csw/include/freetype2 -IlibImaging
-I/opt/csw/include -I/usr/include -I/opt/csw/include/python2.6 -c
libImaging/ZipEncode.c -o build/temp.solaris-2.10-sun4v-2.6/libImaging/
ZipEncode.o
/opt/studio/SOS11/SUNWspro/bin/cc -G build/temp.solaris-2.10-sun4v-2.6/
_imaging.o build/temp.solaris-2.10-sun4v-2.6/decode.o
<cut> ...
-L/usr/lib -ljpeg -lz -lpython2.6 -o build/lib.solaris-2.10-sun4v-2.6/
_imaging.so

rupert.


On May 22, 4:13?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> On Fri, May 22, 2009 at 4:07 PM, rupert.thurner
>
> <rupert.thur... at gmail.com> wrote:
> > On May 22, 3:41?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> >> On Fri, May 22, 2009 at 3:36 PM, rupert.thurner
>
> >> <rupert.thur... at gmail.com> wrote:
> >> > are you sure you are not violating something here?
>
> >> What do you mean ?
> > the "This function is even more special-purpose, and should only be
> > used
> > from Python?s own build procedures. "
>
> As a matter of fact, Python uses build_ext when it's built. ?What the
> documentation means
> here is that you should not use customize_compiler in your code, but
> through build_ext
> and a few other commands in Python's distutils.
>
> I agree it's not very clear though, and that it should be made private
> with a "_"
>
>
>
> >> > the packager of our operating systems python had the compiler in /opt/
> >> > studio/... but we do not.
>
> >> build_ext will pick the CC it finds in the Makefile but overrides it if one
> >> is set in os.environ['CC']
>
> > i cannot find it in the code ... but it seems to work on the top level
> > but is not passed on, so i wonder where in the code this is ?
>
> It's in distutils code, not setuptools. Setuptools is a layer on the
> top of distutils.
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From pje at telecommunity.com  Fri May 22 17:33:54 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 22 May 2009 11:33:54 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: <20090522134539.21531.831596775.divmod.quotient.24492@henry
	.divmod.com>
References: <20090522134539.21531.831596775.divmod.quotient.24492@henry.divmod.com>
Message-ID: <20090522153112.C3C343A40D9@sparrow.telecommunity.com>

At 09:45 AM 5/22/2009 -0400, Jean-Paul Calderone wrote:
>Hello all,
>
>Prior to eggs, if I wanted to have debug and non-debug versions of an
>extension module available, I could build and install the extension
>twice and the modules would happily co-exist, for example:
>
>
>  $ python setup.py build install
>  ...
>  $ python-dbg setup.py build install
>  ...
>  $ .../site-packages/twisted/python$ ls *.so
>  _epoll_d.so  _epoll.so
>  $ .../site-packages/twisted/python$
>If I then ran a debug build of Python, the debug extension would be loaded.
>When I ran a normal build of Python, the non-debug extension would be loaded.
>However, with eggs, each new egg completely replaces the last egg.  There
>seems to be no possibility for side-by-side builds.  Am I overlooking a
>feature of eggs or of setuptools?

Have you tried "setup.py develop"?  This builds extensions in-place 
in the source tree and adds the source tree to sys.path.


From exarkun at divmod.com  Fri May 22 18:00:14 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 22 May 2009 12:00:14 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: <20090522153112.C3C343A40D9@sparrow.telecommunity.com>
Message-ID: <20090522160015.21531.479508022.divmod.quotient.24510@henry.divmod.com>

On Fri, 22 May 2009 11:33:54 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>At 09:45 AM 5/22/2009 -0400, Jean-Paul Calderone wrote:
>>Hello all,
>>
>>Prior to eggs, if I wanted to have debug and non-debug versions of an
>>extension module available, I could build and install the extension
>>twice and the modules would happily co-exist, for example:
>>
>>
>>  $ python setup.py build install
>>  ...
>>  $ python-dbg setup.py build install
>>  ...
>>  $ .../site-packages/twisted/python$ ls *.so
>>  _epoll_d.so  _epoll.so
>>  $ .../site-packages/twisted/python$
>>If I then ran a debug build of Python, the debug extension would be loaded.
>>When I ran a normal build of Python, the non-debug extension would be 
>>loaded.
>>However, with eggs, each new egg completely replaces the last egg.  There
>>seems to be no possibility for side-by-side builds.  Am I overlooking a
>>feature of eggs or of setuptools?
>
>Have you tried "setup.py develop"?  This builds extensions in-place in the 
>source tree and adds the source tree to sys.path.
>

Aware of it, but I haven't tried it.  In this use, it sounds very
similar to "setup.py build_ext -i", which also partially solves the
problem.  Unfortunately, some packages have their source laid out
such that they cannot actually be used in-place, only after they've
been installed.

Thanks,
Jean-Paul

From rupert.thurner at gmail.com  Fri May 22 18:22:03 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Fri, 22 May 2009 09:22:03 -0700 (PDT)
Subject: [Distutils] Distutils.sysconfig.customize_compiler wrong ?
In-Reply-To: <d6be019c-4fe0-42b4-a4d1-2127bfa7445f@m17g2000vbi.googlegroups.com>
References: <37ba61de-7a93-4d90-b292-44164ddb019b@d7g2000prl.googlegroups.com>
	<94bdd2610904270152t17d55c93sb1d2830f027bf797@mail.gmail.com> 
	<49F676BF.4000100@ar.media.kyoto-u.ac.jp>
	<94bdd2610904272239t4a46704fr2b83762783cdfb55@mail.gmail.com> 
	<842251be-8962-48ae-9dc6-aff2c417b442@21g2000vbk.googlegroups.com> 
	<cbfab43c-c714-441a-ab26-bb4f648707fb@h11g2000yqb.googlegroups.com> 
	<94bdd2610905220619u6067d597rc6614235422a3491@mail.gmail.com> 
	<9c823347-aac0-430f-a6ad-2515b1ac69a7@z19g2000vbz.googlegroups.com> 
	<94bdd2610905220641m5902455k935039fa7cd4234e@mail.gmail.com> 
	<deca9543-d5a0-4194-af7e-643a6c95466e@l28g2000vba.googlegroups.com> 
	<94bdd2610905220713h4545ea5ft574b8b423bb230cd@mail.gmail.com> 
	<d6be019c-4fe0-42b4-a4d1-2127bfa7445f@m17g2000vbi.googlegroups.com>
Message-ID: <5aab789a-f7d1-4f12-a91b-3eaa90146303@o14g2000vbo.googlegroups.com>

# python
Python 2.6.1 (r261:67515, Mar  2 2009, 14:14:28) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.

from distutils.ccompiler import new_compiler
compiler = new_compiler(None,None,None,None)
compiler.compiler_type='unix'
from distutils.sysconfig import customize_compiler
customize_compiler(compiler)
compiler.linker_so
['/opt/studio/SOS11/SUNWspro/bin/cc', '-G']
compiler.compiler
['/opt/SUNWspro/bin/cc', '-DNDEBUG', '-O']

export LDSHARED=/opt/SUNWspro/bin/cc
python
>>> from distutils.ccompiler import new_compiler
>>> compiler = new_compiler(None,None,None,None)
>>> compiler.compiler_type='unix'
>>> from distutils.sysconfig import customize_compiler
>>> customize_compiler(compiler)
>>> compiler.linker_so
['/opt/SUNWspro/bin/cc']

and the option "-G" is gone ?

export LDFLAGS=-G as well does the job. but i am wondering if all of
this is the intention ?

rupert.



On May 22, 5:26?pm, "rupert.thurner" <rupert.thur... at gmail.com> wrote:
> ah ... found it. but i do not make any progress with debugging.
> somehow it seems to use the correct compiler while compiling but not
> when linking.
>
> /cs/SUNWspro/bin/cc -DNDEBUG -O -xO3 -xarch=v8 -DHAVE_LIBJPEG -
> DHAVE_LIBZ -DWORDS_BIGENDIAN -I/opt/csw/include/freetype2 -IlibImaging
> -I/opt/csw/include -I/usr/include -I/opt/csw/include/python2.6 -c
> libImaging/ZipEncode.c -o build/temp.solaris-2.10-sun4v-2.6/libImaging/
> ZipEncode.o
> /opt/studio/SOS11/SUNWspro/bin/cc -G build/temp.solaris-2.10-sun4v-2.6/
> _imaging.o build/temp.solaris-2.10-sun4v-2.6/decode.o
> <cut> ...
> -L/usr/lib -ljpeg -lz -lpython2.6 -o build/lib.solaris-2.10-sun4v-2.6/
> _imaging.so
>
> rupert.
>
> On May 22, 4:13?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
>
> > On Fri, May 22, 2009 at 4:07 PM, rupert.thurner
>
> > <rupert.thur... at gmail.com> wrote:
> > > On May 22, 3:41?pm, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> > >> On Fri, May 22, 2009 at 3:36 PM, rupert.thurner
>
> > >> <rupert.thur... at gmail.com> wrote:
> > >> > are you sure you are not violating something here?
>
> > >> What do you mean ?
> > > the "This function is even more special-purpose, and should only be
> > > used
> > > from Python?s own build procedures. "
>
> > As a matter of fact, Python uses build_ext when it's built. ?What the
> > documentation means
> > here is that you should not use customize_compiler in your code, but
> > through build_ext
> > and a few other commands in Python's distutils.
>
> > I agree it's not very clear though, and that it should be made private
> > with a "_"
>
> > >> > the packager of our operating systems python had the compiler in /opt/
> > >> > studio/... but we do not.
>
> > >> build_ext will pick the CC it finds in the Makefile but overrides it if one
> > >> is set in os.environ['CC']
>
> > > i cannot find it in the code ... but it seems to work on the top level
> > > but is not passed on, so i wonder where in the code this is ?
>
> > It's in distutils code, not setuptools. Setuptools is a layer on the
> > top of distutils.
> > _______________________________________________
> > Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From stephen.c.waterbury at nasa.gov  Fri May 22 18:53:40 2009
From: stephen.c.waterbury at nasa.gov (Stephen Waterbury)
Date: Fri, 22 May 2009 12:53:40 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: <20090522160015.21531.479508022.divmod.quotient.24510@henry.divmod.com>
References: <20090522160015.21531.479508022.divmod.quotient.24510@henry.divmod.com>
Message-ID: <4A16D894.4040200@nasa.gov>

Jean-Paul Calderone wrote:
> On Fri, 22 May 2009 11:33:54 -0400, "P.J. Eby" <pje at telecommunity.com> 
> wrote:
>> At 09:45 AM 5/22/2009 -0400, Jean-Paul Calderone wrote:
>>> Hello all,
>>>
>>> Prior to eggs, if I wanted to have debug and non-debug versions of an
>>> extension module available, I could build and install the extension
>>> twice and the modules would happily co-exist, for example:
>>>
>>>
>>>  $ python setup.py build install
>>>  ...
>>>  $ python-dbg setup.py build install
>>>  ...
>>>  $ .../site-packages/twisted/python$ ls *.so
>>>  _epoll_d.so  _epoll.so
>>>  $ .../site-packages/twisted/python$
>>> If I then ran a debug build of Python, the debug extension would be 
>>> loaded.
>>> When I ran a normal build of Python, the non-debug extension would be 
>>> loaded.
>>> However, with eggs, each new egg completely replaces the last egg.  
>>> There
>>> seems to be no possibility for side-by-side builds.  Am I overlooking a
>>> feature of eggs or of setuptools?
>>
>> Have you tried "setup.py develop"?  This builds extensions in-place in 
>> the source tree and adds the source tree to sys.path.
>>
> 
> Aware of it, but I haven't tried it.  In this use, it sounds very
> similar to "setup.py build_ext -i", which also partially solves the
> problem.  Unfortunately, some packages have their source laid out
> such that they cannot actually be used in-place, only after they've
> been installed.

A data point:  I've been using "setup.py develop" with twisted
for over a year now, and it works great for me -- definitely one of
my favorite features of setuptools.  I use it with an svn checkout of
twisted so I can always use the latest -- especially nice with twisted
because of its high quality management (trunk is almost never broken).
That way I can easily track the latest version -- just do an svn
update, no further installation required.

When I want to try a vcs checkout of an immature package and its setup.py
uses distutils instead of setuptools, I'll try changing its setup.py to
use setuptools (usually a one or two line replacement of the import)
and see if "setup.py develop" works, and it almost always does.

Steve

From exarkun at divmod.com  Fri May 22 19:29:02 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 22 May 2009 13:29:02 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: <4A16D894.4040200@nasa.gov>
Message-ID: <20090522172902.21531.921327089.divmod.quotient.24525@henry.divmod.com>

On Fri, 22 May 2009 12:53:40 -0400, Stephen Waterbury <stephen.c.waterbury at nasa.gov> wrote:
> [snip]
>
>A data point:  I've been using "setup.py develop" with twisted
>for over a year now, and it works great for me -- definitely one of
>my favorite features of setuptools.  I use it with an svn checkout of
>twisted so I can always use the latest -- especially nice with twisted
>because of its high quality management (trunk is almost never broken).
>That way I can easily track the latest version -- just do an svn
>update, no further installation required.
>
>When I want to try a vcs checkout of an immature package and its setup.py
>uses distutils instead of setuptools, I'll try changing its setup.py to
>use setuptools (usually a one or two line replacement of the import)
>and see if "setup.py develop" works, and it almost always does.

Hi Steve,

Yea, I would expect it to work alright with Twisted.  The project I
ran into this with is clearsilver, which it doesn't work with, and
another project I had in mind when I wrote about this problem is
pyOpenSSL, which is also laid out in its repository such that it
cannot be used in-place. :(

Jean-Paul

From pje at telecommunity.com  Fri May 22 21:29:44 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 22 May 2009 15:29:44 -0400
Subject: [Distutils] [Trac] Re: Help packaging plugin : Source
	distribution
In-Reply-To: <24ea26600905221043s11522661gc661aea8d8eb75c@mail.gmail.com
 >
References: <24ea26600905200801o22c51b9ex4a4a121e33b6a6d2@mail.gmail.com>
	<20090520150928.GE6761@openplans.org>
	<24ea26600905200915g11ffbf8ah2fdaee18a57b8c93@mail.gmail.com>
	<20090521205347.9854F3A40D9@sparrow.telecommunity.com>
	<24ea26600905221043s11522661gc661aea8d8eb75c@mail.gmail.com>
Message-ID: <20090522192704.D61A63A40D9@sparrow.telecommunity.com>

At 12:43 PM 5/22/2009 -0500, Olemis Lang wrote:
>On Thu, May 21, 2009 at 3:56 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 11:15 AM 5/20/2009 -0500, Olemis Lang wrote:
> >>
> >> On Wed, May 20, 2009 at 10:09 AM, Jeff Hammel <jhammel at openplans.org>
> >> wrote:
> >> >
> >> > You have to verbosely include them with package data.  IMHO, this is a
> >> > setuptools bug.  E.g.:
> >> >
> >> >      package_data={ 'geotrac': ['templates/*', 'htdocs/*'] },
> >> >      include_package_data=True,
> >> >
> >> > This is probably not quite right :(, but you get the idea.
> >>
> >> I just tried this and doesnt include files for sdist ... :(
> >>
> >> Looking forward for further hints : maybe it's just something I'm
> >> doing wrong, dont know, but eggs are ok so I'm not sure
> >
> > If you are not using a revision control system plugin, you will 
> have to list
> > sdist files in MANIFEST.in;
>
>Thanks, I included files in MANIFEST.in now I can distribute source
>packages for TracGViz [1]_ & TracPyTppTheme [2]_ so that EGGs can be
>built for different Py versions or whatever ...
>
>So this is the way !
>
>PS: BTW , I dont get the idea about specifying things twice (I mean
>package_data & MANIFEST.in ;) Is there a ?logical? reason for this ? I
>mean why sdist doesnt look for the data specified in package_data ? Or
>alternately, what is the reason for having MANIFEST.in too ?

That's what distutils does.  setuptools improves on it by adding 
revision control support.  But if you don't use a revision control 
plugin or some other sort of file finder, you have to do it the old 
distutils way.


From pje at telecommunity.com  Fri May 22 21:32:39 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 22 May 2009 15:32:39 -0400
Subject: [Distutils] Side by side debug/non-debug eggs
In-Reply-To: <20090522160015.21531.479508022.divmod.quotient.24510@henry
	.divmod.com>
References: <20090522153112.C3C343A40D9@sparrow.telecommunity.com>
	<20090522160015.21531.479508022.divmod.quotient.24510@henry.divmod.com>
Message-ID: <20090522192956.5A71D3A40D9@sparrow.telecommunity.com>

At 12:00 PM 5/22/2009 -0400, Jean-Paul Calderone wrote:
>On Fri, 22 May 2009 11:33:54 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>>At 09:45 AM 5/22/2009 -0400, Jean-Paul Calderone wrote:
>>>Hello all,
>>>
>>>Prior to eggs, if I wanted to have debug and non-debug versions of an
>>>extension module available, I could build and install the extension
>>>twice and the modules would happily co-exist, for example:
>>>
>>>
>>>  $ python setup.py build install
>>>  ...
>>>  $ python-dbg setup.py build install
>>>  ...
>>>  $ .../site-packages/twisted/python$ ls *.so
>>>  _epoll_d.so  _epoll.so
>>>  $ .../site-packages/twisted/python$
>>>If I then ran a debug build of Python, the debug extension would be loaded.
>>>When I ran a normal build of Python, the non-debug extension would 
>>>be loaded.
>>>However, with eggs, each new egg completely replaces the last egg.  There
>>>seems to be no possibility for side-by-side builds.  Am I overlooking a
>>>feature of eggs or of setuptools?
>>
>>Have you tried "setup.py develop"?  This builds extensions in-place 
>>in the source tree and adds the source tree to sys.path.
>
>Aware of it, but I haven't tried it.  In this use, it sounds very
>similar to "setup.py build_ext -i", which also partially solves the
>problem.  Unfortunately, some packages have their source laid out
>such that they cannot actually be used in-place, only after they've
>been installed.

The only other thing I can think of is installing in 
backward-compatibility mode, i.e. "setup.py install 
--single-version-externally-managed" (with a filename given to --record).


From ziade.tarek at gmail.com  Mon May 25 12:29:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 25 May 2009 12:29:44 +0200
Subject: [Distutils] PEP 376 up-to-date
Message-ID: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>

Hello,

I have updated PEP 376, mainly with Phillip's feedback, and changed
the API section so it looks like the current prototype.

- PEP : http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup
- prototype code : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

I'll wait for a new round of feedbacks, Thanks !

Tarek

-- 
Tarek Ziad? | http://ziade.org

From floris.bruynooghe at gmail.com  Mon May 25 23:42:05 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 25 May 2009 22:42:05 +0100
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
Message-ID: <20090525214205.GA26482@laurie.devork>

On Mon, May 25, 2009 at 12:29:44PM +0200, Tarek Ziad? wrote:
> - prototype code : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

def disable_cache():
    global _CACHE_ENABLED
    _CACHE_ENABLED = True


/me thinks that might not be right...


Not sure if this is supposed to work in a situation where multiple
threads are using this API at the same time.  But for that case the
simple cache management might be too limiting (enable and disable are
global) and refactoring the functions which use the cache into an
object that can be instantiated with a flag whether to use the global
cache or not might be better.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Tue May 26 05:59:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 25 May 2009 23:59:52 -0400
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.co
 m>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
Message-ID: <20090526035707.03D2F3A404D@sparrow.telecommunity.com>

At 12:29 PM 5/25/2009 +0200, Tarek Ziad? wrote:
>Hello,
>
>I have updated PEP 376, mainly with Phillip's feedback, and changed
>the API section so it looks like the current prototype.
>
>- PEP : http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup
>- prototype code : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py
>
>I'll wait for a new round of feedbacks, Thanks !

* There's no reason for anything shown in the module to be private; 
is_egg_info() and egg_info_dirs() would be useful API functions, and 
the additions to DistributionMetadata might as well go in distutils

* Better yet, move DistributionMetadata to pkgutil and have 
distutils.dist import it; that way you won't get tons of distutils.* 
imports any time you import pkgutil.

* get_egg_infos() should take a pathlist argument, which if None can 
default to sys.path.

* Project name normalization and case insensitive comparison is still 
not implemented

* File path normalization (absolutizing, case-normalization, 
de-cross-platforming, etc.) is not implemented

* There is extensive coupling both to sys.path and to the global 
cache; in particular, the owns() operation properly belongs to an 
object representing a *directory*, rather than an object representing 
an individual project.

Note too that such "directory" objects would then also be an 
appropriate cache target, and a suitable home for egg_info_dirs() and 
get_file_users() operations.  This also simplifies get_egg_infos, 
since it would simply either retrieves directories from the cache or 
creates them and optionally caches them, without also meddling in the 
details of cache contents.

Also, an application that wishes to do so can simply create directory 
objects of its own and manage their lifecycle accordingly.

Personally, I would also make an object for a collection of 
directories, but you could possibly get away without it.

(At this point, I have only briefly skimmed the updated PEP; I'll 
take a closer look at it once the API/implementation gets more settled.)


From ziade.tarek at gmail.com  Tue May 26 10:46:39 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 26 May 2009 10:46:39 +0200
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <20090525214205.GA26482@laurie.devork>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<20090525214205.GA26482@laurie.devork>
Message-ID: <94bdd2610905260146t6fea63bt2ab7658b8cb161d3@mail.gmail.com>

On Mon, May 25, 2009 at 11:42 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> Not sure if this is supposed to work in a situation where multiple
> threads are using this API at the same time. ?But for that case the
> simple cache management might be too limiting (enable and disable are
> global) and refactoring the functions which use the cache into an
> object that can be instantiated with a flag whether to use the global
> cache or not might be better.
>

Right, I havn't thought about that. While I don't have a clear idea on
the multi-threading aspect yet,
I think Phillip's proposal on having Directories might help a lot

Regards
Tarek

From ziade.tarek at gmail.com  Tue May 26 11:01:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 26 May 2009 11:01:02 +0200
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <20090526035707.03D2F3A404D@sparrow.telecommunity.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<20090526035707.03D2F3A404D@sparrow.telecommunity.com>
Message-ID: <94bdd2610905260201n63077f41wfae824df13400176@mail.gmail.com>

2009/5/26 P.J. Eby <pje at telecommunity.com>:
> * There is extensive coupling both to sys.path and to the global cache; in
> particular, the owns() operation properly belongs to an object representing
> a *directory*, rather than an object representing an individual project.
>
> Note too that such "directory" objects would then also be an appropriate
> cache target, and a suitable home for egg_info_dirs() and get_file_users()
> operations. ?This also simplifies get_egg_infos, since it would simply
> either retrieves directories from the cache or creates them and optionally
> caches them, without also meddling in the details of cache contents.
>
> Also, an application that wishes to do so can simply create directory
> objects of its own and manage their lifecycle accordingly.
>
> Personally, I would also make an object for a collection of directories, but
> you could possibly get away without it.

I can see the benefit of having such objects, I'll work on that. (and
other points I didn't mention)

Just one point though:

- do we want to scan egg-info directories that are *directly* added in
sys.path ?

  If we do so,  using directories objects would require looking for
the parent directory to create a directory
  object that hold each one of those "top" egg-info directories.  This
seems like a bad idea to me.
  Or maybe a specific global directory object could handle those top
egg-info dirs. Its path would be
  virtual, and it would use "sys.path" instead of "os.listdir()" to
list the elements it works it.


Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Tue May 26 15:51:16 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 26 May 2009 09:51:16 -0400
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <94bdd2610905260201n63077f41wfae824df13400176@mail.gmail.co
 m>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<20090526035707.03D2F3A404D@sparrow.telecommunity.com>
	<94bdd2610905260201n63077f41wfae824df13400176@mail.gmail.com>
Message-ID: <20090526134832.6B7CA3A40D9@sparrow.telecommunity.com>

At 11:01 AM 5/26/2009 +0200, Tarek Ziad? wrote:
>Just one point though:
>
>- do we want to scan egg-info directories that are *directly* added in
>sys.path ?

No.  Such a directory does not represent an importable egg on sys.path.

If you mean an .egg file or directory that's directly on sys.path, 
then that's a bit different.  Are you planning to support those?  In 
that event you would need another EggInfo subclass anyway, and you 
could have the directory (or "PathEntry", perhaps?) constructor 
simply wrap a single such EggInfo in that event.

Or, you can handle it the way pkg_resources does, which is to convert 
target paths into PEP 302 importer objects, and then use the 
importer's type to determine what sort of handler should be used for it.


From setuptools at bugs.python.org  Tue May 26 23:06:37 2009
From: setuptools at bugs.python.org (Jonathan Cervidae)
Date: Tue, 26 May 2009 21:06:37 +0000
Subject: [Distutils] [issue72] easy_install violates SELinux policies
In-Reply-To: <1243371997.17.0.0475707034723.issue72@psf.upfronthosting.co.za>
Message-ID: <1243371997.17.0.0475707034723.issue72@psf.upfronthosting.co.za>


New submission from Jonathan Cervidae <jonathan.cervidae at gmail.com>:

I get AVC's aftef installing egg's with easy_install. The AVS are that the
files are mislabeled as:

unconfined_u:object_r:user_tmp_t:s0

When I we want them to be:

system_u:object_r:lib_t:s0

It stops python from accessing the files. I haven't looked at the source
code, but those contexts suggest that easy_install is downloading the egg
file to a temp directory then moving (not copying) it from the temp dir to
/usr/lib/python<VERSION>/site-packages.

This is clearly a good thing as unnecessary copying should be avoid and
it's easy to fix too.

I notice there is a python module called selinux installed on my system. So
you need to try and import that. If you succeed you can try and restore the
context after moving it to the new location and just ignore any error on that too.

----------
files: fix.txt
messages: 292
nosy: jcervidae
priority: bug
status: unread
title: easy_install violates SELinux policies
Added file: http://bugs.python.org/setuptools/file53/fix.txt

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue72>
_______________________________________________
-------------- next part --------------
[root at jaydee tmp]# python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, tempfile, shutil, subprocess
>>> import selinux # try/except block around this to see if the issue will happen
>>> temp_fd, a_temp_file = tempfile.mkstemp()
>>> shutil.move(a_temp_file,'/usr/lib/python2.5/site-packages/broken.egg')
>>> # Don't know how to do this in Python
... subprocess.call(("restorecon", "-nv", "/usr/lib/python2.5/site-packages/broken.egg"))
restorecon reset /usr/lib/python2.5/site-packages/broken.egg context unconfined_u:object_r:user_tmp_t:s0->system_u:object_r:lib_t:s0
0
>>> # But I do know how to fix them in python
... selinux.restorecon("/usr/lib/python2.5/site-packages/broken.egg")
>>> # No more output because the context is now correct
... subprocess.call(("restorecon", "-nv", "/usr/lib/python2.5/site-packages/broken.egg"))
0
>>> # Calling it if context was already correct because user had an insane selinux policy does no harm either
... selinux.restorecon("/usr/lib/python2.5/site-packages/broken.egg")
>>> os.close(temp_fd)
>>> os.remove("/usr/lib/python2.5/site-packages/broken.egg")
>>> quit()

 	  	 

From ziade.tarek at gmail.com  Wed May 27 10:27:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 27 May 2009 10:27:12 +0200
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <20090526134832.6B7CA3A40D9@sparrow.telecommunity.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<20090526035707.03D2F3A404D@sparrow.telecommunity.com>
	<94bdd2610905260201n63077f41wfae824df13400176@mail.gmail.com>
	<20090526134832.6B7CA3A40D9@sparrow.telecommunity.com>
Message-ID: <94bdd2610905270127y28bc2841l89f60d9005c16f7b@mail.gmail.com>

On Tue, May 26, 2009 at 3:51 PM, P.J. Eby <pje at telecommunity.com> wrote:
> No. ?Such a directory does not represent an importable egg on sys.path.
>

ok


> If you mean an .egg file or directory that's directly on sys.path, then
> that's a bit different.

No I was talking about the .egg-info ones.

>?Are you planning to support those? ?In that event
> you would need another EggInfo subclass anyway, and you could have the
> directory (or "PathEntry", perhaps?) constructor simply wrap a single such
> EggInfo in that event.

I didn't plan to support other formats because I am in favor of a
unique format.
I do think having several formats lead to confusion.

Tarek

-- 
Tarek Ziad? | http://ziade.org

From thomas at thomas-lotze.de  Wed May 27 14:56:08 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Wed, 27 May 2009 14:56:08 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
Message-ID: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>

Hi,

I've been implementing a download API for buildout and have reached the
point where I'd like to try using it for downloading the base config files
referenced from a buildout config by the 'extends' option. This would have
the benefit of allowing to cache those config files and being able to run
a buildout using that feature while being offline.

However, I'd like to discuss two points about doing this:

- I think buildout should not unconditionally use a cached copy of a base
  configuration file. While it is sensible in other use cases to never
  access the network if a file is found in the cache, base configs can
  be expected to change and should IMHO be taken from the cache only when
  offline mode is active or an attempt at downloading failed.

- The download cache to be used is configured as one of the options that
  are read from either ~/.buildout/default.cfg or buildout.cfg and their
  respective bases. In order to keep things simple, I'd suggest to use a
  download cache only if specified directly inside any of these two files,
  and ignore download-cache options in any downloaded base configs. This
  would be an exception to how options read from config files are
  combined, but one I deem worthwhile.

  I'd allow for configuring and using different download caches in those
  two files, though. That way, bases of the user's default config can be
  cached centrally while allowing to keep project-related stuff in a
  different place.

Opinions?

-- 
Thomas




From david.lyon at preisshare.net  Thu May 28 07:19:56 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 28 May 2009 01:19:56 -0400
Subject: [Distutils] pypi Package Testing...
In-Reply-To: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
Message-ID: <f924b071972615dceaa73580f53402fa@preisshare.net>


Hi All,

I've been doing some internal testing on my package manager gui
and with both pip and easy_install and I want to test more...

Under windows... what I notice is that some packages install
under easy_install and other not under pip.. and also vice
versa (I think...) 

So what I want to do next is to actually test that I can
install/deinstall every package on pypi using either pip 
or easy_install on both ubuntu and windows under a few
few different versions of python.. say 2.3, 2.5 and 2.6

Is there anybody that's interested in helping? or receiving
the results of these tests?

David



From sdouche at gmail.com  Thu May 28 11:44:43 2009
From: sdouche at gmail.com (Sebastien Douche)
Date: Thu, 28 May 2009 11:44:43 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
Message-ID: <5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>

On Wed, May 27, 2009 at 14:56, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Hi,

Hi Thomas :)
First at all, thank you for your work.

> I've been implementing a download API for buildout and have reached the
> point where I'd like to try using it for downloading the base config files
> referenced from a buildout config by the 'extends' option. This would have
> the benefit of allowing to cache those config files and being able to run
> a buildout using that feature while being offline.

Do you talk about index and find-links too? And for the bootstrap? I
search a scenario to deploy a buildout fully offline.

> - I think buildout should not unconditionally use a cached copy of a base
> ?configuration file. While it is sensible in other use cases to never
> ?access the network if a file is found in the cache, base configs can
> ?be expected to change and should IMHO be taken from the cache only when
> ?offline mode is active or an attempt at downloading failed.

Right. You think about "offline" option or "install-from-cache"
option? offline seems to be a good candidate.

> - The download cache to be used is configured as one of the options that
> ?are read from either ~/.buildout/default.cfg or buildout.cfg and their
> ?respective bases. In order to keep things simple, I'd suggest to use a
> ?download cache only if specified directly inside any of these two files,
> ?and ignore download-cache options in any downloaded base configs. This
> ?would be an exception to how options read from config files are
> ?combined, but one I deem worthwhile.

Good for me.

> Opinions?

Hmm, just an idea on the same area: caching only some packages (not
only "all" or "none"). Surely out of your scope...


-- 
Sebastien Douche <sdouche at gmail.com>

From ziade.tarek at gmail.com  Thu May 28 12:21:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 28 May 2009 12:21:16 +0200
Subject: [Distutils] pypi Package Testing...
In-Reply-To: <f924b071972615dceaa73580f53402fa@preisshare.net>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<f924b071972615dceaa73580f53402fa@preisshare.net>
Message-ID: <94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>

On Thu, May 28, 2009 at 7:19 AM, David Lyon <david.lyon at preisshare.net> wrote:
>
> Hi All,
>
> I've been doing some internal testing on my package manager gui
> and with both pip and easy_install and I want to test more...
>
> Under windows... what I notice is that some packages install
> under easy_install and other not under pip.. and also vice
> versa (I think...)
>
> So what I want to do next is to actually test that I can
> install/deinstall every package on pypi using either pip
> or easy_install on both ubuntu and windows under a few
> few different versions of python.. say 2.3, 2.5 and 2.6
>
> Is there anybody that's interested in helping? or receiving
> the results of these tests?

I am interested in receiving your results.

Notice that I have started a similar project to prevent regressions in
Distutils.

I am running "smoke test" to see if the current trunk is able to create
the same releases than previous Python versions.

It checkout a project and run "python setup.py sdist" with various
Python interpreter, then compare the output.

Right now I am running this only on numpy and a list of trusted tarballs
for security reasons.

It's here : http://buildbot.ziade.org/waterfall

Tarek


>
> David
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu May 28 12:55:30 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 28 May 2009 06:55:30 -0400
Subject: [Distutils] pypi Package Testing...
In-Reply-To: <94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<f924b071972615dceaa73580f53402fa@preisshare.net>
	<94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
Message-ID: <e1ee33321aebdd2c78c81343fe4d0882@preisshare.net>

On Thu, 28 May 2009 12:21:16 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I am interested in receiving your results.

No problem...

Well I'll keep you posted...

There's a few packages to download.. so it won't be ready by dinner...

> It's here : http://buildbot.ziade.org/waterfall

Yes.. looks good....

I was planning to use buildbot also...

It's a good tool

David


From sdouche at gmail.com  Thu May 28 13:03:55 2009
From: sdouche at gmail.com (Sebastien Douche)
Date: Thu, 28 May 2009 13:03:55 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de> 
	<5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>
Message-ID: <5e1183fa0905280403x30efea64v42e4374ec1168c2c@mail.gmail.com>

On Thu, May 28, 2009 at 11:44, Sebastien Douche <sdouche at gmail.com> wrote:
>> Opinions?
>
> Hmm, just an idea on the same area: caching only some packages (not
> only "all" or "none"). Surely out of your scope...

Oups, missing primary needs: show the location of a download (cache,
url of the website,...).


-- 
Sebastien Douche <sdouche at gmail.com>

From david.lyon at preisshare.net  Thu May 28 13:09:41 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 28 May 2009 07:09:41 -0400
Subject: [Distutils] pypi Package Testing...
In-Reply-To: <94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<f924b071972615dceaa73580f53402fa@preisshare.net>
	<94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
Message-ID: <121eb9ca105c96c97434b7cc390dc373@preisshare.net>

On Thu, 28 May 2009 12:21:16 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I am interested in receiving your results.

First test... install buildbot from pypi on Python 2.6 windows:

===================================================================
Easy Install
Running installer ... C:\Python26\scripts\easy_install.exe buildbot
Searching for buildbot
Reading http://pypi.python.org/simple/buildbot/

Reading http://buildbot.sourceforge.net/

Reading http://buildbot.net/
Reading
https://sourceforge.net/project/showfiles.php?group_id=73177&package_id=73212&release_id=665379

Best match: buildbot 0.7.10p1
Downloading
http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.10p1.zip#md5=51f744937d65635810aa6496eb2bf078

ERRORS: build\bdist.win32\egg\buildbot\ec2buildslave.py:90: SyntaxWarning:
assertion is always true, perhaps remove parentheses?
  assert (aws_id_file_path is None,
build\bdist.win32\egg\buildbot\ec2buildslave.py:93: SyntaxWarning:
assertion is always true, perhaps remove parentheses?
  assert (secret_identifier is not None,

Processing buildbot-0.7.10p1.zip
Running buildbot-0.7.10p1\setup.py -q bdist_egg --dist-dir
c:\docume~1\david\locals~1\temp\easy_install-fokujq\buildbot-0.7.10p1\egg-dist-tmp-hdicbb
zip_safe flag not set; analyzing archive contents...
buildbot.clients.debug: module references __file__
buildbot.scripts.runner: module references __file__
buildbot.status.web.baseweb: module references __file__
buildbot.status.web.grid: module references __file__
buildbot.test.runutils: module references __file__
buildbot.test.test_maildir: module references __file__
buildbot.test.test_mailparse: module references __file__
buildbot.test.test_shell: module references __file__
buildbot.test.test_slavecommand: module references __file__
No eggs found in
c:\docume~1\david\locals~1\temp\easy_install-fokujq\buildbot-0.7.10p1\egg-dist-tmp-hdicbb
(setup script problem?)

===================================================================

From jim at zope.com  Thu May 28 15:07:46 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 May 2009 09:07:46 -0400
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
Message-ID: <A642C7CF-12E1-415E-BB05-8E46E5D11D4F@zope.com>


On May 27, 2009, at 8:56 AM, Thomas Lotze wrote:

> Hi,
>
> I've been implementing a download API for buildout and have reached  
> the
> point where I'd like to try using it for downloading the base config  
> files
> referenced from a buildout config by the 'extends' option. This  
> would have
> the benefit of allowing to cache those config files and being able  
> to run
> a buildout using that feature while being offline.
>
> However, I'd like to discuss two points about doing this:
>
> - I think buildout should not unconditionally use a cached copy of a  
> base
>  configuration file. While it is sensible in other use cases to never
>  access the network if a file is found in the cache, base configs can
>  be expected to change and should IMHO be taken from the cache only  
> when
>  offline mode is active or an attempt at downloading failed.

Absolutely.


> - The download cache to be used is configured as one of the options  
> that
>  are read from either ~/.buildout/default.cfg or buildout.cfg and  
> their
>  respective bases. In order to keep things simple, I'd suggest to  
> use a
>  download cache only if specified directly inside any of these two  
> files,
>  and ignore download-cache options in any downloaded base configs.  
> This
>  would be an exception to how options read from config files are
>  combined, but one I deem worthwhile.

-1 on allowing the settings in only these 2 files.

>  I'd allow for configuring and using different download caches in  
> those
>  two files, though. That way, bases of the user's default config can  
> be
>  cached centrally while allowing to keep project-related stuff in a
>  different place.
>
> Opinions?

The solution seems too complicated to me. Too many special rules.

Maybe there should be a separate settinf for this, like extends-cache  
or something.

Jim

--
Jim Fulton
Zope Corporation



From a.cavallo at mailsnare.com  Thu May 28 15:42:36 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Thu, 28 May 2009 14:42:36 +0100
Subject: [Distutils] pypi Package Testing...
In-Reply-To: <121eb9ca105c96c97434b7cc390dc373@preisshare.net>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
	<121eb9ca105c96c97434b7cc390dc373@preisshare.net>
Message-ID: <200905281442.37197.a.cavallo@mailsnare.com>

On Thursday 28 May 2009 12:09:41 David Lyon wrote:
> On Thu, 28 May 2009 12:21:16 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
>
> wrote:
> > I am interested in receiving your results.
>
> First test... install buildbot from pypi on Python 2.6 windows:

I'm working on these issues using the susebuild server as continuous 
integration build server.

There's a python intepreter based on a recent svn source usable installed 
under /opt:

http://download.opensuse.org/repositories/home:/cavallo71:/python-opt/

I hope this helps.

I'm about to complete a complete continuous build regression test based on an 
automated build using the suse build server including unittest based smoke 
tests.

It should be ready by the end of today.
Regards,
Antonio


From Svenn.Helge.Grindhaug at bccs.uib.no  Thu May 28 09:57:39 2009
From: Svenn.Helge.Grindhaug at bccs.uib.no (Svenn Helge Grindhaug)
Date: Thu, 28 May 2009 09:57:39 +0200
Subject: [Distutils] [Buildout] Include-dirs
Message-ID: <200905280957.40097.svenn@bccs.uib.no>

Hi,

I'm trying to do something similar to the custom egg building in the tutorial, 
http://www.buildout.org/docs/tutorial.html?highlight=location#custom-egg-
building

I have the following files:

setup.py
*********
from setuptools import setup, find_packages

setup(
    name='myproject',
    version='3.3.0',
    packages=find_packages('src'),
    package_dir = {'': 'src'},
    include_package_data = True,
    install_requires=['setuptools',
                      'numpy',
                      'ScientificPython'],
    zip_safe = False,
    )

buildout.cfg
*************

[buildout]
parts = numpy
		ScientificPython 
		mypy 

[numpy]
recipe = zc.recipe.egg
find-links = 
http://sunet.dl.sourceforge.net/sourceforge/numpy/numpy-1.3.0.tar.gz
eggs= numpy

[ScientificPython]
recipe = zc.recipe.egg:custom
egg=ScientificPython
find-links = 
http://sourcesup.cru.fr/frs/download.php/2372/ScientificPython-2.9.0.tar.gz
include-dirs = ${numpy:location}/numpy/core/include

[mypy]
recipe = zc.recipe.egg:script
eggs = numpy
       ScientificPython
interpreter = python

------------

The ScientificPython package need some header files located in the numpy 
package/egg. I get an error when using ${numpy:location}

Error: Referenced option does not exist: numpy location

When I use the absolute path to the include dir it works. What am I doing 
wrong?

Best regards,

Svenn.

From thomas at thomas-lotze.de  Thu May 28 18:08:00 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 28 May 2009 18:08:00 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <5e1183fa0905280403x30efea64v42e4374ec1168c2c@mail.gmail.com>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
	<5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>
	<5e1183fa0905280403x30efea64v42e4374ec1168c2c@mail.gmail.com>
Message-ID: <20090528180800.3050de25@krusty.ws.whq.gocept.com>

Sebastien Douche <sdouche at gmail.com> schrieb:

> Oups, missing primary needs: show the location of a download (cache,
> url of the website,...).

Good idea, I'm going to add loggin anyway. However, I think this
particular information should only be printed in verbose mode.

-- 
Viele Gr??e,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090528/fd51ae41/attachment.pgp>

From thomas at thomas-lotze.de  Thu May 28 18:05:40 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 28 May 2009 18:05:40 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
	<5e1183fa0905280244w186e3227ne2f9f7eb821c486@mail.gmail.com>
Message-ID: <20090528180540.51ea11cf@krusty.ws.whq.gocept.com>

Sebastien Douche <sdouche at gmail.com> schrieb:

> > I've been implementing a download API for buildout and have reached the
> > point where I'd like to try using it for downloading the base config files
> > referenced from a buildout config by the 'extends' option. This would have
> > the benefit of allowing to cache those config files and being able to run
> > a buildout using that feature while being offline.
> 
> Do you talk about index and find-links too? And for the bootstrap? I
> search a scenario to deploy a buildout fully offline.

Those would be useful indeed. However, afaics the respective code
doesn't have access to the zc.buildout package (easy-install downloads
use setuptools and at bootstrap time, zc.buildout hasn't even been
downloaded yet).

> Right. You think about "offline" option or "install-from-cache"
> option? offline seems to be a good candidate.

I've implemented three ways of handling the cache:

- don't use it at all, fail if offline mode is on or the resource
  cannot be accessed

- use a cached copy of a file unconditionally if one is found in the
  cache

- try to download a resource if not in offline mode, only fall back to
  using the cache if this fails

I was talking about using the third option when fetching configuration.
Other client code (i.e. recipes) may choose freely how to use the
cache and whether to make that decision based on a configuration option.

> Hmm, just an idea on the same area: caching only some packages (not
> only "all" or "none"). Surely out of your scope...

This is out of the scope of zc.buildout's core IMO. A particular recipe
might implement any algorithm to decide how to use the cache when
downloading one package or another. I even think this would be too
complex for a core recipe such as zc.recipe.egg.

-- 
Viele Gr??e,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090528/05964c49/attachment.pgp>

From thomas at thomas-lotze.de  Thu May 28 17:51:30 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 28 May 2009 17:51:30 +0200
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <A642C7CF-12E1-415E-BB05-8E46E5D11D4F@zope.com>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
	<A642C7CF-12E1-415E-BB05-8E46E5D11D4F@zope.com>
Message-ID: <20090528175130.15159d74@krusty.ws.whq.gocept.com>

Jim Fulton <jim at zope.com> schrieb:

> > - I think buildout should not unconditionally use a cached copy of a  
> > base
> >  configuration file. While it is sensible in other use cases to never
> >  access the network if a file is found in the cache, base configs can
> >  be expected to change and should IMHO be taken from the cache only  
> > when
> >  offline mode is active or an attempt at downloading failed.
> 
> Absolutely.

I've added the ability to use the cache in this way to the download
API in the meantime.

> > - The download cache to be used is configured as one of the options  
> > that
> >  are read from either ~/.buildout/default.cfg or buildout.cfg and  
> > their
> >  respective bases. In order to keep things simple, I'd suggest to  
> > use a
> >  download cache only if specified directly inside any of these two  
> > files,
> >  and ignore download-cache options in any downloaded base configs.  
> > This
> >  would be an exception to how options read from config files are
> >  combined, but one I deem worthwhile.
> 
> -1 on allowing the settings in only these 2 files.

OK, avoiding these particular special rules is fine with me.

> The solution seems too complicated to me. Too many special rules.
> 
> Maybe there should be a separate settinf for this, like extends-cache  
> or something.

Would it be OK for you to special-case that new option? The problem
that needs to be solved when downloading base configs is that in order
for a particular config file to be taken from the cache if necessary,
the cache location must be known beforehand, so it shouldn't be
possible for the cache location to depend on the contents of the config
file in question. I cannot see another way to achieve this but to read
the extends-cache location from the root config file (either
~/.buildout/default.cfg or buildout.cfg) exclusively.

This wouldn't account for an root config file that is specified to
buildout on the command line and needs to be downloaded itself, but I
think it's OK to assume being online when specifying an online resource
in such an explicit way.

-- 
Viele Gr??e,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090528/98c1fcd0/attachment.pgp>

From jim at zope.com  Thu May 28 22:53:48 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 May 2009 16:53:48 -0400
Subject: [Distutils] Buildout: caching of base configuration ('extends')
In-Reply-To: <20090528175130.15159d74@krusty.ws.whq.gocept.com>
References: <pan.2009.05.27.12.56.07.469073@ID-174572.user.uni-berlin.de>
	<A642C7CF-12E1-415E-BB05-8E46E5D11D4F@zope.com>
	<20090528175130.15159d74@krusty.ws.whq.gocept.com>
Message-ID: <A8BAAC3A-CD45-4D44-AC79-1418134D7E98@zope.com>


On May 28, 2009, at 11:51 AM, Thomas Lotze wrote:

> Jim Fulton <jim at zope.com> schrieb:
>
>>> - I think buildout should not unconditionally use a cached copy of a
>>> base
>>> configuration file. While it is sensible in other use cases to never
>>> access the network if a file is found in the cache, base configs can
>>> be expected to change and should IMHO be taken from the cache only
>>> when
>>> offline mode is active or an attempt at downloading failed.
>>
>> Absolutely.
>
> I've added the ability to use the cache in this way to the download
> API in the meantime.
>
>>> - The download cache to be used is configured as one of the options
>>> that
>>> are read from either ~/.buildout/default.cfg or buildout.cfg and
>>> their
>>> respective bases. In order to keep things simple, I'd suggest to
>>> use a
>>> download cache only if specified directly inside any of these two
>>> files,
>>> and ignore download-cache options in any downloaded base configs.
>>> This
>>> would be an exception to how options read from config files are
>>> combined, but one I deem worthwhile.
>>
>> -1 on allowing the settings in only these 2 files.
>
> OK, avoiding these particular special rules is fine with me.
>
>> The solution seems too complicated to me. Too many special rules.
>>
>> Maybe there should be a separate settinf for this, like extends-cache
>> or something.
>
> Would it be OK for you to special-case that new option?

Yup


> The problem
> that needs to be solved when downloading base configs is that in order
> for a particular config file to be taken from the cache if necessary,
> the cache location must be known beforehand, so it shouldn't be
> possible for the cache location to depend on the contents of the  
> config
> file in question. I cannot see another way to achieve this but to read
> the extends-cache location from the root config file (either
> ~/.buildout/default.cfg or buildout.cfg) exclusively.
>
> This wouldn't account for an root config file that is specified to
> buildout on the command line and needs to be downloaded itself, but I
> think it's OK to assume being online when specifying an online  
> resource
> in such an explicit way.


Agreed.

Jim

--
Jim Fulton
Zope Corporation



From brian at vanguardistas.net  Fri May 29 11:12:02 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Fri, 29 May 2009 11:12:02 +0200
Subject: [Distutils] Low Level API for translating distutils/setuptools
	metatdata to	Debian metadata
Message-ID: <20090529091202.GB22620@Boo.lan>

Hi,

I've just published a very small library that does three things so far:

    * Provides a mapping between python project names and Debian
      binary/source package names
    * Converts setuptools versions to Debian versions while maintaining
      sort order
    * Can introspect an .egg-info directory to figure out the Debian
      dependencies for use in the debian/control file. It can also
      handle/understand extras (I Hope!)

It's basically an attempt to find a common ground between all the
projects doing automated Python->Debian packaging. I have a feeling
everyone is re-implementing this code all the time.

The code is here:
    
    http://pypi.python.org/pypi/van.pydeb/

And an example of direct use in a packaging situation with complex dependencies is here:

    http://svn.debian.org/viewsvn/pkg-zope/zope.component/trunk/debian/rules?view=markup

-- 
Brian Sutherland

From ziade.tarek at gmail.com  Fri May 29 11:14:25 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 29 May 2009 11:14:25 +0200
Subject: [Distutils] Version comparison - round 2
Message-ID: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>

Hello,

I have gathered all the documentation for the versioning lib in a
single document,

  http://bitbucket.org/tarek/distutilsversion/src/tip/README.txt

It explains how the existing tools work and how "verlib", that we
built during Pycon, works.

It still needs to be completed by some people that were present, but a
new feedback round
is more than welcome.

The goal here is to provide a version comparison standard to be
included in Distutils.

We do need such a standard to be able to include the
"install_requires" metadata as planned
(that will be a change in PEP 345)  to be able to provide a standard
for version comparisons.

Cheers
Tarek
-- 
Tarek Ziad? | http://ziade.org

From purpleidea at gmail.com  Thu May 28 22:46:06 2009
From: purpleidea at gmail.com (James)
Date: Thu, 28 May 2009 16:46:06 -0400
Subject: [Distutils] Patch for clean command
Message-ID: <c229b2e40905281346q5491c472pd9512a885e4509e9@mail.gmail.com>

Hi,
I'm certaintly new to distutils and setuptools, however I figured I'd
send in this patch, and either it will get merged because it's a great
idea or someone will perhaps tell me why this doesn't exist already.
(or maybe it does and i can't find it)
In any case, it adds the pyc option to the clean command so that the
.pyc can be removed on request. Personally i'll have a [clean] pyc=1
option in my setup.cfg, but that's for my convenience.
cheers,

_J


[attached: patch and modified file]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: clean.py.patch
Type: text/x-patch
Size: 396 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090528/7c6c58ef/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: distutils-clean.py
Type: text/x-python
Size: 3272 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090528/7c6c58ef/attachment.py>

From ziade.tarek at gmail.com  Fri May 29 16:44:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 29 May 2009 16:44:47 +0200
Subject: [Distutils] Patch for clean command
In-Reply-To: <c229b2e40905281346q5491c472pd9512a885e4509e9@mail.gmail.com>
References: <c229b2e40905281346q5491c472pd9512a885e4509e9@mail.gmail.com>
Message-ID: <94bdd2610905290744t74a82990x32184c50804fd7e3@mail.gmail.com>

Hi James

thanks for the patch. Could you create an issue here
http://bugs.python.org/ please,

this helps for discussions/inclusions of patches

Cheers
Tarek

On Thu, May 28, 2009 at 10:46 PM, James <purpleidea at gmail.com> wrote:
> Hi,
> I'm certaintly new to distutils and setuptools, however I figured I'd
> send in this patch, and either it will get merged because it's a great
> idea or someone will perhaps tell me why this doesn't exist already.
> (or maybe it does and i can't find it)
> In any case, it adds the pyc option to the clean command so that the
> .pyc can be removed on request. Personally i'll have a [clean] pyc=1
> option in my setup.cfg, but that's for my convenience.
> cheers,
>
> _J
>
>
> [attached: patch and modified file]
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | http://ziade.org

From strawman at astraw.com  Fri May 29 17:35:16 2009
From: strawman at astraw.com (Andrew Straw)
Date: Fri, 29 May 2009 08:35:16 -0700
Subject: [Distutils] Low Level API for translating distutils/setuptools
 metatdata to	Debian metadata
In-Reply-To: <20090529091202.GB22620@Boo.lan>
References: <20090529091202.GB22620@Boo.lan>
Message-ID: <4A2000B4.90501@astraw.com>

Brian Sutherland wrote:
> Hi,
> 
> I've just published a very small library that does three things so far:
> 
>     * Provides a mapping between python project names and Debian
>       binary/source package names
>     * Converts setuptools versions to Debian versions while maintaining
>       sort order
>     * Can introspect an .egg-info directory to figure out the Debian
>       dependencies for use in the debian/control file. It can also
>       handle/understand extras (I Hope!)
> 
> It's basically an attempt to find a common ground between all the
> projects doing automated Python->Debian packaging. I have a feeling
> everyone is re-implementing this code all the time.

The stdeb package certainly needs most of these things, so it would be
nice to consolidate the implementation. It looks like there's some very
useful stuff in van.pydeb along with good tests. I'll attempt to convert
stdeb to use it. Do you have a public source code repository for
van.pydeb in case I want to start making patches?

> And an example of direct use in a packaging situation with complex dependencies is here:
> 
>     http://svn.debian.org/viewsvn/pkg-zope/zope.component/trunk/debian/rules?view=markup

As an aside, now that I've been looking at debian/rules files made for
debhelper 7 and python-support, that file looks bloated to my eye. (Of
course, if you're targeting older Debians without dh7, there's not much
you can do.) FWIW stdeb is growing dh7 support in the "dh7" branch. For
example, stdeb just generated this debian/rules file, which is
completely functional:

#!/usr/bin/make -f

# This file was automatically generated by stdeb 0.3+git+dh7 at
# Thu, 28 May 2009 15:38:44 -0700

%:
        dh $@


From dave at boostpro.com  Fri May 29 17:27:23 2009
From: dave at boostpro.com (David Abrahams)
Date: Fri, 29 May 2009 15:27:23 +0000 (UTC)
Subject: [Distutils] [setuptools] eggs and Python Unicode variants (UCS2,
	UCS4)
References: <44D3151F.9070109@infrae.com> <44CE48DD.3020000@infrae.com>
	<44D3151F.9070109@infrae.com>
	<D6AD543D-AA7C-44BD-8AE3-B98138FBC79E@mac.com>
	<5.1.1.6.0.20060806205504.02aa7008@sparrow.telecommunity.com>
Message-ID: <loom.20090529T152513-603@post.gmane.org>

This problem continues to bite:  http://allmydata.org/trac/tahoe/ticket/704
Has any progress been made?  Here's the original thread:  
http://markmail.org/message/bla5vrwlv3kn3n7e

Thanks!



From brian at vanguardistas.net  Fri May 29 19:27:50 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Fri, 29 May 2009 19:27:50 +0200
Subject: [Distutils] Low Level API for translating
	distutils/setuptools	metatdata to Debian metadata
In-Reply-To: <4A2000B4.90501@astraw.com>
References: <20090529091202.GB22620@Boo.lan> <4A2000B4.90501@astraw.com>
Message-ID: <20090529172750.GB24982@Boo.local>

On Fri, May 29, 2009 at 08:35:16AM -0700, Andrew Straw wrote:
> Brian Sutherland wrote:
> > Hi,
> > 
> > I've just published a very small library that does three things so far:
> > 
> >     * Provides a mapping between python project names and Debian
> >       binary/source package names
> >     * Converts setuptools versions to Debian versions while maintaining
> >       sort order
> >     * Can introspect an .egg-info directory to figure out the Debian
> >       dependencies for use in the debian/control file. It can also
> >       handle/understand extras (I Hope!)
> > 
> > It's basically an attempt to find a common ground between all the
> > projects doing automated Python->Debian packaging. I have a feeling
> > everyone is re-implementing this code all the time.
> 
> The stdeb package certainly needs most of these things, so it would be
> nice to consolidate the implementation. It looks like there's some very
> useful stuff in van.pydeb along with good tests. I'll attempt to convert
> stdeb to use it. Do you have a public source code repository for
> van.pydeb in case I want to start making patches?

I added a link to the svn repository to the web page, and am willing to
accept patches!

> > And an example of direct use in a packaging situation with complex dependencies is here:
> > 
> >     http://svn.debian.org/viewsvn/pkg-zope/zope.component/trunk/debian/rules?view=markup
> 
> As an aside, now that I've been looking at debian/rules files made for
> debhelper 7 and python-support, that file looks bloated to my eye. (Of
> course, if you're targeting older Debians without dh7, there's not much
> you can do.) FWIW stdeb is growing dh7 support in the "dh7" branch. For
> example, stdeb just generated this debian/rules file, which is
> completely functional:
> 
> #!/usr/bin/make -f
> 
> # This file was automatically generated by stdeb 0.3+git+dh7 at
> # Thu, 28 May 2009 15:38:44 -0700
> 
> %:
>         dh $@
> 

Yes we were thinking of using something similar; i.e. an includable
makefile. But this does look very interesting, I will definitely
investigate.

But just to point it out, it's way beyond the scope of van.pydeb to
define the rules file, that's for things like stdeb to do.

-- 
Brian Sutherland

From setuptools at bugs.python.org  Fri May 29 21:57:42 2009
From: setuptools at bugs.python.org (Sridhar Ratnakumar)
Date: Fri, 29 May 2009 19:57:42 +0000
Subject: [Distutils] [issue73] setup.py --name prints warning messages to
	stdout, rather than stderr
In-Reply-To: <1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>
Message-ID: <1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>


New submission from Sridhar Ratnakumar <sridharr at activestate.com>:

Warning messages must be printed to stderr. This allows for a programattic way
to find setup.py metadata.

> pwd
../dap.plugins.sql-0.2.1

> apy setup.py  --name 2> /dev/null
'dap.plugins' is declared as a package namespace, but 'dap' is not: please
correct this in setup.py
dap.plugins.sql

> echo $?
0
>

----------
messages: 295
nosy: srid
priority: bug
status: unread
title: setup.py --name prints warning messages to stdout, rather than stderr

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue73>
_______________________________________________

From purpleidea at gmail.com  Fri May 29 17:52:53 2009
From: purpleidea at gmail.com (James)
Date: Fri, 29 May 2009 11:52:53 -0400
Subject: [Distutils] Patch for clean command
In-Reply-To: <94bdd2610905290744t74a82990x32184c50804fd7e3@mail.gmail.com>
References: <c229b2e40905281346q5491c472pd9512a885e4509e9@mail.gmail.com>
	<94bdd2610905290744t74a82990x32184c50804fd7e3@mail.gmail.com>
Message-ID: <c229b2e40905290852k53664ff4g4dc7ada4f451b964@mail.gmail.com>

Done:

file 14112 created
msg 88512 created
issue 6142 created
http://bugs.python.org/issue6142

hth,
_J


On 5/29/09, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi James
>
>  thanks for the patch. Could you create an issue here
>  http://bugs.python.org/ please,
>
>  this helps for discussions/inclusions of patches
>
>  Cheers
>  Tarek
>
>
>  On Thu, May 28, 2009 at 10:46 PM, James <purpleidea at gmail.com> wrote:
>  > Hi,
>  > I'm certaintly new to distutils and setuptools, however I figured I'd
>  > send in this patch, and either it will get merged because it's a great
>  > idea or someone will perhaps tell me why this doesn't exist already.
>  > (or maybe it does and i can't find it)
>  > In any case, it adds the pyc option to the clean command so that the
>  > .pyc can be removed on request. Personally i'll have a [clean] pyc=1
>  > option in my setup.cfg, but that's for my convenience.
>  > cheers,
>  >
>  > _J
>  >
>  >
>  > [attached: patch and modified file]
>  >
>
> > _______________________________________________
>  > Distutils-SIG maillist  -  Distutils-SIG at python.org
>  > http://mail.python.org/mailman/listinfo/distutils-sig
>  >
>  >
>
>
>
>
>  --
>  Tarek Ziad? | http://ziade.org
>

From david.lyon at preisshare.net  Sun May 31 01:27:06 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 30 May 2009 19:27:06 -0400
Subject: [Distutils]
 =?utf-8?q?Low_Level_API_for_translating_distutils/set?=
 =?utf-8?q?uptools_metatdata_to=09Debian_metadata?=
In-Reply-To: <4A2000B4.90501@astraw.com>
References: <20090529091202.GB22620@Boo.lan> <4A2000B4.90501@astraw.com>
Message-ID: <b94538deaf36b06f35b6b78a89cb989e@preisshare.net>

On Fri, 29 May 2009 08:35:16 -0700, Andrew Straw <strawman at astraw.com>
wrote:
> Brian Sutherland wrote:
>> I've just published a very small library that does three things so far:
>> 
>>     * Provides a mapping between python project names and Debian
>>       binary/source package names
>>     * Converts setuptools versions to Debian versions while maintaining
>>       sort order
>>     * Can introspect an .egg-info directory to figure out the Debian
>>       dependencies for use in the debian/control file. It can also
>>       handle/understand extras (I Hope!)
>> 
>> It's basically an attempt to find a common ground between all the
>> projects doing automated Python->Debian packaging. I have a feeling
>> everyone is re-implementing this code all the time.

Maybe not everybody - but I am sure there are a few...

at least there are many who would like to do it..

Recently I posted a suggestion to Andrea Gavana about py2exe on the
wxPython list about his excellent gui interface for py2exe.

My use case is this... I'm writing a package manager for python..
I want it to be deployed to every python platform... I dont have
all those platforms....

I want p2exe or distutils (Distribution Utils?? is that what it
stands for?) to take my program and make it for every platform...

 - windows
 - mac
 - linux (ubuntu,debian,suse,redhat)

It's actually not so many..

No, I am not talking jibberish or pie in the sky here..

Here's specifically what needs doing.....

 1) create a three page "wizard" style app in wxpython

 2) the app has checkboxes to allow the user to select the 
    operating systems that they want to deploy to

 3) p2app for mac, py2exe for windows.. whichever one
    for linux is driven.. much like gui2exe

 4) Cheetah templates are used to create the necessary
    output scripts from the meta information.

    (building scripts in code is so 1970s.. we don't
     want to do it that way again)

 5) The scripts get run and the user ends up with
    .debs .rpm... windows installer  
 
basic structure...

for plat in user_selected_platforms:
    build_for_platform(plat)

def build_for_platform(platform):
    if platform == "mac":
        ..
    elif ....

I think distutils has been in a "frozen" state for the
last few years. With no notable improvements apart
from bug fixes and internal work.

Given that the general consensus is largely to leave
package/application management to the operating 
system, what I describe above is a more modern way to 
accomplish what people are trying to do with less 
effort on the part of all...

If a user makes a python app... why shouldn't they
be able to run a distutils script to "distribute"
to a platform they don't have....

Building ubuntu or debian packages is no longer
a black art....

It's 2009 now nearly 2010...

When was the last time distutils ever got a
major upgrade....?

Is this not what "disutils" should be doing?

in this day and age?

David



From yanegomi at gmail.com  Sun May 31 04:22:15 2009
From: yanegomi at gmail.com (Garrett Cooper)
Date: Sat, 30 May 2009 19:22:15 -0700
Subject: [Distutils] [issue73] setup.py --name prints warning messages
	to stdout, rather than stderr
In-Reply-To: <1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>
References: <1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>
	<1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>
Message-ID: <364299f40905301922h6de6d196t504dbed0002e1350@mail.gmail.com>

On Fri, May 29, 2009 at 12:57 PM, Sridhar
Ratnakumar<setuptools at bugs.python.org> wrote:
>
> New submission from Sridhar Ratnakumar <sridharr at activestate.com>:
>
> Warning messages must be printed to stderr. This allows for a programattic way
> to find setup.py metadata.
>
>> pwd
> ../dap.plugins.sql-0.2.1
>
>> apy setup.py ?--name 2> /dev/null
> 'dap.plugins' is declared as a package namespace, but 'dap' is not: please
> correct this in setup.py
> dap.plugins.sql
>
>> echo $?
> 0
>>
>
> ----------
> messages: 295
> nosy: srid
> priority: bug
> status: unread
> title: setup.py --name prints warning messages to stdout, rather than stderr

    It should be easy to filter / redirect critical, debug, error, and
warning messages to sys.stderr if there's a common logger for
setuptools...
My 2 cents,
-Garrett

From solipsis at pitrou.net  Sun May 31 20:04:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 May 2009 18:04:20 +0000 (UTC)
Subject: [Distutils] Mutual recursion between setuptools and distutils with
	Python 2.7
Message-ID: <loom.20090531T180210-628@post.gmane.org>

Hello,

I'm not sure where to post this (distutils or setuptools bug?), so I'm trying
here first. Using the CPython trunk, there's an infinite recursion due to a
mutual recursion between setuptools and distutils. Here is a snippet of a
traceback (trimmed for obvious reasons):

$ usr/bin/easy_install -Z -i
http://www.turbogears.org/2.0/downloads/current/index tg.devtools
Searching for tg.devtools
Reading http://www.turbogears.org/2.0/downloads/current/index/tg.devtools/
Best match: tg.devtools 2.0
Downloading http://www.turbogears.org/2.0/downloads/current/tg.devtools-2.0.tar.gz
Processing tg.devtools-2.0.tar.gz
Running tg.devtools-2.0/setup.py -q bdist_egg --dist-dir
/home/antoine/tmp/easy_install-AkgVEJ/tg.devtools-2.0/egg-dist-tmp-DQjVL4
Traceback (most recent call last):
  File "usr/bin/easy_install", line 8, in <module>
    load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')()
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
1671, in main
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
1659, in with_ei_usage
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
1675, in <lambda>
  File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in
run_commands
    self.run_command(cmd)
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in
run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
211, in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
446, in easy_install
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
476, in install_item
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
655, in install_eggs
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
930, in build_and_install
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line
919, in run_setup
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 27, in run_setup
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 63, in run
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 29, in <lambda>
  File "setup.py", line 53, in <module>
    return None
  File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in
run_commands
    self.run_command(cmd)
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in
run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 167,
in run
  File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in
run_command
    self.distribution.run_command(command)
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in
run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177,
in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252,
in find_sources
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 306,
in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 330,
in add_defaults
  File "/home/antoine/cpython/__svn__/Lib/distutils/command/sdist.py", line 288,
in add_defaults
    for pkg, src_dir, build_dir, filenames in build_py.data_files:
  File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 39,
in __getattr__
  File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 44,
in _get_data_files
  File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 92,
in analyze_manifest
  File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in
run_command
    self.distribution.run_command(command)
  File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in
run_command
    cmd_obj.run()

[...]

  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177,
in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252,
in find_sources
  File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 305,
in run
  File "/home/antoine/cpython/__svn__/Lib/distutils/filelist.py", line 45, in
findall
    self.allfiles = findall(dir)
  File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 71, in findall
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk
    for x in walk(path, topdown, onerror, followlinks):
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk
    for x in walk(path, topdown, onerror, followlinks):
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk
    for x in walk(path, topdown, onerror, followlinks):
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk
    for x in walk(path, topdown, onerror, followlinks):
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk
    for x in walk(path, topdown, onerror, followlinks):
  File "/home/antoine/cpython/__svn__/Lib/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/home/antoine/cpython/__svn__/Lib/genericpath.py", line 41, in isdir
    st = os.stat(s)
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 87, in wrap
RuntimeError: maximum recursion depth exceeded



From setuptools at bugs.python.org  Sun May 31 22:31:15 2009
From: setuptools at bugs.python.org (Nicholas Riley)
Date: Sun, 31 May 2009 20:31:15 +0000
Subject: [Distutils] [issue74] pkg_resources.ImpWrapper documented but no
	longer exists
In-Reply-To: <1243801875.8.0.475522244222.issue74@psf.upfronthosting.co.za>
Message-ID: <1243801875.8.0.475522244222.issue74@psf.upfronthosting.co.za>


New submission from Nicholas Riley <com-pythonbugs at sabi.net>:

pkg_resources.ImpWrapper was (and is) mentioned in the documentation 
(pkg_resources.txt), and as a result has been used by modules such as modulegraph 
(http://svn.pythonmac.org/modulegraph/modulegraph/trunk/modulegraph/modulegraph.py
).  However, recent versions of pkg_resources remove this definition.

Given this class appears in the documentation and still exists, just not with that 
name, it would be useful if backwards compatibility could be retained.

----------
messages: 296
nosy: nriley
priority: bug
status: unread
title: pkg_resources.ImpWrapper documented but no longer exists

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue74>
_______________________________________________

From tseaver at palladion.com  Sun May 31 22:40:51 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 31 May 2009 16:40:51 -0400
Subject: [Distutils] [issue73] setup.py --name prints warning messages
 to stdout, rather than stderr
In-Reply-To: <364299f40905301922h6de6d196t504dbed0002e1350@mail.gmail.com>
References: <1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>	<1243627062.66.0.928839843565.issue73@psf.upfronthosting.co.za>
	<364299f40905301922h6de6d196t504dbed0002e1350@mail.gmail.com>
Message-ID: <gvuq0p$clu$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Garrett Cooper wrote:
> On Fri, May 29, 2009 at 12:57 PM, Sridhar
> Ratnakumar<setuptools at bugs.python.org> wrote:
>> New submission from Sridhar Ratnakumar <sridharr at activestate.com>:
>>
>> Warning messages must be printed to stderr. This allows for a programattic way
>> to find setup.py metadata.
>>
>>> pwd
>> ../dap.plugins.sql-0.2.1
>>
>>> apy setup.py  --name 2> /dev/null
>> 'dap.plugins' is declared as a package namespace, but 'dap' is not: please
>> correct this in setup.py
>> dap.plugins.sql
>>
>>> echo $?
>> 0
>> ----------
>> messages: 295
>> nosy: srid
>> priority: bug
>> status: unread
>> title: setup.py --name prints warning messages to stdout, rather than stderr
> 
>     It should be easy to filter / redirect critical, debug, error, and
> warning messages to sys.stderr if there's a common logger for
> setuptools...

This use case isn't logging:  the entire point of running the command
'setup.py --name' is to print that text.  IMHO it belongs on stdout,
rather than stnderr.  Switching it will break extant / deployed scripts.

Other information printed while executing some command could be logging,
and therefore a potential candidate for the logging module (which itself
has issues about spew to standard pipes).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKIutT+gerLs4ltQ4RArVnAKCDBMfFCcbYnPsRA4SRtT+FlrBQVACdFZBi
J0JrglifE0+Cv+iJmq+zCzc=
=+ODo
-----END PGP SIGNATURE-----


From a.cavallo at mailsnare.com  Sun May 31 21:57:18 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Sun, 31 May 2009 20:57:18 +0100
Subject: [Distutils] Python svn builds
Message-ID: <200905312057.18783.a.cavallo@mailsnare.com>

Hi all,

I've just published few python rpms for the trunk source. You can download and 
install the packages from (either by hand or using yum/yast):

http://download.opensuse.org/repositories/home:/cavallo71:/opt-python-
interpreters/

The interpreters will install under so it will leave the system python 
untouched:

   /opt/opt-python-svnXXXXX-2.7a0

XXXXX is the svn revision. Once installed you can use them just sourcing the 
file:

$> . /opt/opt-python-svnXXXXX-2.7a0/opt-python-svnXXXXX-env.sh
$> python
Python 2.7a0 (trunk, May 24 2009, 21:18:39)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
Type "help", "copyright", "credits" or "license" for more information.

There is a web page containing the build results (including a small simple 
smoke test):
   http://pyvm.sourceforge.net/

Module developer can use these build to test against the more up to date 
python source and anticipating bugs.
Developer can use it to run regression/integration tests.


I hope that will help,
Regards,
Antonio