From pje at telecommunity.com  Mon Aug  1 17:41:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 11:41:47 -0400
Subject: [Distutils] PyPI update broke setuptools
In-Reply-To: <5.1.1.6.0.20050605180352.02208810@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>

The new version of PyPI changed the third assumption below, breaking 
setuptools' ability to do fallback searches for a request whose 
upper/lowercase didn't precisely match the cases used by the package 
author.  Is there any chance someone could revert this, since this breaks a 
ton of software in the field?

At 06:22 PM 6/5/2005 -0400, Phillip J. Eby wrote:
>For the benefit of the PyPI maintainers, here is a summary of EasyInstall's
>fairly minimal assumptions about PyPI's current behavior, assuming that
>"base-url" is the root URL of the package index:
>
>     * Going to "base-url/SomePackage" produces an HTML page that either has
>a title containing "Index of Packages" and links to zero or more pages for
>specific versions, or else it is a single-version package page.
>
>     * Single-version package pages may have a home page and download URL
>link, each of which occurs after '<th>Home Page' and '<th>Download URL'
>respectively, if present.
>
>     * Going to "base-url/" (note trailing '/') produces an HTML page
>containing links to all active versions of registered packages
>
>     * Links to package pages always have URLs of the form
>"base-url/SomePackage/itsVersion" - i.e., exactly two path parts following
>the base URL, with no query strings, parameters, fragments, etc.
>
>EasyInstall should continue to work with PyPI if these assumptions continue
>to hold.


From hawk78_it at yahoo.it  Mon Aug  1 22:58:35 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Mon, 1 Aug 2005 22:58:35 +0200
Subject: [Distutils] [Bulk] Re:  Setuptools CVS broken for python2.3
In-Reply-To: <5.1.1.6.0.20050731122834.02818238@mail.telecommunity.com>
References: <5.1.1.6.0.20050731122834.02818238@mail.telecommunity.com>
Message-ID: <200508012258.36125.hawk78_it@yahoo.it>

Alle 18:31, domenica 31 luglio 2005, hai scritto:
> At 05:29 PM 7/31/2005 +0200, Vincenzo Di Massa wrote:
> >Is setuptools CVS going to work using python 2.3?
> >After 25 of July it does not work anymore.
>
> See also this post:
>
>     http://mail.python.org/pipermail/distutils-sig/2005-July/004850.html
>
> and follow the instructions therein to make sure you have a correct
> checkout.  The CVS version of setuptools won't work at all without the
> .egg-info directory from CVS.

Thank you for this info.
Now I can install the cvs version.
But there is still a problem that is not 2.3 related: if I use the setup.py 
with the --prefix option I get this error:
Installing easy_install.py script to tmp2.4/bin
error: tmp2.4/bin/easy_install.py: No such file or directory

It works using ez_setup.py and the -d option.

Is it safe to use the -d to build packages (deb) of setuptools?
That is what I'm doing now...

Thank you again.

Vincenzo

From hawk78_it at yahoo.it  Mon Aug  1 23:07:16 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Mon, 1 Aug 2005 23:07:16 +0200
Subject: [Distutils] [Bulk] Re:  Setuptools CVS broken for python2.3
In-Reply-To: <5.1.1.6.0.20050731122834.02818238@mail.telecommunity.com>
References: <5.1.1.6.0.20050731122834.02818238@mail.telecommunity.com>
Message-ID: <200508012307.16530.hawk78_it@yahoo.it>

Alle 18:31, domenica 31 luglio 2005, hai scritto:
> At 05:29 PM 7/31/2005 +0200, Vincenzo Di Massa wrote:
> >Is setuptools CVS going to work using python 2.3?
> >After 25 of July it does not work anymore.
>
> See also this post:
>
>     http://mail.python.org/pipermail/distutils-sig/2005-July/004850.html
>
> and follow the instructions therein to make sure you have a correct
> checkout.  The CVS version of setuptools won't work at all without the
> .egg-info directory from CVS.

Thank you for this info.
Now I can install the cvs version.
But there is still a problem that is not 2.3 related: if I use the setup.py 
with the --prefix option I get this error:
Installing easy_install.py script to tmp2.4/bin
error: tmp2.4/bin/easy_install.py: No such file or directory

It works using ez_setup.py and the -d option.

Is it safe to use the -d to build packages (deb) of setuptools?
That is what I'm doing now...

Thank you again.

Vincenzo

From ianb at colorstudy.com  Tue Aug  2 00:09:15 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 01 Aug 2005 17:09:15 -0500
Subject: [Distutils] pkg_resources: import string
Message-ID: <42EE9D8B.4010801@colorstudy.com>

I was thinking some more about WSGI configuration, and much 
configuration involves some "import a string" routing, that looks like 
importstring("os.path:sep") or somesuch.  But we implement it slightly 
differently.  And maybe there could be a PEP and whatnot, but I don't 
even know what module that would go into, and it would just have to be 
backported for a long time anyway... but then it seemed like it would 
fit nicely into pkg_resources, and just about anywhere I want to import 
strings I expect pkg_resources to be around as well.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Tue Aug  2 02:16:54 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 20:16:54 -0400
Subject: [Distutils] pkg_resources: import string
In-Reply-To: <42EE9D8B.4010801@colorstudy.com>
Message-ID: <5.1.1.6.0.20050801201118.025d7ad8@mail.telecommunity.com>

At 05:09 PM 8/1/2005 -0500, Ian Bicking wrote:
>I was thinking some more about WSGI configuration, and much
>configuration involves some "import a string" routing, that looks like
>importstring("os.path:sep") or somesuch.  But we implement it slightly
>differently.  And maybe there could be a PEP and whatnot, but I don't
>even know what module that would go into, and it would just have to be
>backported for a long time anyway... but then it seemed like it would
>fit nicely into pkg_resources, and just about anywhere I want to import
>strings I expect pkg_resources to be around as well.

     from pkg_resources import EntryPoint

     def import_string(s):
         return EntryPoint.parse("x="+s).load(False)


From ianb at colorstudy.com  Tue Aug  2 18:27:24 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 02 Aug 2005 11:27:24 -0500
Subject: [Distutils] pkg_resources: import string
In-Reply-To: <5.1.1.6.0.20050801201118.025d7ad8@mail.telecommunity.com>
References: <5.1.1.6.0.20050801201118.025d7ad8@mail.telecommunity.com>
Message-ID: <42EF9EEC.4030401@colorstudy.com>

Phillip J. Eby wrote:
> At 05:09 PM 8/1/2005 -0500, Ian Bicking wrote:
> 
>> I was thinking some more about WSGI configuration, and much
>> configuration involves some "import a string" routing, that looks like
>> importstring("os.path:sep") or somesuch.  But we implement it slightly
>> differently.  And maybe there could be a PEP and whatnot, but I don't
>> even know what module that would go into, and it would just have to be
>> backported for a long time anyway... but then it seemed like it would
>> fit nicely into pkg_resources, and just about anywhere I want to import
>> strings I expect pkg_resources to be around as well.
> 
> 
>     from pkg_resources import EntryPoint
> 
>     def import_string(s):
>         return EntryPoint.parse("x="+s).load(False)

OK... the error messages will be a little funny due to the appropriation 
of the functionality from a different feature.  It would be nice if this 
was a separate function.  Also, what about evaluating the text after ':' 
in the scope of the module, instead of using getattr?  Though that does 
open the possibility of security issues (which are still present with 
getattr, but an actual exploit is rather unlikely).

Relatedly, in most places where I want to use this, I now also want to 
use a entry point spec.  But I don't know what entry point specs should 
look like.  So maybe even better would be a combination of the two. 
This might look like:

def import_string(spec, group=None):
     if group is not None and ' ' in spec:
         dist, name = spec.split(None, 1)
         return load_entry_point(dist, group, name)
     return EntryPoint.parse('x='+spec).load(False)

This uses a convention of "PackageName entry_point_name", with the space 
distinguishing this from other imports.  But I think a space is a rather 
vague way to make the distinction... maybe 
"PackageName[entry_point_name]"?  Then if [ comes before any : it's a 
entry point, otherwise it's an importable string.

If that sounds good I can submit a patch that does more thorough error 
checking, and separates the importing from EntryPoint.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Tue Aug  2 18:54:25 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 02 Aug 2005 12:54:25 -0400
Subject: [Distutils] pkg_resources: import string
In-Reply-To: <42EF9EEC.4030401@colorstudy.com>
References: <5.1.1.6.0.20050801201118.025d7ad8@mail.telecommunity.com>
	<5.1.1.6.0.20050801201118.025d7ad8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050802124245.02637608@mail.telecommunity.com>

At 11:27 AM 8/2/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 05:09 PM 8/1/2005 -0500, Ian Bicking wrote:
>>
>>>I was thinking some more about WSGI configuration, and much
>>>configuration involves some "import a string" routing, that looks like
>>>importstring("os.path:sep") or somesuch.  But we implement it slightly
>>>differently.  And maybe there could be a PEP and whatnot, but I don't
>>>even know what module that would go into, and it would just have to be
>>>backported for a long time anyway... but then it seemed like it would
>>>fit nicely into pkg_resources, and just about anywhere I want to import
>>>strings I expect pkg_resources to be around as well.
>>
>>     from pkg_resources import EntryPoint
>>     def import_string(s):
>>         return EntryPoint.parse("x="+s).load(False)
>
>OK... the error messages will be a little funny due to the appropriation 
>of the functionality from a different feature.

Huh?  They're just ImportErrors, and they don't mention EntryPoint in any 
way.  Unless you mean syntax errors, in which case you can always trap 
ValueError and reraise it.


>   It would be nice if this was a separate function.  Also, what about 
> evaluating the text after ':' in the scope of the module, instead of 
> using getattr?

Um, no.  If you want expressions, write code.  :)


>Relatedly, in most places where I want to use this, I now also want to use 
>a entry point spec.  But I don't know what entry point specs should look 
>like.  So maybe even better would be a combination of the two. This might 
>look like:
>
>def import_string(spec, group=None):
>     if group is not None and ' ' in spec:
>         dist, name = spec.split(None, 1)
>         return load_entry_point(dist, group, name)
>     return EntryPoint.parse('x='+spec).load(False)
>
>This uses a convention of "PackageName entry_point_name", with the space 
>distinguishing this from other imports.  But I think a space is a rather 
>vague way to make the distinction... maybe 
>"PackageName[entry_point_name]"?  Then if [ comes before any : it's a 
>entry point, otherwise it's an importable string.
>
>If that sounds good I can submit a patch that does more thorough error 
>checking, and separates the importing from EntryPoint.

I don't like this.  It's way too vague a concept that doesn't fit into 
pkg_resources architecture yet.  I don't mind making a nice import-string 
function, but this package/entry-point thing is getting too...  blurry.  I 
need a better idea of how this fits into things overall, as I can more 
clearly see use cases for:

* Import a list of all entry points matching some description
* Import the first entry point matching something
* Create a dictionary of lists of imported entry points

etc.  Whereas for the "import a specific entry point from a specific 
place", I see only application-specific needs.  For example, to get a 
wsgi.app_factories or wsgi.middleware_factories entry, where the group is 
implicit and not part of the syntax.  This is why I have difficulty seeing 
a syntax for loading entry points as being part of pkg_resources.  I mean, 
it's hard to get much simpler at the API level than what pkg_resources has 
for entry points right now, and the API supports all these use cases and 
more.  Adding syntax for your specific application/framework doesn't make 
sense to me because I don't see how anybody else is going to use it.

Also, the syntax you propose seems broken to me because it doesn't support 
versioning, and because the brackets can be confused with "extras" 
syntax.  If you want to use it in your own framework, I certainly can't 
stop you, but I don't want to mix something that's so close to -- yet 
completely different from -- the basic requirement syntax into the general API.


From limodou at gmail.com  Fri Aug  5 13:59:31 2005
From: limodou at gmail.com (limodou)
Date: Fri, 5 Aug 2005 19:59:31 +0800
Subject: [Distutils] Some questions about setuptools
Message-ID: <505f13c050805045978fce052@mail.gmail.com>

I'm new to setuptools.

One question is:

Recently I want to use setuptools for a project. My command line is just like :

python setup.py sdist --formats=gztar

But I found some deleted files also included in the package. These
files are .pyc suffixed. I use subversion. I checked the setuptools'
code, and found that setuptools indeed follows the .svn/entries file,
but it only retrieves files, and check if the file is exist. But in my
entries file, just like:

<entry
  name="EasyElements.pyc"
  kind="file"
  deleted="true"
  revision="33"/>

You can see the "deleted" field is "true", so the file should not be
included in the package, even through the file is exist. So I think
maybe the .pyc, .pyo file should be omited.

The another question is:

I have below directory structure:

A/
    setup.py
    B/
         __init__.py
         b.py
         t.txt
         .svn/
               entries

So B is a package dir. But as I run setup.py, the t.txt is not include
in the package. I followed the source code and found it was because
that dir A hasn't in version control, i.e. there is not .svn directory
in dir A, so the setuptools could not traversal the subdirectory. I
don't know how to write my setup.py script.

I'm sorry may be this letter is not suit for this maillist.


-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From pje at telecommunity.com  Fri Aug  5 17:30:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 05 Aug 2005 11:30:17 -0400
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <505f13c050805045978fce052@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>

At 07:59 PM 8/5/2005 +0800, limodou wrote:
>But I found some deleted files also included in the package. These
>files are .pyc suffixed. I use subversion. I checked the setuptools'
>code, and found that setuptools indeed follows the .svn/entries file,
>but it only retrieves files, and check if the file is exist. But in my
>entries file, just like:
>
><entry
>   name="EasyElements.pyc"
>   kind="file"
>   deleted="true"
>   revision="33"/>
>
>You can see the "deleted" field is "true", so the file should not be
>included in the package, even through the file is exist. So I think
>maybe the .pyc, .pyo file should be omited.

Did you try checking in your changes first?


>The another question is:
>
>I have below directory structure:
>
>A/
>     setup.py
>     B/
>          __init__.py
>          b.py
>          t.txt
>          .svn/
>                entries
>
>So B is a package dir. But as I run setup.py, the t.txt is not include
>in the package. I followed the source code and found it was because
>that dir A hasn't in version control, i.e. there is not .svn directory
>in dir A, so the setuptools could not traversal the subdirectory. I
>don't know how to write my setup.py script.

You need to put directory A under revision control as well; the entire 
project needs to be in Subversion (or CVS) for setuptools' automatic source 
finding to work.


From pje at telecommunity.com  Fri Aug  5 22:31:22 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 05 Aug 2005 16:31:22 -0400
Subject: [Distutils] Problems with Python eggs/setuptools
Message-ID: <5.1.1.6.0.20050805163114.02817c18@mail.telecommunity.com>

At 07:29 PM 8/5/2005 +0200, Walter D?rwald wrote:
>Hello Philip!
>
>I've started to play around with your easy_install script, since managing 
>our packages gets more complicated with each version.
>
>After I've installed setuptools via ez_setup.py I've tried downloading the 
>ll-xist package via "python -m easy_install ll-xist" but got a stack trace 
>instead (see the attached stacktrace.txt). The problem seems to be the 
>umlaut in my name. As PyPI requires UTF-8 encoded strings now, I've put a
>    author=u"Walter D?rwald".encode("utf-8")
>in my setup.py when I registered the package (but a plain
>    author=u"Walter D?rwald"
>in the setup.py in the package).
>
>To work around this problem it's possible to set the system default 
>encoding to Latin-1. I don't know if this is a problem with setuptools or 
>distutils, but doing a simple "python setup.py install" works.

Note that this problem is a distutils problem writing PKG-INFO files; it's 
not anything specific to setuptools.  I would recommend you use the 
.encode('utf-8') workaround until there is an official policy for the 
distutils to deal with encodings in PKG-INFO.


>The other problem seems more severe: ll-xist is installed as the packages 
>ll.xist, but the package init file ll/__init__.py is *not* provided by 
>this package, but by the ll-core package instead. If I understood 
>http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages 
>correctly distributing subpackages via setuptools only works if the 
>package init file is empty, which it isn't in this case.

Technically, you can do it, it's just not guaranteed that the non-empty one 
will be executed, unless it's first on sys.path.


>  Is there any workaround for this?

You can distribute an identical __init__.py with each distribution that 
shares that parent package.  If there is too much to duplicate, I would 
suggest moving the contents out to a module (e.g. _my_init.py), and then 
putting something like this in the __init__.py of each distributed package:

     import pkg_resources; del pkg_resources
     from _my_init import *

and make sure that all the distributions that don't contain the _my_init 
module have declared a dependency to the one that does.

The reason for the pkg_resources import is to ensure that it has a chance 
to set up the namespace package; if the modules were installed via a .pth 
file, and pkg_resources has not been imported yet, then the runtime system 
may not have made the package a namespace package yet, and it absolutely 
needs to be before the _my_init module gets imported, so that it will be on 
the package __path__.

On occasion I've thought of maybe executing *all* the __init__ modules, but 
it seems potentially error-prone.  If you want to try it, change this line 
in the _handle_ns routine of pkg_resources:

     module = sys.modules.get(packageName) or loader.load_module(packageName)

to read instead:

     module = loader.load_module(packageName)

and this will cause it to load *all* of the __init__ modules.  However, the 
order of execution is still not guaranteed, and I'm not certain that it 
might not cause an __init__ to be reloaded if its grandparent directory is 
included twice on sys.path.  Anyway, let me know if that change works for 
you, and I will think some more on the "load all __init__ modules" 
strategy.  Even if I do implement it, I will want to mainly advise people 
to use empty __init__ modules for any new namespace packages they create.


From limodou at gmail.com  Sat Aug  6 02:35:52 2005
From: limodou at gmail.com (limodou)
Date: Sat, 6 Aug 2005 08:35:52 +0800
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
Message-ID: <505f13c05080517352942f61c@mail.gmail.com>

2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> At 07:59 PM 8/5/2005 +0800, limodou wrote:
> >But I found some deleted files also included in the package. These
> >files are .pyc suffixed. I use subversion. I checked the setuptools'
> >code, and found that setuptools indeed follows the .svn/entries file,
> >but it only retrieves files, and check if the file is exist. But in my
> >entries file, just like:
> >
> ><entry
> >   name="EasyElements.pyc"
> >   kind="file"
> >   deleted="true"
> >   revision="33"/>
> >
> >You can see the "deleted" field is "true", so the file should not be
> >included in the package, even through the file is exist. So I think
> >maybe the .pyc, .pyo file should be omited.
> 
> Did you try checking in your changes first?
> 

Yes, I did. In entries file, the deleted entries are still in there.

> 
> >The another question is:
> >
> >I have below directory structure:
> >
> >A/
> >     setup.py
> >     B/
> >          __init__.py
> >          b.py
> >          t.txt
> >          .svn/
> >                entries
> >
> >So B is a package dir. But as I run setup.py, the t.txt is not include
> >in the package. I followed the source code and found it was because
> >that dir A hasn't in version control, i.e. there is not .svn directory
> >in dir A, so the setuptools could not traversal the subdirectory. I
> >don't know how to write my setup.py script.
> 
> You need to put directory A under revision control as well; the entire
> project needs to be in Subversion (or CVS) for setuptools' automatic source
> finding to work.
> 
> 

I think I can only do like what your said at present.

Thanks.

-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From pje at telecommunity.com  Sat Aug  6 07:31:19 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 06 Aug 2005 01:31:19 -0400
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <505f13c05080517352942f61c@mail.gmail.com>
References: <5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>

At 08:35 AM 8/6/2005 +0800, limodou wrote:
>2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> > At 07:59 PM 8/5/2005 +0800, limodou wrote:
> > >But I found some deleted files also included in the package. These
> > >files are .pyc suffixed. I use subversion. I checked the setuptools'
> > >code, and found that setuptools indeed follows the .svn/entries file,
> > >but it only retrieves files, and check if the file is exist. But in my
> > >entries file, just like:
> > >
> > ><entry
> > >   name="EasyElements.pyc"
> > >   kind="file"
> > >   deleted="true"
> > >   revision="33"/>
> > >
> > >You can see the "deleted" field is "true", so the file should not be
> > >included in the package, even through the file is exist. So I think
> > >maybe the .pyc, .pyo file should be omited.
> >
> > Did you try checking in your changes first?
> >
>
>Yes, I did. In entries file, the deleted entries are still in there.

That seems odd; I grepped through several large projects of mine for a 
'deleted' entry in an entries file, and didn't find any.  What version of 
Subversion do you use?


From limodou at gmail.com  Sat Aug  6 07:39:34 2005
From: limodou at gmail.com (limodou)
Date: Sat, 6 Aug 2005 13:39:34 +0800
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
Message-ID: <505f13c050805223974c336ce@mail.gmail.com>

2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> At 08:35 AM 8/6/2005 +0800, limodou wrote:
> >2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> > > At 07:59 PM 8/5/2005 +0800, limodou wrote:
> > > >But I found some deleted files also included in the package. These
> > > >files are .pyc suffixed. I use subversion. I checked the setuptools'
> > > >code, and found that setuptools indeed follows the .svn/entries file,
> > > >but it only retrieves files, and check if the file is exist. But in my
> > > >entries file, just like:
> > > >
> > > ><entry
> > > >   name="EasyElements.pyc"
> > > >   kind="file"
> > > >   deleted="true"
> > > >   revision="33"/>
> > > >
> > > >You can see the "deleted" field is "true", so the file should not be
> > > >included in the package, even through the file is exist. So I think
> > > >maybe the .pyc, .pyo file should be omited.
> > >
> > > Did you try checking in your changes first?
> > >
> >
> >Yes, I did. In entries file, the deleted entries are still in there.
> 
> That seems odd; I grepped through several large projects of mine for a
> 'deleted' entry in an entries file, and didn't find any.  What version of
> Subversion do you use?
> 
> 
0.5a13

I want to ask if deleted files entries are also in .svn/entries?


-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From pje at telecommunity.com  Sat Aug  6 07:41:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 06 Aug 2005 01:41:58 -0400
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <505f13c050805223974c336ce@mail.gmail.com>
References: <5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>

At 01:39 PM 8/6/2005 +0800, limodou wrote:
>2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> > At 08:35 AM 8/6/2005 +0800, limodou wrote:
> > >2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> > > > At 07:59 PM 8/5/2005 +0800, limodou wrote:
> > > > >But I found some deleted files also included in the package. These
> > > > >files are .pyc suffixed. I use subversion. I checked the setuptools'
> > > > >code, and found that setuptools indeed follows the .svn/entries file,
> > > > >but it only retrieves files, and check if the file is exist. But in my
> > > > >entries file, just like:
> > > > >
> > > > ><entry
> > > > >   name="EasyElements.pyc"
> > > > >   kind="file"
> > > > >   deleted="true"
> > > > >   revision="33"/>
> > > > >
> > > > >You can see the "deleted" field is "true", so the file should not be
> > > > >included in the package, even through the file is exist. So I think
> > > > >maybe the .pyc, .pyo file should be omited.
> > > >
> > > > Did you try checking in your changes first?
> > > >
> > >
> > >Yes, I did. In entries file, the deleted entries are still in there.
> >
> > That seems odd; I grepped through several large projects of mine for a
> > 'deleted' entry in an entries file, and didn't find any.  What version of
> > Subversion do you use?
> >
> >
>0.5a13

No, what version of *Subversion* are you using, not what setuptools version.


>I want to ask if deleted files entries are also in .svn/entries?

I'm not seeing any in my projects, that's why I'm asking.  Of course, I 
have configured Subversion to ignore *.pyc and *.pyo files; it looks to me 
like maybe you've done something different with yours?


From limodou at gmail.com  Sat Aug  6 07:52:13 2005
From: limodou at gmail.com (limodou)
Date: Sat, 6 Aug 2005 13:52:13 +0800
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805223974c336ce@mail.gmail.com>
	<5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
Message-ID: <505f13c05080522523cf81101@mail.gmail.com>

2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> At 01:39 PM 8/6/2005 +0800, limodou wrote:
> >2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> > > At 08:35 AM 8/6/2005 +0800, limodou wrote:
> > > >2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> > > > > At 07:59 PM 8/5/2005 +0800, limodou wrote:
> > > > > >But I found some deleted files also included in the package. These
> > > > > >files are .pyc suffixed. I use subversion. I checked the setuptools'
> > > > > >code, and found that setuptools indeed follows the .svn/entries file,
> > > > > >but it only retrieves files, and check if the file is exist. But in my
> > > > > >entries file, just like:
> > > > > >
> > > > > ><entry
> > > > > >   name="EasyElements.pyc"
> > > > > >   kind="file"
> > > > > >   deleted="true"
> > > > > >   revision="33"/>
> > > > > >
> > > > > >You can see the "deleted" field is "true", so the file should not be
> > > > > >included in the package, even through the file is exist. So I think
> > > > > >maybe the .pyc, .pyo file should be omited.
> > > > >
> > > > > Did you try checking in your changes first?
> > > > >
> > > >
> > > >Yes, I did. In entries file, the deleted entries are still in there.
> > >
> > > That seems odd; I grepped through several large projects of mine for a
> > > 'deleted' entry in an entries file, and didn't find any.  What version of
> > > Subversion do you use?
> > >
> > >
> >0.5a13
> 
> No, what version of *Subversion* are you using, not what setuptools version.
> 
> 
> >I want to ask if deleted files entries are also in .svn/entries?
> 
> I'm not seeing any in my projects, that's why I'm asking.  Of course, I
> have configured Subversion to ignore *.pyc and *.pyo files; it looks to me
> like maybe you've done something different with yours?
> 
> 

TortoiseSVN 1.1.3, Build 2502, UNICODE
Subversion 1.1.3, 

-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From dangoor at gmail.com  Sat Aug  6 14:51:16 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Sat, 6 Aug 2005 08:51:16 -0400
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805223974c336ce@mail.gmail.com>
	<5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
Message-ID: <3f085ecd050806055122cd0942@mail.gmail.com>

I'm using svn 1.1.4 and I just confirmed that I have deleted files
(that are committed as deleted) in my .svn/entries files.

Kevin

From limodou at gmail.com  Sat Aug  6 18:24:10 2005
From: limodou at gmail.com (limodou)
Date: Sun, 7 Aug 2005 00:24:10 +0800
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805223974c336ce@mail.gmail.com>
	<5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
Message-ID: <505f13c0508060924181bafad@mail.gmail.com>

2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> At 01:39 PM 8/6/2005 +0800, limodou wrote:
> >2005/8/6, Phillip J. Eby <pje at telecommunity.com>:
> > > At 08:35 AM 8/6/2005 +0800, limodou wrote:
> > > >2005/8/5, Phillip J. Eby <pje at telecommunity.com>:
> > > > > At 07:59 PM 8/5/2005 +0800, limodou wrote:
> > > > > >But I found some deleted files also included in the package. These
> > > > > >files are .pyc suffixed. I use subversion. I checked the setuptools'
> > > > > >code, and found that setuptools indeed follows the .svn/entries file,
> > > > > >but it only retrieves files, and check if the file is exist. But in my
> > > > > >entries file, just like:
> > > > > >
> > > > > ><entry
> > > > > >   name="EasyElements.pyc"
> > > > > >   kind="file"
> > > > > >   deleted="true"
> > > > > >   revision="33"/>
> > > > > >
> > > > > >You can see the "deleted" field is "true", so the file should not be
> > > > > >included in the package, even through the file is exist. So I think
> > > > > >maybe the .pyc, .pyo file should be omited.
> > > > >
> > > > > Did you try checking in your changes first?
> > > > >
> > > >
> > > >Yes, I did. In entries file, the deleted entries are still in there.
> > >
> > > That seems odd; I grepped through several large projects of mine for a
> > > 'deleted' entry in an entries file, and didn't find any.  What version of
> > > Subversion do you use?
> > >
> > >
> >0.5a13
> 
> No, what version of *Subversion* are you using, not what setuptools version.
> 
> 
> >I want to ask if deleted files entries are also in .svn/entries?
> 
> I'm not seeing any in my projects, that's why I'm asking.  Of course, I
> have configured Subversion to ignore *.pyc and *.pyo files; it looks to me
> like maybe you've done something different with yours?
> 
> 
I think the problem is solved. As I configured svn to ignore *.pyc,
there are no deleted file entries in the .svn/entries file. So the
process is perfect.

Thanks a lot for your help.


-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From pje at telecommunity.com  Sat Aug  6 18:24:16 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 06 Aug 2005 12:24:16 -0400
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <3f085ecd050806055122cd0942@mail.gmail.com>
References: <5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
	<505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805223974c336ce@mail.gmail.com>
	<5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050806121601.025a7370@mail.telecommunity.com>

At 08:51 AM 8/6/2005 -0400, Kevin Dangoor wrote:
>I'm using svn 1.1.4 and I just confirmed that I have deleted files
>(that are committed as deleted) in my .svn/entries files.

I've attached a patch that attempts to detect and ignore deleted entries; 
please let me know if it works for you.
-------------- next part --------------
Index: setuptools/command/sdist.py
===================================================================
RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/sdist.py,v
retrieving revision 1.4
diff -u -r1.4 sdist.py
--- setuptools/command/sdist.py	9 Jul 2005 04:21:22 -0000	1.4
+++ setuptools/command/sdist.py	6 Aug 2005 16:23:18 -0000
@@ -84,7 +84,11 @@
     (convert_path('CVS/Entries'),
         re_finder(re.compile(r"^\w?/([^/]+)/", re.M))),
     (convert_path('.svn/entries'),
-        re_finder(re.compile(r'name="([^"]+)"'), unescape)),
+        re_finder(
+            re.compile(r'name="([^"]+)"(?![^>]+deleted="true")', re.I),
+            unescape
+        )
+    ),
     (convert_path('.svn/dir-props'), externals_finder),
 ]
 
@@ -117,7 +121,3 @@
 
 
 
-
-
-
-

From limodou at gmail.com  Sat Aug  6 18:53:17 2005
From: limodou at gmail.com (limodou)
Date: Sun, 7 Aug 2005 00:53:17 +0800
Subject: [Distutils] Some questions about setuptools
In-Reply-To: <5.1.1.6.0.20050806121601.025a7370@mail.telecommunity.com>
References: <505f13c050805045978fce052@mail.gmail.com>
	<5.1.1.6.0.20050805112859.025a5e88@mail.telecommunity.com>
	<505f13c05080517352942f61c@mail.gmail.com>
	<5.1.1.6.0.20050806013019.025c4170@mail.telecommunity.com>
	<505f13c050805223974c336ce@mail.gmail.com>
	<5.1.1.6.0.20050806014025.028bb4b0@mail.telecommunity.com>
	<3f085ecd050806055122cd0942@mail.gmail.com>
	<5.1.1.6.0.20050806121601.025a7370@mail.telecommunity.com>
Message-ID: <505f13c05080609535e974b27@mail.gmail.com>

2005/8/7, Phillip J. Eby <pje at telecommunity.com>:
> At 08:51 AM 8/6/2005 -0400, Kevin Dangoor wrote:
> >I'm using svn 1.1.4 and I just confirmed that I have deleted files
> >(that are committed as deleted) in my .svn/entries files.
> 
> I've attached a patch that attempts to detect and ignore deleted entries;
> please let me know if it works for you.
> 
This patch runs well. 

I think the real reason is because I didn't ignore the .pyc files, so
the deleted .pyc files exist in the .svn/entries. If it is a common
file, as I delete it from svn, the real file will be removed, so
setuptools will not find the file, everything is ok. But as I test the
python file, .pyc will be created automatically, and the setuptools
will not notice that file was deleted, it will still get the filename
from .svn/entries, so that the deleted .pyc files are involved in the
package. So I think the patch is right, so I don't need to care about
to must ignore the .pyc , .pyo files.


-- 
I like python! 
My Donews Blog: http://www.donews.net/limodou
New Google Maillist: http://groups-beta.google.com/group/python-cn

From pje at telecommunity.com  Sun Aug  7 02:17:32 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 06 Aug 2005 20:17:32 -0400
Subject: [Distutils] setuptools updated in CVS
Message-ID: <5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>

FYI, I've just completed another round of changes to setuptools in CVS, 
including performance enhancements and new extensibility features (with 
documentation).  (And the 'sdist' command now incorporates the workaround 
patch for 'delete="true"' in .svn/entries files.)

setup() can now take a 'setup_requires' argument that specifies 
requirements that must be present before setup() runs; this allows you to 
depend on projects like buildutils, py2exe, py2app, etc. for setup 
arguments, commands, etc..

distutils extensions can now also define entry points to define new setup() 
arguments and new EGG-INFO metadata files, as well as setup commands.  With 
this release, it should be possible to take a distutils extension like 
"buildutils" and implement all of its distutils hooks just by defining 
"entry points" in its setup script, to declare setup commands, setup() 
args, and metadata types.  Entry points can declare extras that they depend 
on, too, so this means that commands can depend on external packages, and 
setuptools will attempt to download and install those packages in the setup 
directory if the package isn't already available on sys.path.

This is probably the last round of non-bugfix changes before releasing 
0.6a1, apart from further documentation enhancements.  Before 0.6a1 is 
released, I'd like to add more tips on non-root installations with 
EasyInstall, and create an updated API reference for pkg_resources.

Anyway, I haven't done a lot of regression testing yet, mostly just new 
features testing, so these changes may still be a bit rough around the 
edges.  I'd appreciate any feedback.  Thanks.


From LISTSERV at LSV.CERAMICS.ORG  Mon Aug  8 11:47:31 2005
From: LISTSERV at LSV.CERAMICS.ORG (L-Soft list server at clayart (1.8d))
Date: Mon, 8 Aug 2005 05:47:31 -0400
Subject: [Distutils] Message ("Your message dated Mon,
	8 Aug 2005 12:00:37 +0200...")
Message-ID: <427A664D0002F58B@mercury.acers> (added by mercury.acers)

Your message  dated Mon, 8 Aug  2005 12:00:37 +0200 with  subject "Message
could not be delivered" has been submitted to the moderator of the CLAYART
list: Mel Jacobson <melpots2 at PCLINK.COM>.

From pje at telecommunity.com  Mon Aug  8 18:45:54 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 08 Aug 2005 12:45:54 -0400
Subject: [Distutils] Problems with Python eggs/setuptools
In-Reply-To: <42F78822.8020608@livinglogic.de>
References: <5.1.1.6.0.20050805163114.02817c18@mail.telecommunity.com>
	<5.1.1.6.0.20050805163114.02817c18@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050808123516.025e4fb8@mail.telecommunity.com>

At 06:28 PM 8/8/2005 +0200, Walter D?rwald wrote:
>Phillip J. Eby wrote:
>
>>At 07:29 PM 8/5/2005 +0200, Walter D?rwald wrote:
>>
>>>Hello Philip!
>>>
>>>I've started to play around with your easy_install script, since 
>>>managing our packages gets more complicated with each version.
>>>
>>>After I've installed setuptools via ez_setup.py I've tried downloading 
>>>the ll-xist package via "python -m easy_install ll-xist" but got a stack 
>>>trace instead (see the attached stacktrace.txt). The problem seems to be 
>>>the umlaut in my name. As PyPI requires UTF-8 encoded strings now, I've put a
>>>    author=u"Walter D?rwald".encode("utf-8")
>>>in my setup.py when I registered the package (but a plain
>>>    author=u"Walter D?rwald"
>>>in the setup.py in the package).
>>>
>>>To work around this problem it's possible to set the system default 
>>>encoding to Latin-1. I don't know if this is a problem with setuptools 
>>>or distutils, but doing a simple "python setup.py install" works.
>>Note that this problem is a distutils problem writing PKG-INFO files; 
>>it's not anything specific to setuptools.  I would recommend you use the 
>>.encode('utf-8') workaround until there is an official policy for the 
>>distutils to deal with encodings in PKG-INFO.
>
>As PyPI espects UTF-8 now, I'd say it makes sense to use UTF-8 in package 
>info files too.

That's probably true, but this probably needs at least some distutils-sig 
discussion and maybe a PEP update.


>>>The other problem seems more severe: ll-xist is installed as the 
>>>packages ll.xist, but the package init file ll/__init__.py is *not* 
>>>provided by this package, but by the ll-core package instead. If I 
>>>understood 
>>>http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages 
>>>correctly distributing subpackages via setuptools only works if the 
>>>package init file is empty, which it isn't in this case.
>>Technically, you can do it, it's just not guaranteed that the non-empty 
>>one will be executed, unless it's first on sys.path.
> >
>>>  Is there any workaround for this?
>>You can distribute an identical __init__.py with each distribution that 
>>shares that parent package.
>
>I tried to avoid that, because the Windows installer then asks the user, 
>whether files should be overwritten or not when installing the second 
>package, and for package managers like RPM, Debian .deb or Gentoo emerge 
>it's also problematic, if one file (even if it is empty) is provided by 
>several packages.

Maybe the best solution, then, is to arrange things such that these 
installers install the packages as eggs, since this would then avoid any 
overwrite issues.  This is pretty much how eggs are supposed to work with 
packaging systems in the long run.


>>and make sure that all the distributions that don't contain the _my_init 
>>module have declared a dependency to the one that does.
>
>OK, this would work, but the clearer alternative would be to simply move 
>everything from ll to ll.core and have ll as an empty package module 
>again. Nevertheless this would require that each package includes the 
>empty __init__.py again, with the problems mentioned above.

Hm.  Tricky.  I'll have to give this some thought, because setuptools *has* 
to have the __init__, since separately distributed packages are in separate 
directories or zipfiles, but it never occurred to me that you could hack 
the distutils to do partial distribution in this way.  So, I never came up 
with a strategy for it.

Ideally, it sounds like I should have a way to ensure that when setuptools 
builds an egg, it should create empty __init__.py files if they don't 
exist.  That way, you could leave out the __init__.py for other builds, but 
it would ensure the package exists for the egg build.  The tricky part is 
that I can't do it in the build directory, because then it might sneak into 
other distributions.  But it's probably doable.


>>If you want to try it, change this line in the _handle_ns routine of 
>>pkg_resources:
>>     module = sys.modules.get(packageName) or loader.load_module(packageName)
>>to read instead:
>>     module = loader.load_module(packageName)
>>and this will cause it to load *all* of the __init__ modules.  However, 
>>the order of execution is still not guaranteed, and I'm not certain that 
>>it might not cause an __init__ to be reloaded if its grandparent 
>>directory is included twice on sys.path.  Anyway, let me know if that 
>>change works for you, and I will think some more on the "load all 
>>__init__ modules" strategy.  Even if I do implement it, I will want to 
>>mainly advise people to use empty __init__ modules for any new namespace 
>>packages they create.
>
>If I've understoodd this correcty, it doesn't solve my problem, as I only 
>have *one* __init__.py. And as setuptools puts different subpackages in 
>different directories, that get added via .pth files, I'm not sure, if 
>there can be a simple solution.

Well, if I create the __init__ in the zipfile, and pkg_resources executes 
all the __init__ files, then it should work even with your current layout, 
as long as there's a dependency declared from the package without the 
__init__ to the package with it.


From hawk78_it at yahoo.it  Tue Aug  9 11:53:00 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Tue, 9 Aug 2005 11:53:00 +0200
Subject: [Distutils] setuptools updated in CVS
In-Reply-To: <5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
References: <5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
Message-ID: <200508091153.00959.hawk78_it@yahoo.it>

Hi Philip,
using your last cvs I get this error:

hawk at dimassa:~/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1$ python ez_setup.py -d ../tmp .
Traceback (most recent call last):
  File "ez_setup.py", line 163, in ?
    main(sys.argv[1:])
  File "ez_setup.py", line 147, in main
    pkg_resources.require(req)
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 503, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 480, in resolve
    dist = best[req.key] = env.best_match(req, self, installer)
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 638, in best_match
    dist = working_set.find(req)
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 381, in find
    if dist is not None and dist not in req:
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 2040, in __contains__
    item = item.parsed_version
  File "/home/hawk/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1/pkg_resources.py", line 1874, in __getattr__
    return getattr(self._provider, attr)
AttributeError: PathMetadata instance has no attribute 'parsed_version'


Thank you for your help!

Vincenzo

Alle 02:17, domenica 07 agosto 2005, Phillip J. Eby ha scritto:
> FYI, I've just completed another round of changes to setuptools in CVS, 
> including performance enhancements and new extensibility features (with 
> documentation).  (And the 'sdist' command now incorporates the workaround 
> patch for 'delete="true"' in .svn/entries files.)
> 
> setup() can now take a 'setup_requires' argument that specifies 
> requirements that must be present before setup() runs; this allows you to 
> depend on projects like buildutils, py2exe, py2app, etc. for setup 
> arguments, commands, etc..
> 
> distutils extensions can now also define entry points to define new setup() 
> arguments and new EGG-INFO metadata files, as well as setup commands.  With 
> this release, it should be possible to take a distutils extension like 
> "buildutils" and implement all of its distutils hooks just by defining 
> "entry points" in its setup script, to declare setup commands, setup() 
> args, and metadata types.  Entry points can declare extras that they depend 
> on, too, so this means that commands can depend on external packages, and 
> setuptools will attempt to download and install those packages in the setup 
> directory if the package isn't already available on sys.path.
> 
> This is probably the last round of non-bugfix changes before releasing 
> 0.6a1, apart from further documentation enhancements.  Before 0.6a1 is 
> released, I'd like to add more tips on non-root installations with 
> EasyInstall, and create an updated API reference for pkg_resources.
> 
> Anyway, I haven't done a lot of regression testing yet, mostly just new 
> features testing, so these changes may still be a bit rough around the 
> edges.  I'd appreciate any feedback.  Thanks.
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 

		
___________________________________ 
Yahoo! Messenger: chiamate gratuite in tutto il mondo 
http://it.beta.messenger.yahoo.com

From dangoor at gmail.com  Tue Aug  9 17:40:10 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 9 Aug 2005 11:40:10 -0400
Subject: [Distutils] bdist_egg and source files
Message-ID: <3f085ecd050809084074dffde1@mail.gmail.com>

Something that just struck me as I kept running bdist_egg
--exclude-source-files is that it seems like "bdist" would imply
exclusion of source files. Perhaps there could be an sdist_egg that
creates an egg that includes all of the source...

Kevin

From pje at telecommunity.com  Tue Aug  9 17:52:49 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 09 Aug 2005 11:52:49 -0400
Subject: [Distutils] setuptools updated in CVS
In-Reply-To: <200508091153.00959.hawk78_it@yahoo.it>
References: <5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
	<5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>

At 11:53 AM 8/9/2005 +0200, Vincenzo Di Massa wrote:
>Hi Philip,
>using your last cvs I get this error:
>
>hawk at dimassa:~/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1$ 
>python ez_setup.py -d ../tmp .

Unfortunately I was not able to duplicate this error, although I got a 
different error which may be related.  I just checked in a fix to 
pkg_resources for the other error; please let me know if it fixes your 
problem.  By the way, if you didn't do so before, you should remove your 
setuptools.egg-info directory and "cvs update -d" to ensure that you get 
the correct contents of that directory.


From pje at telecommunity.com  Tue Aug  9 19:33:57 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 09 Aug 2005 13:33:57 -0400
Subject: [Distutils] bdist_egg and source files
In-Reply-To: <3f085ecd050809084074dffde1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050809133141.02772008@mail.telecommunity.com>

At 11:40 AM 8/9/2005 -0400, Kevin Dangoor wrote:
>Something that just struck me as I kept running bdist_egg
>--exclude-source-files is that it seems like "bdist" would imply
>exclusion of source files.

Source is important for tracebacks and debugging, especially if an egg is 
installed unpacked.  The --exclude-source-files option is intended to allow 
space conservation when creating applications for distribution, or for "IP 
protection" uses.  These are both fairly specialized build situations, not 
the common case for distributing an open source library.


From dangoor at gmail.com  Tue Aug  9 19:42:38 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 9 Aug 2005 13:42:38 -0400
Subject: [Distutils] bdist_egg and source files
In-Reply-To: <5.1.1.6.0.20050809133141.02772008@mail.telecommunity.com>
References: <3f085ecd050809084074dffde1@mail.gmail.com>
	<5.1.1.6.0.20050809133141.02772008@mail.telecommunity.com>
Message-ID: <3f085ecd0508091042761a775b@mail.gmail.com>

On 8/9/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:40 AM 8/9/2005 -0400, Kevin Dangoor wrote:
> >Something that just struck me as I kept running bdist_egg
> >--exclude-source-files is that it seems like "bdist" would imply
> >exclusion of source files.
> 
> Source is important for tracebacks and debugging, especially if an egg is
> installed unpacked.  The --exclude-source-files option is intended to allow
> space conservation when creating applications for distribution, or for "IP
> protection" uses.  These are both fairly specialized build situations, not
> the common case for distributing an open source library.

And it now occurs to me that "sdist_egg" doesn't make sense either,
because the egg has compiled C modules and such. Your explanation
about the debugging value of the source also explains the
--always-unzip easy_install option, which I thought much about.

(I happen to fall into the "space conservation for distributing
applications" category, which I agree is somewhat specialized.)

Thanks for the explanation.

Kevin

From hawk78_it at yahoo.it  Tue Aug  9 21:41:53 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Tue, 9 Aug 2005 21:41:53 +0200
Subject: [Distutils] [Bulk] Re:  setuptools updated in CVS
In-Reply-To: <5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
References: <5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
	<5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
Message-ID: <200508092141.53895.hawk78_it@yahoo.it>

I just checked out the new version but...

hawk at hawk:~/tmp/setuptools/python-setuptools-0.6a1$ python ez_setup.py -d ../tmp graph
Traceback (most recent call last):
  File "ez_setup.py", line 163, in ?
    main(sys.argv[1:])
  File "ez_setup.py", line 147, in main
    pkg_resources.require(req)
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 503, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 480, in resolve
    dist = best[req.key] = env.best_match(req, self, installer)
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 638, in best_match
    dist = working_set.find(req)
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 381, in find
    if dist is not None and dist not in req:
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 2040, in __contains__
    item = item.parsed_version
  File "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", line 1874, in __getattr__
    return getattr(self._provider, attr)
AttributeError: PathMetadata instance has no attribute 'parsed_version'


This is another machine: so the error is not caused by a python misconfiguration caused by me.

I can give you an ssh account on the machine in question if you need.

Thank you, Vincenzo.

Alle 17:52, marted? 09 agosto 2005, Phillip J. Eby ha scritto:
> At 11:53 AM 8/9/2005 +0200, Vincenzo Di Massa wrote:
> >Hi Philip,
> >using your last cvs I get this error:
> >
> >hawk at dimassa:~/stanzino/Projects/PythonModulePackaging/setuptools/setuptools-deb/cvs/python-setuptools-0.6a1$ 
> >python ez_setup.py -d ../tmp .
> 
> Unfortunately I was not able to duplicate this error, although I got a 
> different error which may be related.  I just checked in a fix to 
> pkg_resources for the other error; please let me know if it fixes your 
> problem.  By the way, if you didn't do so before, you should remove your 
> setuptools.egg-info directory and "cvs update -d" to ensure that you get 
> the correct contents of that directory.
> 
> 

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From ianb at colorstudy.com  Wed Aug 10 18:12:03 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 11:12:03 -0500
Subject: [Distutils] easy_install: --tag-svn-revision
Message-ID: <42FA2753.8030403@colorstudy.com>

Should easy_install have a --tag-svn-revision option like 
setuptools/setup.py has?  I'm feeling a desire for it now, while 
installing someone else's package that's only available in svn.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Wed Aug 10 19:58:45 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 13:58:45 -0400
Subject: [Distutils] [Bulk] Re:  setuptools updated in CVS
In-Reply-To: <200508092141.53895.hawk78_it@yahoo.it>
References: <5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
	<5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
	<5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810135544.025d4708@mail.telecommunity.com>

At 09:41 PM 8/9/2005 +0200, Vincenzo Di Massa wrote:
>I just checked out the new version but...
>
>hawk at hawk:~/tmp/setuptools/python-setuptools-0.6a1$ python ez_setup.py -d 
>../tmp graph

Okay, I figured out why I can't reproduce this; when working from the CVS 
version you need to use "setup.py develop" or "setup.py install" or even 
"setup.py egg_info" or "setup.py test" in order to *build& setuptools 
before attempting to use it. Otherwise you don't have a PKG-INFO file, and 
thus don't have a valid egg yet.

I'll have to update the documentation to reflect that necessary step.

>Traceback (most recent call last):
>   File "ez_setup.py", line 163, in ?
>     main(sys.argv[1:])
>   File "ez_setup.py", line 147, in main
>     pkg_resources.require(req)
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 503, in require
>     needed = self.resolve(parse_requirements(requirements))
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 480, in resolve
>     dist = best[req.key] = env.best_match(req, self, installer)
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 638, in best_match
>     dist = working_set.find(req)
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 381, in find
>     if dist is not None and dist not in req:
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 2040, in __contains__
>     item = item.parsed_version
>   File 
> "/mnt/disk2/5/hawk/tmp/setuptools/python-setuptools-0.6a1/pkg_resources.py", 
> line 1874, in __getattr__
>     return getattr(self._provider, attr)
>AttributeError: PathMetadata instance has no attribute 'parsed_version'


From ianb at colorstudy.com  Wed Aug 10 20:00:37 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 13:00:37 -0500
Subject: [Distutils] setuptools: package management and explicit version
	numbers
Message-ID: <42FA40C5.7020504@colorstudy.com>

Has there been any progress on some of the package management tools 
people were talking about?  I.e., list and delete packages?  (At the 
moment no other features come to mind, but I'm definitely seeing a 
purpose for these functions)  Someone said they were going to give it a 
go, but I can't remember who.

Another option that I think would be useful is to modify requirements in 
a package to make version numbers explicit.  So I might have something 
in development and it requires a bunch of packages.  It just requires 
the Most Recent version of those packages; I haven't tested anything but 
that, so I don't know if older versions would work, and I can't know if 
newer versions will work.

When I get ready to release, I want to be conservative.  Which means 
that I know that the exact versions I am using work well, but nothing 
else; I don't know that past or future versions of the packages I depend 
on will work.  So I'd like to change my requirements so that they 
specify the exact versions I'm using at the time of release.

As I think about it, it sounds pretty easy really.  I just find all the 
requirements, see what version is installed, and rewrite requires.txt 
with those versions.  At first I thought it should be a package 
management function, but should this be part of setuptools?  A new 
command, or maybe an option to egg_info?  It seems like it fits into 
egg_info well.  This means requirements should be specified only in 
requires.txt, not in setup().

There's some other features I can think of specifically using subversion 
together with setuptools, but maybe I'll mull on those a bit longer. 
Nothing fancy, but codifying a specific pattern, something like:

easy_install.py -e -b dev package_name
cd dev/package_name
# I would like "in-development" packages to have some specific version
# but I don't know what that version should be...?
sudo python setup.py develop -m
# time passes, development is done, release is ready...
python setup.py svntag 0.5
# changes PKG-INFO with new version, copies trunk to tags/0.5, does an
# svn switch

huh... maybe I can add such a command with an entry point?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Wed Aug 10 20:03:16 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 13:03:16 -0500
Subject: [Distutils] [Bulk] Re:  setuptools updated in CVS
In-Reply-To: <5.1.1.6.0.20050810135544.025d4708@mail.telecommunity.com>
References: <5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>	<5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>	<5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
	<5.1.1.6.0.20050810135544.025d4708@mail.telecommunity.com>
Message-ID: <42FA4164.4090209@colorstudy.com>

Phillip J. Eby wrote:
> At 09:41 PM 8/9/2005 +0200, Vincenzo Di Massa wrote:
> 
>>I just checked out the new version but...
>>
>>hawk at hawk:~/tmp/setuptools/python-setuptools-0.6a1$ python ez_setup.py -d 
>>../tmp graph
> 
> 
> Okay, I figured out why I can't reproduce this; when working from the CVS 
> version you need to use "setup.py develop" or "setup.py install" or even 
> "setup.py egg_info" or "setup.py test" in order to *build& setuptools 
> before attempting to use it. Otherwise you don't have a PKG-INFO file, and 
> thus don't have a valid egg yet.
> 
> I'll have to update the documentation to reflect that necessary step.

Wouldn't it be better to detect that PKG-INFO is missing, and give an 
error that suggests the package has not been properly build with 
egg_info?  I think as a general rule confusing error messages should be 
fixed with better error messages, not documentation.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Wed Aug 10 20:15:40 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 13:15:40 -0500
Subject: [Distutils] easy_install: custom package indexes
Message-ID: <42FA444C.5040300@colorstudy.com>

I have a custom package index that lists some Subversion repositories at 
http://pythonpaste.org/package_index.html ; I'm just looking for a 
little feedback on best practices.  For instance, here is a link:

http://svn.colorstudy.com/trunk/SQLObject#egg=SQLObject

(note that there's a small bug in setuptools that'll keep these from 
working at the moment, but putting that aside...)

Should that look like #egg=SQLObject-DEV ?  Or something else?  I think 
that tag will get ignored later in the process; should I modify setup.py 
(or setup.cfg) in some way to install the subversion checkout with an 
appropriate version number?

If that package_index.html is added with --find-links (or added to 
~/.pydistutils.cfg or elsewhere) then the subversion version will always 
be installed when no version is given in the package specifier 
("easy_install.py SQLObject").  I don't know if that's right.  I could 
change the PyPI entry and put the subversion repository there, and then 
I think the most up-to-date stable version is more likely to be 
installed (unless they do "easy_install.py SQLObject==DEV"?).  But I 
don't control the PyPI entries for some of these projects.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Wed Aug 10 20:55:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 14:55:17 -0400
Subject: [Distutils] easy_install: --tag-svn-revision
In-Reply-To: <42FA2753.8030403@colorstudy.com>
Message-ID: <5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>

At 11:12 AM 8/10/2005 -0500, Ian Bicking wrote:
>Should easy_install have a --tag-svn-revision option like
>setuptools/setup.py has?  I'm feeling a desire for it now, while
>installing someone else's package that's only available in svn.

One minor concern: should --tag-svn-revision apply to dependencies too?

I'm -0 on this, since --editable lets you do this with only a little 
fuss.  If you can come up with a rationale for dealing with dependencies or 
multiple easy_install targets, and a patch, I'll take it.


From pje at telecommunity.com  Wed Aug 10 20:59:12 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 14:59:12 -0400
Subject: [Distutils] [Bulk] Re:  setuptools updated in CVS
In-Reply-To: <42FA4164.4090209@colorstudy.com>
References: <5.1.1.6.0.20050810135544.025d4708@mail.telecommunity.com>
	<5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
	<5.1.1.6.0.20050806200716.025a7c48@mail.telecommunity.com>
	<5.1.1.6.0.20050809115057.00a04540@mail.telecommunity.com>
	<5.1.1.6.0.20050810135544.025d4708@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810145654.028cd350@mail.telecommunity.com>

At 01:03 PM 8/10/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 09:41 PM 8/9/2005 +0200, Vincenzo Di Massa wrote:
>>
>>>I just checked out the new version but...
>>>
>>>hawk at hawk:~/tmp/setuptools/python-setuptools-0.6a1$ python ez_setup.py 
>>>-d ../tmp graph
>>
>>Okay, I figured out why I can't reproduce this; when working from the CVS 
>>version you need to use "setup.py develop" or "setup.py install" or even 
>>"setup.py egg_info" or "setup.py test" in order to *build& setuptools 
>>before attempting to use it. Otherwise you don't have a PKG-INFO file, 
>>and thus don't have a valid egg yet.
>>I'll have to update the documentation to reflect that necessary step.
>
>Wouldn't it be better to detect that PKG-INFO is missing, and give an 
>error that suggests the package has not been properly build with egg_info?

It's actually there, but it's using AttributeError currently; I'll change 
it to a ValueError so it doesn't get masked.



From pje at telecommunity.com  Wed Aug 10 22:04:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 16:04:35 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <42FA40C5.7020504@colorstudy.com>
Message-ID: <5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>

At 01:00 PM 8/10/2005 -0500, Ian Bicking wrote:
>Has there been any progress on some of the package management tools
>people were talking about?  I.e., list and delete packages?  (At the
>moment no other features come to mind, but I'm definitely seeing a
>purpose for these functions)  Someone said they were going to give it a
>go, but I can't remember who.

It was Paul Moore.

I recently had a look at Ruby's "gems" system, which is surprisingly 
similar to setuptools and Python Eggs, except that they also have the list 
and delete functions, functions to verify signatures and check installation 
integrity, and a command to run a package index server off the information 
in the gems you have installed!  I'm thinking that in the long run (i.e. 
after I get the pkg_resources manual finished), I'll make a 'nest' script 
along the lines of their 'gem' program, and use that as the place for 
adding the list/delete/upgrade stuff.

One interesting benefit to the server concept is that you could use it to 
create a browser-based interface to browse through eggs' contents.  I 
suppose you could also include options to install/upgrade/delete eggs as 
well, at least if you have an appropriate way to restrict access.  It would 
let us implement a cross-platform GUI for setuptools.

Of course, I doubt I'd bundle it directly with setuptools.  Instead, I'd 
just define an entry point for the server command in the setuptools egg, 
with an "extra" set to download the egg with the actual server 
functionality.  In fact, everything except the "nest" script itself and the 
"install" command (which would be an alias for easy_install, such that 
'nest install foo' == 'easy_install foo') could be in separate "nest eggs".  :)


>Another option that I think would be useful is to modify requirements in
>a package to make version numbers explicit.  So I might have something
>in development and it requires a bunch of packages.  It just requires
>the Most Recent version of those packages; I haven't tested anything but
>that, so I don't know if older versions would work, and I can't know if
>newer versions will work.
>
>When I get ready to release, I want to be conservative.  Which means
>that I know that the exact versions I am using work well, but nothing
>else; I don't know that past or future versions of the packages I depend
>on will work.  So I'd like to change my requirements so that they
>specify the exact versions I'm using at the time of release.
>
>As I think about it, it sounds pretty easy really.  I just find all the
>requirements, see what version is installed, and rewrite requires.txt
>with those versions.  At first I thought it should be a package
>management function, but should this be part of setuptools?  A new
>command, or maybe an option to egg_info?  It seems like it fits into
>egg_info well.  This means requirements should be specified only in
>requires.txt, not in setup().

Ugh.  Well, you could do that, but you're going to get a warning about 
requires.txt being defined but no requirements specified in setup().

It probably makes more sense to have a function that takes a set of 
requirement strings and returns new requirement strings, and put that in 
setup().


>There's some other features I can think of specifically using subversion
>together with setuptools, but maybe I'll mull on those a bit longer.
>Nothing fancy, but codifying a specific pattern, something like:
>
>easy_install.py -e -b dev package_name
>cd dev/package_name
># I would like "in-development" packages to have some specific version
># but I don't know what that version should be...?
>sudo python setup.py develop -m
># time passes, development is done, release is ready...
>python setup.py svntag 0.5
># changes PKG-INFO with new version, copies trunk to tags/0.5, does an
># svn switch
>
>huh... maybe I can add such a command with an entry point?

You can add whatever commands you want with entry points.  Further, you can 
define aliases (per project, user, or Python installation) that group 
related commands and options into a single virtual command, using the 
'setup.py alias' command.


From ianb at colorstudy.com  Wed Aug 10 22:08:38 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 15:08:38 -0500
Subject: [Distutils] easy_install: --tag-svn-revision
In-Reply-To: <5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
References: <5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
Message-ID: <42FA5EC6.8030605@colorstudy.com>

Phillip J. Eby wrote:
> At 11:12 AM 8/10/2005 -0500, Ian Bicking wrote:
> 
>> Should easy_install have a --tag-svn-revision option like
>> setuptools/setup.py has?  I'm feeling a desire for it now, while
>> installing someone else's package that's only available in svn.
> 
> 
> One minor concern: should --tag-svn-revision apply to dependencies too?
> 
> I'm -0 on this, since --editable lets you do this with only a little 
> fuss.  If you can come up with a rationale for dealing with dependencies 
> or multiple easy_install targets, and a patch, I'll take it.

I want to install a package that isn't using setuptools, so I can't run 
"setup.py develop" without changing the setup.py file to use setuptools.

I don't think --tag-svn-revision should apply to dependencies.  Maybe it 
should even be an error if you try to use it on a package that isn't a 
subversion repository.

Really the reason I brought it up is that easy_install can apply 
setuptools commands to non-setuptools distributions, but "setup.py 
develop" can't.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Wed Aug 10 22:42:33 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 16:42:33 -0400
Subject: [Distutils] easy_install: custom package indexes
In-Reply-To: <42FA444C.5040300@colorstudy.com>
Message-ID: <5.1.1.6.0.20050810163509.028c7680@mail.telecommunity.com>

At 01:15 PM 8/10/2005 -0500, Ian Bicking wrote:
>I have a custom package index that lists some Subversion repositories at
>http://pythonpaste.org/package_index.html ; I'm just looking for a
>little feedback on best practices.  For instance, here is a link:
>
>http://svn.colorstudy.com/trunk/SQLObject#egg=SQLObject
>
>(note that there's a small bug in setuptools that'll keep these from
>working at the moment, but putting that aside...)

Does urllib2 barf on fragment identifiers?  I'm surprised.


>Should that look like #egg=SQLObject-DEV ?

Yes.


>   Or something else?

No.

I like these easy questions.  :)

See also the CVS version of setuptools.txt, which touches on these matters 
under the heading "Making your package available for EasyInstall".


>  I think
>that tag will get ignored later in the process; should I modify setup.py
>(or setup.cfg) in some way to install the subversion checkout with an
>appropriate version number?

Well, you can put this in setup.cfg to do that:

     [egg_info]
     tag_svn_revision = 1


>If that package_index.html is added with --find-links (or added to
>~/.pydistutils.cfg or elsewhere) then the subversion version will always
>be installed when no version is given in the package specifier
>("easy_install.py SQLObject").  I don't know if that's right.

'DEV' is a lower version number than zero, in pkg_resources' versioning 
scheme, so if there's a better version available via --find-links, it will 
get picked unless you express a preference for source (--editable) or you 
specify DEV as the version you want.  However, if you don't include any 
binary links in --find-links, then yes, you'll always end up with the 
development version.

Simple solution: make your package index page also include binary links or 
a source package link.  These take precedence over checkout links.


From pje at telecommunity.com  Wed Aug 10 23:12:04 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 17:12:04 -0400
Subject: [Distutils] easy_install: --tag-svn-revision
In-Reply-To: <42FA5EC6.8030605@colorstudy.com>
References: <5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
	<5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810164325.02744b70@mail.telecommunity.com>

At 03:08 PM 8/10/2005 -0500, Ian Bicking wrote:
>I don't think --tag-svn-revision should apply to dependencies.  Maybe it 
>should even be an error if you try to use it on a package that isn't a 
>subversion repository.

On the other hand, I don't suppose it would hurt to just ignore it if it 
doesn't apply.  I'm uneasy about dependencies, of course, but then again, 
where's the harm there either?

The biggest potential harm of --tag-svn-revision is that it makes things a 
*higher* revision number by default.  I.e. "1.2-r27" is a higher version 
number than "1.2", so if you're using a strategy where you use the *next* 
version number in your svn version of setup.py, then you're going to have 
problems when the official version is released.  In other words, if svn's 
setup.py says "1.2" is the current version, but that's the version you're 
*developing* (as opposed to enhancing), then the numbering works out wrong.


From ianb at colorstudy.com  Thu Aug 11 00:23:56 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 17:23:56 -0500
Subject: [Distutils] easy_install: --tag-svn-revision
In-Reply-To: <5.1.1.6.0.20050810164325.02744b70@mail.telecommunity.com>
References: <5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
	<5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
	<5.1.1.6.0.20050810164325.02744b70@mail.telecommunity.com>
Message-ID: <42FA7E7C.3040304@colorstudy.com>

Phillip J. Eby wrote:
> At 03:08 PM 8/10/2005 -0500, Ian Bicking wrote:
> 
>> I don't think --tag-svn-revision should apply to dependencies.  Maybe 
>> it should even be an error if you try to use it on a package that 
>> isn't a subversion repository.
> 
> 
> On the other hand, I don't suppose it would hurt to just ignore it if it 
> doesn't apply.  I'm uneasy about dependencies, of course, but then 
> again, where's the harm there either?

The harm in adding the revision there?  Not that great, I guess.  It's 
kind of a stop-gap measure either way; modifying setup.cfg in the 
repository itself seems like the right way to handle the versions.

> The biggest potential harm of --tag-svn-revision is that it makes things 
> a *higher* revision number by default.  I.e. "1.2-r27" is a higher 
> version number than "1.2", so if you're using a strategy where you use 
> the *next* version number in your svn version of setup.py, then you're 
> going to have problems when the official version is released.  In other 
> words, if svn's setup.py says "1.2" is the current version, but that's 
> the version you're *developing* (as opposed to enhancing), then the 
> numbering works out wrong.

True.  I'm never sure what version I should use in a repository.  The 
next version isn't true, the last version isn't true. 
"next_version-DEV" is probably right, I guess.

Nevertheless, it's still possible to use --tag-build to fix version 
numbers in case of problems.  It's not very sticky, though; it'd be easy 
to forget how it should be properly applied, or how it was applied in 
the past.  The ability to list packages would help resolve what problems 
do occur; especially listed in the order pkg_resources thinks they go 
in, showing which one is active, etc.

The main reason for stuff like this is to make it possible to use 
easy_install.py for an entire deployment without requiring all the 
software to be written with setuptools in mind.  So even if it is 
suboptimal, it's still of some use.

Another option might be something like "python -m setuptools setup.py" 
which would run setup.py with setuptools.  But then this is really just 
what easy_install does, except with a larger command set (egg_info in 
particular).  "easy_install.py --setup checkout-dir/ 
setuptools_commands..." perhaps?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From p.f.moore at gmail.com  Thu Aug 11 00:26:33 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 10 Aug 2005 23:26:33 +0100
Subject: [Distutils] setuptools: package management and explicit version
	numbers
In-Reply-To: <5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
References: <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
Message-ID: <79990c6b050810152617042569@mail.gmail.com>

On 8/10/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:00 PM 8/10/2005 -0500, Ian Bicking wrote:
> >Has there been any progress on some of the package management tools
> >people were talking about?  I.e., list and delete packages?  (At the
> >moment no other features come to mind, but I'm definitely seeing a
> >purpose for these functions)  Someone said they were going to give it a
> >go, but I can't remember who.
> 
> It was Paul Moore.

It was indeed. I haven't forgotten, but a number of things have kept
me from making progress yet. The first is the usual lack of time, but
that's not a limiting factor here, really. I've also been waiting for
the reorganisation of the API that Phillip promised, but again that's
not a real problem - I can use CVS if I need to.

My main issue is one of perspective, I guess. I'm not a package
maintainer, so much of the setuptools stuff (resource APIs, entry
points etc, etc) is not directly relevant to me. I'm a package *user*,
and so my viewpoint is of eggs as an alternative to Windows
bdist_wininst installers.

Hence my interest in management tools - installers come with a number
of benefits "free", as part of the Windows installer architecture -
you get an uninstall "command", you can see a listing of installed
packages, and you can see the version numbers of those packages. Until
I can do the same with eggs, they are a step backwards for me. So
that's why I want to build these features.

OK, so given that I have to write some code to bring eggs onto a par
with installers, where's the benefit of eggs, *to me*, which would
make me want to switch to them in the first place? And that's where I
get stalled. I can see lots of benefits to eggs - for example Trac's
new feature allowing you to just drop an egg containing a plugin into
a predefined directory is really exciting - but they don't apply to my
simple requirement to install packages like cx_Oracle, pywin32,
ctypes, Cheetah, into my standard Python installation.

So - a question for Phillip. Do you see eggs supplanting bdist_wininst
installers on Windows as the standard way of distributing Python
packages? If so, what do you see as the benefit to the end user, which
would prompt that switch? *Not* to the package developer, please note
- ultimately a developer can just as easily run python setup.py
bdist_wininst as python setup.py bdist_egg, and I doubt that users
will switch enthusiastically to using eggs simply because developers
switch to only offering them... (Switch maybe, but I doubt the
enthusiasm :-))

This is more than just a philosophical issue, in that a lot of the egg
infrastructure (development installs, for example) don't seem to me to
be relevant to the end user scenario. So, from my perspective, I don't
understand the imperatives which make them useful, and consequently I
can't do a good job of designing list/uninstall commands around them.
And I have no need for the facilities, so no incentive to cater for
them (beyond the desire to do a good job...)

As I say, I still intend to do this, but I'm waiting to get a clearer
picture of how eggs fit into the end-user environment (as opposed to
the developer environment) before I do so.

Ian - how this fits in with your requirements, I'm not sure. I really
don't follow your comments at all well, as I don't have the relevant
background (at the level I develop stuff, the prerequisite version
compatibility issues you describe just don't come into it - I only
distribute to a very limited set of targets, all of which are closely
controlled in terms of what is installed, so I have the luxury of
expecting things to "just work"). It's on my list of things to do, to
go back over the various recent postings and try to understand the
developer perspective better. But I still view the end-user side of
things as the first priority.

Hope this helps,
Paul.

From pje at telecommunity.com  Thu Aug 11 00:57:46 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 18:57:46 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050810152617042569@mail.gmail.com>
References: <5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810184549.025df9d0@mail.telecommunity.com>

At 11:26 PM 8/10/2005 +0100, Paul Moore wrote:
>Hence my interest in management tools - installers come with a number
>of benefits "free", as part of the Windows installer architecture -
>you get an uninstall "command", you can see a listing of installed
>packages, and you can see the version numbers of those packages. Until
>I can do the same with eggs, they are a step backwards for me. So
>that's why I want to build these features.
>
>OK, so given that I have to write some code to bring eggs onto a par
>with installers, where's the benefit of eggs, *to me*, which would
>make me want to switch to them in the first place? And that's where I
>get stalled. I can see lots of benefits to eggs - for example Trac's
>new feature allowing you to just drop an egg containing a plugin into
>a predefined directory is really exciting - but they don't apply to my
>simple requirement to install packages like cx_Oracle, pywin32,
>ctypes, Cheetah, into my standard Python installation.
>
>So - a question for Phillip. Do you see eggs supplanting bdist_wininst
>installers on Windows as the standard way of distributing Python
>packages?

Guardedly, I'd say yes -- once the tools mature, I think they'll become a 
popular distribution mechanism.  (OTOH, EasyInstall already converts 
bdist_wininst installers to almost-perfect eggs already, so this is sort of 
moot.)  However, the aim is somewhat more at providing a better way for 
application developers to ship applications, plugins, and needed libraries.


>If so, what do you see as the benefit to the end user, which
>would prompt that switch? *Not* to the package developer, please note

Better applications and packages, because with eggs you can specify 
dependencies.  This means that people will develop smaller more reusable 
packages rather than reinventing wheels, and the overall ecosystem will 
improve in software quality.

Yeah, I know that's not what you meant, but ultimately it is the real 
benefit to doing all this.  :)


>- ultimately a developer can just as easily run python setup.py
>bdist_wininst as python setup.py bdist_egg, and I doubt that users
>will switch enthusiastically to using eggs simply because developers
>switch to only offering them... (Switch maybe, but I doubt the
>enthusiasm :-))
>
>This is more than just a philosophical issue, in that a lot of the egg
>infrastructure (development installs, for example) don't seem to me to
>be relevant to the end user scenario.

Who is an "end user" here?  To me, if you're installing stuff into your 
Python installation, you're obviously a developer.  Eggs are especially 
meant to help install stuff for applications whose users don't even know 
what Python *is* - i.e., "real" end-users in my book.  :)

Now, if by "end user" you mean a developer who isn't distributing anything 
of their own, then yeah, having some egg management tools besides Windows 
Explorer might be nice.  :)  For that matter, even if you are doing more 
sophisticated things with eggs, better management tools would be nice.


>So, from my perspective, I don't
>understand the imperatives which make them useful,

If you look at it from the POV of a non-distributing developer using 
Windows, then the main benefit is going to be having multiple versions of 
something installed at the same time.  A second benefit is dependency 
resolution via EasyInstall; i.e. that you can get all the packages 
something needs to work, without having to track them down one by 
one.  Future additional benefits may include signature checking and the 
like (as soon as we come up with a signature format).


From ianb at colorstudy.com  Thu Aug 11 00:58:21 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 17:58:21 -0500
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050810152617042569@mail.gmail.com>
References: <42FA40C5.7020504@colorstudy.com>	
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
Message-ID: <42FA868D.3000908@colorstudy.com>

Paul Moore wrote:
> On 8/10/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
>>At 01:00 PM 8/10/2005 -0500, Ian Bicking wrote:
>>
>>>Has there been any progress on some of the package management tools
>>>people were talking about?  I.e., list and delete packages?  (At the
>>>moment no other features come to mind, but I'm definitely seeing a
>>>purpose for these functions)  Someone said they were going to give it a
>>>go, but I can't remember who.
>>
>>It was Paul Moore.
> 
> 
> It was indeed. I haven't forgotten, but a number of things have kept
> me from making progress yet. The first is the usual lack of time, but
> that's not a limiting factor here, really. I've also been waiting for
> the reorganisation of the API that Phillip promised, but again that's
> not a real problem - I can use CVS if I need to.
> 
> My main issue is one of perspective, I guess. I'm not a package
> maintainer, so much of the setuptools stuff (resource APIs, entry
> points etc, etc) is not directly relevant to me. I'm a package *user*,
> and so my viewpoint is of eggs as an alternative to Windows
> bdist_wininst installers.
> 
> Hence my interest in management tools - installers come with a number
> of benefits "free", as part of the Windows installer architecture -
> you get an uninstall "command", you can see a listing of installed
> packages, and you can see the version numbers of those packages. Until
> I can do the same with eggs, they are a step backwards for me. So
> that's why I want to build these features.

I suspect the management tool has two parts -- one is extending 
pkg_resources so these operations are easy (list, delete).  Then you 
want to integrate that into Windows' installer architecture -- 
registering the packages and uninstallers with Windows, maybe making a 
GUI frontend (though you could almost get away without it).  But since 
I'm not using Windows, the frontend I imagine is obviously very 
different.  My frontend is probably really easy, but I have no idea what 
a GUI frontend would entail.

> OK, so given that I have to write some code to bring eggs onto a par
> with installers, where's the benefit of eggs, *to me*, which would
> make me want to switch to them in the first place? And that's where I
> get stalled. I can see lots of benefits to eggs - for example Trac's
> new feature allowing you to just drop an egg containing a plugin into
> a predefined directory is really exciting - but they don't apply to my
> simple requirement to install packages like cx_Oracle, pywin32,
> ctypes, Cheetah, into my standard Python installation.

I think eggs are sometimes a distraction when considering features.  Or 
at least they are for me.  I know there's some eggs produced when I 
install stuff with easy_install, but I don't really touch them or care 
about them much.  Right now I don't have any incentive to distribute 
anything but source packages (from sdist); easy_install and setuptools 
do a fine job of using these, and for the users I interact with (other 
programmers) the source package is more usable.

But anyway, the feature for me is the installation of different package 
versions.  This makes me less worried about installing things globally. 
  This in turn makes management a lot easier.

I suspect Windows usage patterns (or just your patterns) aren't the 
same, so you aren't as concerned about this.

Another issue is installation of dependencies.  This is still just 
potential for me; I haven't really had the satisfying experience of 
getting a package to install lots of dependencies for me.  But I think 
I'm getting closer.  This is something I'm used to with software 
installation, but not Python.  Windows doesn't do this anyway, so you 
probably aren't clamouring for it.  I think it changes the larger 
ecosystem and encourages sharing; but the payoff isn't as immediate 
because there isn't as much sharing as there could be.

And last is formal metadata on a package, which is what makes plugins 
workable.  You'll only see this as people actually start using that 
metadata, so there's not much appeal there yet.

> This is more than just a philosophical issue, in that a lot of the egg
> infrastructure (development installs, for example) don't seem to me to
> be relevant to the end user scenario. So, from my perspective, I don't
> understand the imperatives which make them useful, and consequently I
> can't do a good job of designing list/uninstall commands around them.
> And I have no need for the facilities, so no incentive to cater for
> them (beyond the desire to do a good job...)

The development installs are there for developers.  Software doesn't 
appear out of thin air!

> Ian - how this fits in with your requirements, I'm not sure. I really
> don't follow your comments at all well, as I don't have the relevant
> background (at the level I develop stuff, the prerequisite version
> compatibility issues you describe just don't come into it - I only
> distribute to a very limited set of targets, all of which are closely
> controlled in terms of what is installed, so I have the luxury of
> expecting things to "just work"). It's on my list of things to do, to
> go back over the various recent postings and try to understand the
> developer perspective better. But I still view the end-user side of
> things as the first priority.

I'm not surprised most of my comments haven't made sense to you -- I'm 
just plodding along and trying to figure details out.  I'm hoping when I 
get all the details figured out that the larger picture will make more 
sense to me, but I haven't quite gotten there yet.  Mostly in terms of 
how this stuff *should* be used, as opposed to how it *can* be used. 
Plus you only see my questions so far, anything I understand I've kept 
to myself ;)

I'm doing a presentation on this stuff tomorrow, though, so hopefully 
I'll get closer today.  It'll feature lots of command-line transcripts 
you probably will find rather unexciting ;)


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Thu Aug 11 01:06:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 19:06:38 -0400
Subject: [Distutils] easy_install: --tag-svn-revision
In-Reply-To: <42FA7E7C.3040304@colorstudy.com>
References: <5.1.1.6.0.20050810164325.02744b70@mail.telecommunity.com>
	<5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
	<5.1.1.6.0.20050810145249.027ccc10@mail.telecommunity.com>
	<5.1.1.6.0.20050810164325.02744b70@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810190018.025e5110@mail.telecommunity.com>

At 05:23 PM 8/10/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 03:08 PM 8/10/2005 -0500, Ian Bicking wrote:
>>
>>>I don't think --tag-svn-revision should apply to dependencies.  Maybe it 
>>>should even be an error if you try to use it on a package that isn't a 
>>>subversion repository.
>>
>>On the other hand, I don't suppose it would hurt to just ignore it if it 
>>doesn't apply.  I'm uneasy about dependencies, of course, but then again, 
>>where's the harm there either?
>
>The harm in adding the revision there?  Not that great, I guess.  It's 
>kind of a stop-gap measure either way; modifying setup.cfg in the 
>repository itself seems like the right way to handle the versions.
>
>>The biggest potential harm of --tag-svn-revision is that it makes things 
>>a *higher* revision number by default.  I.e. "1.2-r27" is a higher 
>>version number than "1.2", so if you're using a strategy where you use 
>>the *next* version number in your svn version of setup.py, then you're 
>>going to have problems when the official version is released.  In other 
>>words, if svn's setup.py says "1.2" is the current version, but that's 
>>the version you're *developing* (as opposed to enhancing), then the 
>>numbering works out wrong.
>
>True.  I'm never sure what version I should use in a repository.  The next 
>version isn't true, the last version isn't true. "next_version-DEV" is 
>probably right, I guess.

Not quite; the '-' means that 'DEV' is a patchlevel, and goes right back to 
the "later version" problem.  You need something like '1.2dev' or '1.2pre' 
to make it a pre-release version.


>Nevertheless, it's still possible to use --tag-build to fix version 
>numbers in case of problems.  It's not very sticky, though; it'd be easy 
>to forget how it should be properly applied, or how it was applied in the 
>past.

Personally, I create aliases for different build patterns, and just run 
those aliases.  "setup.py alias" is your friend.  :)


>Another option might be something like "python -m setuptools setup.py" 
>which would run setup.py with setuptools.  But then this is really just 
>what easy_install does, except with a larger command set (egg_info in 
>particular).  "easy_install.py --setup checkout-dir/ 
>setuptools_commands..." perhaps?

Try Ryan's buildutils (just "easy_install buildutils").  His 'pbu' script 
runs setup.py with setuptools installed, and as of the CVS version, 
setuptools now embeds itself deep into the distutils so that even a script 
that doesn't use setuptools should end up using setuptools anyway, at least 
enough to enable the full command set.

Whether that *actually* works with the current versions of buildutils and 
setuptools, I couldn't say.  I don't know if buildutils has been updated to 
work with the entry point stuff for distutils extensions yet, although even 
if it hasn't it might still work anyway.


From ianb at colorstudy.com  Thu Aug 11 01:57:44 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 10 Aug 2005 18:57:44 -0500
Subject: [Distutils] easy_install: custom package indexes
In-Reply-To: <5.1.1.6.0.20050810163509.028c7680@mail.telecommunity.com>
References: <5.1.1.6.0.20050810163509.028c7680@mail.telecommunity.com>
Message-ID: <42FA9478.20201@colorstudy.com>

Phillip J. Eby wrote:
> At 01:15 PM 8/10/2005 -0500, Ian Bicking wrote:
> 
>> I have a custom package index that lists some Subversion repositories at
>> http://pythonpaste.org/package_index.html ; I'm just looking for a
>> little feedback on best practices.  For instance, here is a link:
>>
>> http://svn.colorstudy.com/trunk/SQLObject#egg=SQLObject
>>
>> (note that there's a small bug in setuptools that'll keep these from
>> working at the moment, but putting that aside...)
> 
> 
> Does urllib2 barf on fragment identifiers?  I'm surprised.

Doesn't barf so much as pass them through, then Apache barfs.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Thu Aug 11 02:39:02 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Aug 2005 20:39:02 -0400
Subject: [Distutils] easy_install: custom package indexes
In-Reply-To: <42FA9478.20201@colorstudy.com>
References: <5.1.1.6.0.20050810163509.028c7680@mail.telecommunity.com>
	<5.1.1.6.0.20050810163509.028c7680@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050810203818.02864ef8@mail.telecommunity.com>

At 06:57 PM 8/10/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 01:15 PM 8/10/2005 -0500, Ian Bicking wrote:
>>
>>>I have a custom package index that lists some Subversion repositories at
>>>http://pythonpaste.org/package_index.html ; I'm just looking for a
>>>little feedback on best practices.  For instance, here is a link:
>>>
>>>http://svn.colorstudy.com/trunk/SQLObject#egg=SQLObject
>>>
>>>(note that there's a small bug in setuptools that'll keep these from
>>>working at the moment, but putting that aside...)
>>
>>Does urllib2 barf on fragment identifiers?  I'm surprised.
>
>Doesn't barf so much as pass them through, then Apache barfs.

Okay.  I've applied your patch for this, as well as the others, along with 
fixes for the issues encountered by Vincent and Walter.  So, as soon as 
SourceForge anon CVS updates, you'll have 'em.


From p.f.moore at gmail.com  Thu Aug 11 11:40:43 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Aug 2005 10:40:43 +0100
Subject: [Distutils] setuptools: package management and explicit version
	numbers
In-Reply-To: <42FA868D.3000908@colorstudy.com>
References: <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
Message-ID: <79990c6b050811024057f99096@mail.gmail.com>

On 8/10/05, Ian Bicking <ianb at colorstudy.com> wrote:
> Paul Moore wrote:
> > Hence my interest in management tools - installers come with a number
> > of benefits "free", as part of the Windows installer architecture -
> > you get an uninstall "command", you can see a listing of installed
> > packages, and you can see the version numbers of those packages. Until
> > I can do the same with eggs, they are a step backwards for me. So
> > that's why I want to build these features.
> 
> I suspect the management tool has two parts -- one is extending
> pkg_resources so these operations are easy (list, delete).  Then you
> want to integrate that into Windows' installer architecture --
> registering the packages and uninstallers with Windows, maybe making a
> GUI frontend (though you could almost get away without it).  But since
> I'm not using Windows, the frontend I imagine is obviously very
> different.  My frontend is probably really easy, but I have no idea what
> a GUI frontend would entail.

I'm not interested in integrating with the Windows installer stuff per
se. My key concern here is that C:\Python24 (or wherever Python is
installed) is owned by Python - I want no reason or need to use OS
commands directly on its contents. As long as I have a way of getting
the information I'm after, and performing the actions I need to, I'm
happy. On that basis, something like

    easy_install.py list

(with command line output) is perfectly acceptable. Indeed, it's
probably an improvement, to the extent that it can be used in code,
not just read by the user.

BUT (and it's a big but) when easy_install.py list doesn't show me
everything installed in Python's directory, I have a problem. I
exclude stuff installed via python setup.py install (I never do that,
because it's unmanaged) or "just dropped in there" (same reason). But
there's a transitional problem with bdist_wininst installers, which
needs some thinking about (and no, easy_install's "convert a wininst
installer to an egg" feature isn't an answer - it loses things like
cx_Oracle's documentation, and I suspect that it breaks horribly for
pywin32, which uses a postinstall script).

As this issue is transitional only, it's not the end of the world, as
long as eggs really do become the package distribution medium of
choice for Python. I'd like to see some sign that eggs are making
inroads against bdist_wininst, first, though.

> I think eggs are sometimes a distraction when considering features.  Or
> at least they are for me.  I know there's some eggs produced when I
> install stuff with easy_install, but I don't really touch them or care
> about them much.  Right now I don't have any incentive to distribute
> anything but source packages (from sdist); easy_install and setuptools
> do a fine job of using these, and for the users I interact with (other
> programmers) the source package is more usable.

Source distributions, I currently immediately build into Windows
installers (python setup.py bdist_wininst) and install. If they
contain C extensions that I can't build myself (for whatever reason) I
bleat pathetically at the author in the hope that he'll take pity on
me and provide Windows builds, and if that doesn't work, I generally
(have to) give up.

So unless you are building packages with C extensions having complex
build requirements, you'll get no incentives from me - the source
package is fine. But I'll use it to build Windows installers, and
never bother with eggs.

(Is there ever going to be a situation where code *won't work" unless
run from an egg? I can see that being an issue, unless there's a way
availabe to wrap eggs in Windows installers, RPMs, debs, etc - there
will always be people who value "integration with platform standards"
over other benefits).

> But anyway, the feature for me is the installation of different package
> versions.  This makes me less worried about installing things globally.
>  This in turn makes management a lot easier.
> 
> I suspect Windows usage patterns (or just your patterns) aren't the
> same, so you aren't as concerned about this.

Correct. I don't worry about package versions much, beyond getting the
latest one. I tend to treat applications which don't work with "the
latest" versions of dependent packages as broken, and either attempt
to fix them, or avoid using them, depending on time and the importance
of the package to me.

> Another issue is installation of dependencies.  This is still just
> potential for me; I haven't really had the satisfying experience of
> getting a package to install lots of dependencies for me.  But I think
> I'm getting closer.  This is something I'm used to with software
> installation, but not Python.  Windows doesn't do this anyway, so you
> probably aren't clamouring for it.  I think it changes the larger
> ecosystem and encourages sharing; but the payoff isn't as immediate
> because there isn't as much sharing as there could be.

As I've hinted in previous messages, I have no interest whatsoever in
automatic location and download of packages - both for environmental
(firewall) reasons, and personal ones (security concerns, plus a
dislike of giving up control...) So, for me, getting a package to list
its dependencies is all I need - I'm still going to impose on myself
the requirement to manually download and install those dependencies.

This self-imposed manual process doesn't handle packages with
conflicting dependencies well, but that's never been a practical issue
for me so far (keeping my fingers crossed...)

> And last is formal metadata on a package, which is what makes plugins
> workable.  You'll only see this as people actually start using that
> metadata, so there's not much appeal there yet.

And that's the real killer benefit, to me, but it affects plugins much
more than standalone packages, so it isn't clear to me how relevant it
is to these discussions.

> The development installs are there for developers.  Software doesn't
> appear out of thin air!

Agreed, and I understand the benefits. But I don't have the experience
to understand the trade-offs and/or risks, so I have to be very
cautious in what I promise. (For example, if I code an "uninstall"
command, and someone runs it on a development install, does that
delete all of that person's working code? How can I make sure that
doesn't happen without breaking the "normal" uninstall functionality?
Given that I can't really test this, am I better just saying "not
supported if run against development installs"?)

> I'm doing a presentation on this stuff tomorrow, though, so hopefully
> I'll get closer today.  It'll feature lots of command-line transcripts
> you probably will find rather unexciting ;)

That comment would amuse my colleagues in the office - I'm the
"command line geek" round here :-)

Paul.

From pje at telecommunity.com  Thu Aug 11 17:29:48 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Aug 2005 11:29:48 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050811024057f99096@mail.gmail.com>
References: <42FA868D.3000908@colorstudy.com> <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
Message-ID: <5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>

At 10:40 AM 8/11/2005 +0100, Paul Moore wrote:
>there's a transitional problem with bdist_wininst installers, which
>needs some thinking about (and no, easy_install's "convert a wininst
>installer to an egg" feature isn't an answer - it loses things like
>cx_Oracle's documentation,

FYI, if there's a source distribution, the new --editable option (in CVS) 
allows you to download and extract the source for editing, without building 
it or anything.


>  and I suspect that it breaks horribly for
>pywin32, which uses a postinstall script).

Hm.  It's true that I haven't done anything to handle postinstall scripts; 
but I was under the impression that pywin32 self-registers when you try to 
use it.  I'll have to look into that.

I could probably actually add postinstall hooks to EasyInstall, except that 
it sort of goes against the concept of eggs being a "zero install" 
format.  It's worth thinking about/investigating though.


>As this issue is transitional only, it's not the end of the world, as
>long as eggs really do become the package distribution medium of
>choice for Python. I'd like to see some sign that eggs are making
>inroads against bdist_wininst, first, though.

That's not going to happen real soon; only a relatively tiny number of 
people even know eggs exist, and as long as they have a reasonably-usable 
bdist_wininst available then it's certainly a valid choice to just 
distribute that, thereby pleasing EasyInstall users and non-users alike.


>So unless you are building packages with C extensions having complex
>build requirements, you'll get no incentives from me - the source
>package is fine. But I'll use it to build Windows installers, and
>never bother with eggs.

Some packages of course may be eventually only be distributed as eggs.  For 
example, I'm switching all of my win32 binary distributions to eggs, which 
means you'll have to compile from source if you want a bdist_wininst.  But 
I'm likely to be in the minority for some time to come.  Transitions like 
this don't happen overnight, especially not based on an 0.6 alpha 
infrastructure.  :)


>(Is there ever going to be a situation where code *won't work" unless
>run from an egg?

Plugins, definitely.  And over time, the definition of what constitutes a 
"plugin" is likely to be ever-expanding.  For example, setuptools now 
supports plugins to add distutils commands, setup() arguments, and so 
on.  The *only* way to leverage these features is with an egg, even if it's 
a "development-mode" egg.


>  I can see that being an issue, unless there's a way
>availabe to wrap eggs in Windows installers, RPMs, debs, etc

There is in principle, but the respective bdist commands would need some 
updating in order to work that way.  Currently, you'd have to run 
EasyInstall first, and then package the resulting egg file or directory 
tree.  This is an area that needs some actual tool development, to produce 
some scripts like 'egg2rpm', 'egg2wininst', etc.  Or better yet, setuptools 
should probably grow replacement bdist commands, although these could also 
be distributed as extension eggs at first.


>Agreed, and I understand the benefits. But I don't have the experience
>to understand the trade-offs and/or risks, so I have to be very
>cautious in what I promise. (For example, if I code an "uninstall"
>command, and someone runs it on a development install, does that
>delete all of that person's working code? How can I make sure that
>doesn't happen without breaking the "normal" uninstall functionality?

Don't delete anything that's not in the directory or directories your tool 
is managing.  Development installations are outside the normal 
site-packages area, and deleting the '.egg-link' file from the 
site-packages directory "uninstalls" it.


From p.f.moore at gmail.com  Thu Aug 11 18:12:56 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Aug 2005 17:12:56 +0100
Subject: [Distutils] setuptools: package management and explicit version
	numbers
In-Reply-To: <5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
References: <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
Message-ID: <79990c6b050811091268cd0646@mail.gmail.com>

On 8/11/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:40 AM 8/11/2005 +0100, Paul Moore wrote:
> >there's a transitional problem with bdist_wininst installers, which
> >needs some thinking about (and no, easy_install's "convert a wininst
> >installer to an egg" feature isn't an answer - it loses things like
> >cx_Oracle's documentation,
> 
> FYI, if there's a source distribution, the new --editable option (in CVS)
> allows you to download and extract the source for editing, without building
> it or anything.

I'm not sure how that helps. If there's a source, I can just run
setup.py bdist_wininst anyway, so I don't see the difference.

> I could probably actually add postinstall hooks to EasyInstall, except that
> it sort of goes against the concept of eggs being a "zero install"
> format.  It's worth thinking about/investigating though.

Given that the trend seems to be to install eggs via easy_install, I'm
not sure "zero install" still applies. But if I can still just drop
eggs into sys.path, maybe it does. Must review this stuff again.

> That's not going to happen real soon; only a relatively tiny number of
> people even know eggs exist, and as long as they have a reasonably-usable
> bdist_wininst available then it's certainly a valid choice to just
> distribute that, thereby pleasing EasyInstall users and non-users alike.

That's what I thought. And it makes me reluctant to bother with eggs
for standard packages at all, sadly. (Plugins etc are a completely
different matter - for them, I think it's a wonderful technology!)

> Some packages of course may be eventually only be distributed as eggs.  For
> example, I'm switching all of my win32 binary distributions to eggs, which
> means you'll have to compile from source if you want a bdist_wininst.

Which is probably what I'll do. But I thought setuptools no longer
works if it's installed via bdist_wininst?

Sorry, I have to run now (shouldn't have started this email...) I'll
comment more later.

Paul.

From ianb at colorstudy.com  Thu Aug 11 19:10:19 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 11 Aug 2005 12:10:19 -0500
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050811091268cd0646@mail.gmail.com>
References: <42FA40C5.7020504@colorstudy.com>	
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>	
	<79990c6b050810152617042569@mail.gmail.com>	
	<42FA868D.3000908@colorstudy.com>	
	<79990c6b050811024057f99096@mail.gmail.com>	
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
Message-ID: <42FB867B.5060706@colorstudy.com>

Paul Moore wrote:
>>FYI, if there's a source distribution, the new --editable option (in CVS)
>>allows you to download and extract the source for editing, without building
>>it or anything.
> 
> 
> I'm not sure how that helps. If there's a source, I can just run
> setup.py bdist_wininst anyway, so I don't see the difference.

The only real reason to use --editable is to get access to 
easy_install's ability to find, download, and unpack packages.  If you 
aren't interested in that, then it isn't really important.  I imagine at 
some time in the future easy_install will also read and confirm 
signatures, and may have things like GUI frontends.  But I think that's 
a ways off, and some things require a larger discussion (like signatures).

>>I could probably actually add postinstall hooks to EasyInstall, except that
>>it sort of goes against the concept of eggs being a "zero install"
>>format.  It's worth thinking about/investigating though.
> 
> 
> Given that the trend seems to be to install eggs via easy_install, I'm
> not sure "zero install" still applies. But if I can still just drop
> eggs into sys.path, maybe it does. Must review this stuff again.

If you drop an egg in sys.path, you have to use 
pkg_resources.require('PackageName') to actually load it.  easy_install 
also manipulates a .pth file, so require() isn't needed (but won't 
hurt).  For egg-aware applications this isn't an issue -- which almost 
the same as saying that for apps using eggs as plugins it isn't an issue.

>>That's not going to happen real soon; only a relatively tiny number of
>>people even know eggs exist, and as long as they have a reasonably-usable
>>bdist_wininst available then it's certainly a valid choice to just
>>distribute that, thereby pleasing EasyInstall users and non-users alike.
> 
> 
> That's what I thought. And it makes me reluctant to bother with eggs
> for standard packages at all, sadly. (Plugins etc are a completely
> different matter - for them, I think it's a wonderful technology!)

Sure; as a developer setuptools has useful features besides eggs, and is 
a superset of distutils.  I don't think there's any real reason *not* to 
use setuptools; but it's also fine to ignore setuptools' extra features.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Thu Aug 11 22:15:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Aug 2005 16:15:34 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <42FB867B.5060706@colorstudy.com>
References: <79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050811160623.027d8fb8@mail.telecommunity.com>

At 12:10 PM 8/11/2005 -0500, Ian Bicking wrote:
>I imagine at some time in the future easy_install will also read and 
>confirm signatures, and may have things like GUI frontends.  But I think 
>that's a ways off, and some things require a larger discussion (like 
>signatures).

It's certainly possible for people to sign eggs now with the setuptools 
'upload' command (the --sign option invokes GPG), it's just that 
easy_install doesn't do any signature verification yet.  I have no real 
idea as to how that should work with respect to setting up policies or 
trust chains or any of that stuff.  Also, I'm not sure as yet how to 
retrieve signature info from PyPI, because I've never used it.

However, if somebody wants to sign their eggs and send them to PyPI using 
"upload --sign", and can then also suggest what should be done to verify 
the signatures (preferably including what GPG commands to run to do the 
verification!), then I'll certainly take a look at it.  Ideally, if this 
were done right it would work for source distributions and bdist_wininst 
installers as well as eggs, as long as EasyInstall can find the associated 
signature.


From p.f.moore at gmail.com  Thu Aug 11 22:39:25 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Aug 2005 21:39:25 +0100
Subject: [Distutils] setuptools: package management and explicit version
	numbers
In-Reply-To: <79990c6b050811091268cd0646@mail.gmail.com>
References: <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
Message-ID: <79990c6b050811133929513ef5@mail.gmail.com>

> Sorry, I have to run now (shouldn't have started this email...) I'll
> comment more later.

After a lot of thinking, and some experimentation, and an abortive
attempt at an email which ended up being *far* more negative than I
want, I have decided not to comment any more at this stage. I'll go
back to lurking until I have a better feel for eggs, their benefits,
and how they fit into the overall package distribution equation.

Sorry, but you guys are doing good work, and I don't want to spend my
time moaning.

Paul.

PS I will make one comment - I really do think that ez_setup should
have an option to disable downloads. When I tried installing the
PyProtocols egg, ez_setup happily grabbed the setuptools egg off the
web, installed it, and ran code from it. I know it needs to, but
that's a huge security risk - I'm not particularly obsessed by
security, but even I found that a bit scary. Arguably, no-download
should be the default, and auto-download should be the optional
behaviour.

From pje at telecommunity.com  Fri Aug 12 00:17:05 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Aug 2005 18:17:05 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050811133929513ef5@mail.gmail.com>
References: <79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>

At 09:39 PM 8/11/2005 +0100, Paul Moore wrote:
>PS I will make one comment - I really do think that ez_setup should
>have an option to disable downloads. When I tried installing the
>PyProtocols egg, ez_setup happily grabbed the setuptools egg off the
>web, installed it, and ran code from it. I know it needs to, but
>that's a huge security risk - I'm not particularly obsessed by
>security, but even I found that a bit scary.

Note that you can trivially prevent this behavior by ensuring that you 
always have the latest setuptools egg on your machine ahead of time.  ;)

Also note that when you run *any* setup script, it already has the 
opportunity to execute arbitrary code on your machine, and could download 
whatever else it wanted to anyway, so I consider this mostly a non-issue, 
securitywise.  Even reading the setup script source doesn't help; a 
malicious author (or tamperer) could easily bury the evil code in a 
seemingly innocuous import deep in the code being distributed.

Anyway, what I could possibly do for the setuptools egg itself is start 
building up a table of MD5 hashes in ez_setup.py to allow verification of 
the downloaded egg.  ez_setup isn't part of the egg, so I could maybe do 
that with an external tool of some kind.  Of course, the list of hashes 
would get ever-longer with each release, and it wouldn't work for 
installing new versions of setuptools with an older version of ez_setup.


>Arguably, no-download
>should be the default, and auto-download should be the optional
>behaviour.

Not really.  The entire point of ez_setup is to make the process 
hands-free.  I'm fine with trying to make it more tamper-proof in case of a 
compromise of python.org or your personal network connection, but if one of 
those was compromised, who's to say that ez_setup itself wasn't compromised 
too?  Or the code you're installing?  At some point, you're just plain 
screwed, and installing things by hand isn't actually an improvement unless 
you're going to personally vet every single line of code or have a 
signature whose certificate chain you trust.  And once we have such 
signature chains available, then easy_install can become just as capable of 
validating them as you are.

But even then, it will likely be a false sense of security, because I 
expect that 1) most authors won't bother to sign their packages to start 
with, and 2) of those that do, most will likely use self-signed 
certificates, so knowing what signatures are "valid" will be difficult.  If 
you have no way to know if a particular cert is really the author's, then 
you are right back to square zero.  That is, if you trust the source of the 
package enough to run its setup script in the first place, you're trusting 
them to run arbitrary code on your computer.

So, if the package author wants to download and install the package's 
dependencies, at that point you're just quibbling over details; you already 
gave them permission to do as they wish with your machine.  At least when 
you install their code as a pre-built egg you don't have to run any code 
you don't want to, and easy_install runs source package setup scripts in a 
sandbox to prevent ill-mannered (but not malicious) setup scripts from 
throwing crap on your system wherever they want to.  (And yes, such setup 
scripts *do* exist, which is what prompted me to add the sandboxing code.)

I think that in the long-enough-run, we'll solve the cert chain problems 
through social means, and eventually signing PyPI uploads will be de 
rigeur.  But at the moment, most package authors can't be bothered to fill 
out all their PyPI metadata and links or upload their packages to PyPI, so 
it's not realistic to expect there's going to be much signature support out 
there any time soon.  (Ignoring also issues like projects already married 
to SourceForge or other download systems that don't have an easy way to 
distribute the signatures.)

And without those signatures, your hand-installation procedure provides you 
with *zero* additional security unless you're personally inspecting every 
single line of code you install.  Heck, you're running downloaded .exe 
files with unsigned code, for heaven's sake!  And you're worried because 
ez_setup downloads the setuptools egg?  Crikey.  :)

(Of course, as with anything else, I could be completely off-base here, and 
I'm sure someone here will straighten me out if I am.)


From ianb at colorstudy.com  Fri Aug 12 01:09:59 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 11 Aug 2005 18:09:59 -0500
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
References: <79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
Message-ID: <42FBDAC7.90203@colorstudy.com>

Phillip J. Eby wrote:
>> Arguably, no-download
>> should be the default, and auto-download should be the optional
>> behaviour.
> 
> 
> Not really.  The entire point of ez_setup is to make the process 
> hands-free.  I'm fine with trying to make it more tamper-proof in case 
> of a compromise of python.org or your personal network connection, but 
> if one of those was compromised, who's to say that ez_setup itself 
> wasn't compromised too?  Or the code you're installing?  At some point, 
> you're just plain screwed, and installing things by hand isn't actually 
> an improvement unless you're going to personally vet every single line 
> of code or have a signature whose certificate chain you trust.  And once 
> we have such signature chains available, then easy_install can become 
> just as capable of validating them as you are.

I think from a make-people-feel-comfortable perspective, it might be 
better if ez_setup informed the user of what it's doing (installing a 
build dependency) and get a confirmation.  For instance, it can be 
disconcerting to do something that shouldn't require any privilege 
(e.g., setup.py --help-commands) and end up triggering something that 
does require privilege (global installation of a package).  Just an "I'm 
going to do this; OK?" question would be reassuring.

> And without those signatures, your hand-installation procedure provides 
> you with *zero* additional security unless you're personally inspecting 
> every single line of code you install.  Heck, you're running downloaded 
> .exe files with unsigned code, for heaven's sake!  And you're worried 
> because ez_setup downloads the setuptools egg?  Crikey.  :)

Realistically it's very hard to do coordinated attacks; it's as things 
get automated that larger holes can become exploitable and dangerous. 
One scary one is if someone uses a Wiki page as a package index, and 
other people reference that without understanding the (considerable) 
danger.  I *almost* did that myself, then I thought again and realized 
what a bad idea it would be.

But anyway, I think lots of attacks can be foiled by checking 
consistency.  E.g., check the file you downloaded against an MD5 
checksum stored elsewhere.  This doesn't prevent someone from uploading 
a completely bogus file, if they are able to get access to someone's 
PyPI account, for instance.

But other layers of consistency are possible.  For instance, for a 
package to be "trusted" by PyPI (on some level), maybe an email 
confirmation of substantive package updates would be required (like new 
releases, new versions of files, etc).  This is just another consistency 
check -- make sure that the person on the other end of the registered 
email address approves what the person with the login account is doing 
(of course usually those are the same person).

Anyway, those are just a few idea off the top of my head.  I find 
cryptographic standards of security can be a little misguided at times. 
  When I read this article I found it to be a quite refreshing way to 
think about security: http://iang.org/ssl/wytm.html

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Fri Aug 12 01:57:49 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Aug 2005 19:57:49 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <42FBDAC7.90203@colorstudy.com>
References: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050811193935.02660c48@mail.telecommunity.com>

At 06:09 PM 8/11/2005 -0500, Ian Bicking wrote:
>I think from a make-people-feel-comfortable perspective, it might be 
>better if ez_setup informed the user of what it's doing (installing a 
>build dependency) and get a confirmation.  For instance, it can be 
>disconcerting to do something that shouldn't require any privilege (e.g., 
>setup.py --help-commands) and end up triggering something that does 
>require privilege (global installation of a package).  Just an "I'm going 
>to do this; OK?" question would be reassuring.

But then, how do you do that in such a way that an automated installation 
process (other than EasyInstall) won't hang?

I suppose I could have the download function display a message followed by 
a countdown timer that would allow you to abort by hitting ^C.  That way, 
an unattended process or lazy user (or slow reader :) could just proceed 
without needing to do anything.

The only problem I see with that is that drawing the user's attention to 
something that 99% of the time is going to be okay seems like a bad 
idea.  It's like "WARNING: I'm about to do something exactly like what 
you'd do yourself by hand!"

I'll have to find a suitable way to spin the message, something like:

"""Hello!  ez_setup has detected that you don't have a recent-enough 
version of setuptools on your computer to be able to run this script.  I'll 
be happy to download and install it for you (along with any other packages 
this script might need), but some firewalls may not allow programs like me 
to download software from the Internet.  So I'll pause for a few seconds 
before starting the first download, to give you a chance to read this 
message, so you'll know you need to grant me access if something pops up 
asking if I should be allowed to connect to python.org.  Thanks!

Beginning download in 20...  19...  18...
"""

Hopefully, something like that could be made friendly enough so that most 
people just ignore it.


>  One scary one is if someone uses a Wiki page as a package index, and 
> other people reference that without understanding the (considerable) 
> danger.  I *almost* did that myself, then I thought again and realized 
> what a bad idea it would be.

That's why the best thing is to publish to PyPI if you can; source checkout 
links can always go in URLs embedded in 'long_description', and 
easy_install will still find them.


>But other layers of consistency are possible.  For instance, for a package 
>to be "trusted" by PyPI (on some level), maybe an email confirmation of 
>substantive package updates would be required (like new releases, new 
>versions of files, etc).  This is just another consistency check -- make 
>sure that the person on the other end of the registered email address 
>approves what the person with the login account is doing (of course 
>usually those are the same person).

At the very least, sending them emails about stuff that's happening would 
ensure they find out their account has been hacked.  Assuming the address 
is still valid, of course, which isn't always the case.  :(



From ianb at colorstudy.com  Fri Aug 12 06:32:56 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 11 Aug 2005 23:32:56 -0500
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <5.1.1.6.0.20050811193935.02660c48@mail.telecommunity.com>
References: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<5.1.1.6.0.20050811193935.02660c48@mail.telecommunity.com>
Message-ID: <42FC2678.1060307@colorstudy.com>

Phillip J. Eby wrote:
> At 06:09 PM 8/11/2005 -0500, Ian Bicking wrote:
> 
>> I think from a make-people-feel-comfortable perspective, it might be 
>> better if ez_setup informed the user of what it's doing (installing a 
>> build dependency) and get a confirmation.  For instance, it can be 
>> disconcerting to do something that shouldn't require any privilege 
>> (e.g., setup.py --help-commands) and end up triggering something that 
>> does require privilege (global installation of a package).  Just an 
>> "I'm going to do this; OK?" question would be reassuring.
> 
> 
> But then, how do you do that in such a way that an automated 
> installation process (other than EasyInstall) won't hang?
> 
> I suppose I could have the download function display a message followed 
> by a countdown timer that would allow you to abort by hitting ^C.  That 
> way, an unattended process or lazy user (or slow reader :) could just 
> proceed without needing to do anything.
 >
> The only problem I see with that is that drawing the user's attention to 
> something that 99% of the time is going to be okay seems like a bad 
> idea.  It's like "WARNING: I'm about to do something exactly like what 
> you'd do yourself by hand!"

Hopefully setuptools won't get installed 99% of the time, just once or 
twice per machine.  Because setuptools installation can happen even when 
nothing installation-related is being requested, it's a bit out of the 
norm.  Hence the confirmation, or at least prominent notification.

I also, like most unix users, don't usually start by running a command 
as root, so ez_setup will fail in that situation.  At least by putting 
up the interactive message it's not going to be as surprising when that 
happens.

>> But other layers of consistency are possible.  For instance, for a 
>> package to be "trusted" by PyPI (on some level), maybe an email 
>> confirmation of substantive package updates would be required (like 
>> new releases, new versions of files, etc).  This is just another 
>> consistency check -- make sure that the person on the other end of the 
>> registered email address approves what the person with the login 
>> account is doing (of course usually those are the same person).
> 
> 
> At the very least, sending them emails about stuff that's happening 
> would ensure they find out their account has been hacked.  Assuming the 
> address is still valid, of course, which isn't always the case.  :(

Until you start getting phishing emails trying to pretend that your 
account is hacked.  Ah, life on the internet... ):


-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From p.f.moore at gmail.com  Fri Aug 12 13:02:54 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 12 Aug 2005 12:02:54 +0100
Subject: [Distutils] setuptools: package management and explicit version
	numbers
In-Reply-To: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
References: <42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<79990c6b050811133929513ef5@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
Message-ID: <79990c6b050812040238706aea@mail.gmail.com>

On 8/11/05, Phillip J. Eby <pje at telecommunity.com> wrote:

> And without those signatures, your hand-installation procedure provides you
> with *zero* additional security unless you're personally inspecting every
> single line of code you install.  Heck, you're running downloaded .exe
> files with unsigned code, for heaven's sake!  And you're worried because
> ez_setup downloads the setuptools egg?  Crikey.  :)

Told you I'm not security-conscious (hey, I'm not conscious most of
the time! :-))

I'm a naive user who knows the Internet's a scary place, but doesn't
really think people are going to bother mocking up a website just to
pick on users of Python's PIL module. So if I go to the website and
*see* that it looks OK, I trust it.

But ez_setup just went off and got something, from somewhere. I never
saw the page with the link on it, so what if the link ez_setup used
was wrong? I never got to see a nice reassuring webpage with Fredrik's
name on it, so how can I be sure I got the right place?

I'm not *actually* that naive, but I do tend to prefer to be very
"manual" when I interact with the internet, just because I trust
myself (probably incorrectly!) more than I trust an automated
program...

OK, I retract the suggestion that no download be the default, but I'd
still like a "manual download" option, which doesn't grab stuff
automatically. After all, ez_setup has the option to go to a local
cache (I can't recall how it works, but I know you mentioned it
before). Why can't I say that I trust the cache (it's been vetted,
virus scanned, whatever) so use that, but *don't* go elsewhere? Then I
download what I think I need, do the install, and get messages
reporting any eggs I missed. I grab those, vet them, and try again.
Repeat as needed....

Paul.

From pje at telecommunity.com  Fri Aug 12 17:57:14 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 12 Aug 2005 11:57:14 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <79990c6b050812040238706aea@mail.gmail.com>
References: <5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<79990c6b050811133929513ef5@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050812114753.025dbc10@mail.telecommunity.com>

At 12:02 PM 8/12/2005 +0100, Paul Moore wrote:
>OK, I retract the suggestion that no download be the default, but I'd
>still like a "manual download" option, which doesn't grab stuff
>automatically.

I can't really do this for ez_setup (which doesn't have access to command 
line parameters or distutils config settings), but it should be possible 
for easy_install.  I could maybe have a --local-only option that refuses to 
do downloads from any URL other than file: URLs.  However, for it to take 
effect when you're running another package's "setup.py install", you'd need 
to set it in your per-user or sitewide distutils config file, because it 
won't be usable on the command line.


>  After all, ez_setup has the option to go to a local
>cache (I can't recall how it works, but I know you mentioned it
>before).

--find-links=/some/directory

But that's an easy_install option, not an ez_setup option.  ez_setup is all 
about downloading setuptools itself, and the only "local caches" it 
recognizes are the current directory, and an installed egg on sys.path.

So really, your vetting process for installing a package would be to read 
its setup script to see what version of setuptools it uses, so you can 
download and install the setuptools egg before proceeding.  You can also 
read the setup script to find out what dependencies the package has, 
keeping in mind of course that any package that uses entry points, 
require(), etc. is not going to be happy if you install its dependencies in 
non-egg form.


From pje at telecommunity.com  Fri Aug 12 18:03:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 12 Aug 2005 12:03:44 -0400
Subject: [Distutils] setuptools: package management and explicit version
 numbers
In-Reply-To: <42FC2678.1060307@colorstudy.com>
References: <5.1.1.6.0.20050811193935.02660c48@mail.telecommunity.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<42FA40C5.7020504@colorstudy.com>
	<5.1.1.6.0.20050810150144.027644d0@mail.telecommunity.com>
	<79990c6b050810152617042569@mail.gmail.com>
	<42FA868D.3000908@colorstudy.com>
	<79990c6b050811024057f99096@mail.gmail.com>
	<5.1.1.6.0.20050811100520.0263b2c8@mail.telecommunity.com>
	<79990c6b050811091268cd0646@mail.gmail.com>
	<5.1.1.6.0.20050811173109.025d95e8@mail.telecommunity.com>
	<5.1.1.6.0.20050811193935.02660c48@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050812115720.01add790@mail.telecommunity.com>

At 11:32 PM 8/11/2005 -0500, Ian Bicking wrote:
>Hopefully setuptools won't get installed 99% of the time, just once or 
>twice per machine.  Because setuptools installation can happen even when 
>nothing installation-related is being requested, it's a bit out of the 
>norm.  Hence the confirmation, or at least prominent notification.

Technically, only the download can happen when nothing installation-related 
is being requested.


>I also, like most unix users, don't usually start by running a command as 
>root, so ez_setup will fail in that situation.  At least by putting up the 
>interactive message it's not going to be as surprising when that happens.

You only let root connect to the internet?  :)

Seriously, if you run setup.py --help-commands or some such, all that's 
going to happen is that setuptools.egg gets downloaded into the current 
directory and stuck on sys.path for the duration of the script.  The 
piggyback installation only takes place as part of a "setup.py 
install".  So, you shouldn't need root just to run the setup script if 
you're not installing.  Similarly, if the setup script has any 
'setup_requires' eggs, those eggs just get downloaded to the current 
directory and put on sys.path as well - and they do not do a piggyback 
install, because you might need them only once.

In any case, piggyback installation only happens when you "setup.py 
install", not for any other operation whatsoever.  Download of setup-time 
dependencies, however, occurs on the first run of setup, no matter what the 
command.  I suppose I need to add an explanation of the setup_requires 
downloads, too.


From ianb at colorstudy.com  Fri Aug 12 18:13:26 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 12 Aug 2005 11:13:26 -0500
Subject: [Distutils] setuptools presentation
Message-ID: <42FCCAA6.1040100@colorstudy.com>

I made a presentation on setuptools & company last night at the Chicago 
Python user group.  For the interested, the slides:
   http://ianbicking.org/docs/setuptools-presentation/
source:
http://ianbicking.org/docs/setuptools-presentation/setuptools-slides.txt

Hit the /O in the corner to get all the HTML in one page.  The slides 
kind of dwindle at the end, because I got tired of collecting stuff. 
But eh, they're just slides, not documentation.  I think it went well. 
There were about 15 people there.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Fri Aug 12 19:23:05 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 12 Aug 2005 13:23:05 -0400
Subject: [Distutils] setuptools presentation
In-Reply-To: <42FCCAA6.1040100@colorstudy.com>
Message-ID: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>

At 11:13 AM 8/12/2005 -0500, Ian Bicking wrote:
>I made a presentation on setuptools & company last night at the Chicago
>Python user group.  For the interested, the slides:
>    http://ianbicking.org/docs/setuptools-presentation/

Nice!

Error on slide 13: you can't use --multi-version with install; you're 
probably thinking "sudo python setup.py easy_install --multi-version ." there.

Re: slide 31, the commands for installing pieces are used by the various 
bdist commands (including bdist_egg) to install to a temporary directory 
before archiving the directory to make an egg, windows installer, or "dumb" 
zipfile.

Re: slide 33, users only get to turn sdist distributions into bdist ones if 
they have a C compiler, or the package is Python-only.

Re: slide 38, it's "depends", not "deepends", and I've removed it 
anyway.  :)  Also, the 'test' command isn't "limited" to unittest, it just 
expects a unittest-compatible wrapper.  For example, doctest has functions 
that wrap doctests as unittest suites, so it's quite possible to use 
that.  As far as I'm concerned, unittest suites are the WSGI of testing; if 
somebody creates a fancy new test framework, they should darn well make it 
possible to put a unittest suite wrapper around it, so that people can 
integrate their existing tests.  :)


From ianb at colorstudy.com  Fri Aug 12 21:35:10 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 12 Aug 2005 14:35:10 -0500
Subject: [Distutils] setuptools presentation
In-Reply-To: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
Message-ID: <42FCF9EE.6070401@colorstudy.com>

CC'ing to bring together opposing viewpoints... (this is referring to 
setuptools' "setup.py test" command, which expects to have a TestSuite 
to run)

Phillip J. Eby wrote:
> Also, the 'test' command isn't "limited" to unittest, it 
> just expects a unittest-compatible wrapper.  For example, doctest has 
> functions that wrap doctests as unittest suites, so it's quite possible 
> to use that.  As far as I'm concerned, unittest suites are the WSGI of 
> testing; if somebody creates a fancy new test framework, they should 
> darn well make it possible to put a unittest suite wrapper around it, so 
> that people can integrate their existing tests.  :)

Well, I don't know what exactly my opinion is.  At a py.test user I 
don't have TestSuites for my tests.  I've argued py.test should load 
TestCase-based tests by default, but this is kind of the opposite.  I 
don't think it would be that hard to produce such suites; the test items 
that py.test collects could just be stuffed into a TestCase.

The result would be acceptable as a sort of "this package thinks it runs 
okay" test.  It's not the frontend I'd like to give to users.

That said, if "python setup.py test ..." was completely equivalent to 
"py.test ..." then that would be great, because though the interface 
would be different from projects that use unittest, the entry point 
would be the same (assuming py.test is installed).  I suppose a package 
could add an entry point that overrides the normal setuptools test 
command...?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From hpk at trillke.net  Fri Aug 12 22:10:22 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 12 Aug 2005 22:10:22 +0200
Subject: [Distutils] [py-dev] Re:  setuptools presentation
In-Reply-To: <42FCF9EE.6070401@colorstudy.com>
References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
	<42FCF9EE.6070401@colorstudy.com>
Message-ID: <20050812201022.GC11996@solar.trillke.net>

On Fri, Aug 12, 2005 at 14:35 -0500, Ian Bicking wrote:
> CC'ing to bring together opposing viewpoints... (this is referring to 
> setuptools' "setup.py test" command, which expects to have a TestSuite 
> to run)
>
> Phillip J. Eby wrote:
> >Also, the 'test' command isn't "limited" to unittest, it 
> >just expects a unittest-compatible wrapper.  For example, doctest has 
> >functions that wrap doctests as unittest suites, so it's quite possible 
> >to use that.  As far as I'm concerned, unittest suites are the WSGI of 
> >testing; if somebody creates a fancy new test framework, they should 
> >darn well make it possible to put a unittest suite wrapper around it, so 
> >that people can integrate their existing tests.  :)
> 
> Well, I don't know what exactly my opinion is.  At a py.test user I 
> don't have TestSuites for my tests.  I've argued py.test should load 
> TestCase-based tests by default, but this is kind of the opposite.  I 
> don't think it would be that hard to produce such suites; the test items 
> that py.test collects could just be stuffed into a TestCase.

I agree without having looked deeply into it.  Providing TestSuite
instances that adapt running tests in a simple way with part 
of the py.test machinery should be quite feasible.  Would probably
mean that some of py.test's execution features would be switched
off (like stdout/stderr capturing). 

However, as Ian hints at, py.test itself is already a project independent
entry point for running tests in a given directory or for a given project. 

> The result would be acceptable as a sort of "this package thinks it runs 
> okay" test.  It's not the frontend I'd like to give to users.

unsurprisingly, i agree :-)

> That said, if "python setup.py test ..." was completely equivalent to 
> "py.test ..." then that would be great, because though the interface 
> would be different from projects that use unittest, the entry point 
> would be the same (assuming py.test is installed).

would make sense, i think.  

> I suppose a package could add an entry point that overrides
> the normal setuptools test command...?

Yes, i guess something needs to be configurable there as far as i
understand the situation.  I also presume that "setup.py test"
would allow a custom test method to actually perform the execution
of tests, not just provide a TestSuite.  

If so, it should at best become a simple matter of how a
package can signal to setuptools that it wants its tests to be
handled by py.test in which case the work would be defered to
the (neccessarily) installed py library when "setup.py test"
is invoked.  It shouldn't be required from each application
to provide this glue code as it should be generic. 

actually, how does "setup.py test" work today and which
applications/projects are integrated with it if i may ask? 

cheers, 

    holger

From ianb at colorstudy.com  Fri Aug 12 22:30:33 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 12 Aug 2005 15:30:33 -0500
Subject: [Distutils] [py-dev] Re:  setuptools presentation
In-Reply-To: <20050812201022.GC11996@solar.trillke.net>
References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
	<42FCF9EE.6070401@colorstudy.com>
	<20050812201022.GC11996@solar.trillke.net>
Message-ID: <42FD06E9.70405@colorstudy.com>

holger krekel wrote:
>>Well, I don't know what exactly my opinion is.  At a py.test user I 
>>don't have TestSuites for my tests.  I've argued py.test should load 
>>TestCase-based tests by default, but this is kind of the opposite.  I 
>>don't think it would be that hard to produce such suites; the test items 
>>that py.test collects could just be stuffed into a TestCase.
> 
> 
> I agree without having looked deeply into it.  Providing TestSuite
> instances that adapt running tests in a simple way with part 
> of the py.test machinery should be quite feasible.  Would probably
> mean that some of py.test's execution features would be switched
> off (like stdout/stderr capturing). 
> 
> However, as Ian hints at, py.test itself is already a project independent
> entry point for running tests in a given directory or for a given project. 
> 
> 
>>The result would be acceptable as a sort of "this package thinks it runs 
>>okay" test.  It's not the frontend I'd like to give to users.
> 
> 
> unsurprisingly, i agree :-)

Actually I mistyped -- for developers (or potential developers) I think 
py.test's full featureset and frontend should be available.  However, 
for mere library users there's a use case where they want to simply know 
that the tests pass, that the package is ok; failures are not expected. 
  At least, when I was talking about this with some Perl people, they 
specifically said they like that CPAN automatically runs tests when you 
download a package.

I think a uniform command-line interface is nice in this case; the user 
is just look for "ok" or "not ok", they aren't looking for a development 
tool.  And it's possible that "setup.py test" should be that uniform 
interface.  That doesn't exclude the possibility of a separate command 
like "setup.py pytest".

>>I suppose a package could add an entry point that overrides
>>the normal setuptools test command...?
> 
> Yes, i guess something needs to be configurable there as far as i
> understand the situation.  I also presume that "setup.py test"
> would allow a custom test method to actually perform the execution
> of tests, not just provide a TestSuite.  
> 
> If so, it should at best become a simple matter of how a
> package can signal to setuptools that it wants its tests to be
> handled by py.test in which case the work would be defered to
> the (neccessarily) installed py library when "setup.py test"
> is invoked.  It shouldn't be required from each application
> to provide this glue code as it should be generic. 
> 
> actually, how does "setup.py test" work today and which
> applications/projects are integrated with it if i may ask? 

AFAIK no applications are integrated.  Well, maybe PEAK... this is all 
rather new.  It's documented here:

http://peak.telecommunity.com/DevCenter/setuptools#test-build-package-and-run-a-unittest-suite

However, I believe each command to setup.py is simply an "entry_point". 
  I can't find documentation for that at the moment.  But it's something 
like this in the setup.py file:

   entry_points={
     'distutils.commands': [
       'test=test_module:test_command',
     ],
   }

And then that test_command object (a class) has a specific interface 
(based on the distutils.Command class).  So potentially a py.test-using 
project could do:

   entry_points={
     'distutils.commands': [
       'test=mypkg.commands:test']}

   # in mypkg.commands:
   from py.test.disutils import test


I think that would override "setup.py test" for just that package.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pobrien at orbtech.com  Fri Aug 12 22:06:27 2005
From: pobrien at orbtech.com (Patrick K. O'Brien)
Date: Fri, 12 Aug 2005 15:06:27 -0500
Subject: [Distutils] [py-dev] Re:  setuptools presentation
In-Reply-To: <42FCF9EE.6070401@colorstudy.com>
References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
	<42FCF9EE.6070401@colorstudy.com>
Message-ID: <42FD0143.9020904@orbtech.com>

Ian Bicking wrote:
> CC'ing to bring together opposing viewpoints... (this is referring to
> setuptools' "setup.py test" command, which expects to have a TestSuite
> to run)
> 
> Phillip J. Eby wrote:
> 
>> Also, the 'test' command isn't "limited" to unittest, it just expects
>> a unittest-compatible wrapper.  For example, doctest has functions
>> that wrap doctests as unittest suites, so it's quite possible to use
>> that.  As far as I'm concerned, unittest suites are the WSGI of
>> testing; if somebody creates a fancy new test framework, they should
>> darn well make it possible to put a unittest suite wrapper around it,
>> so that people can integrate their existing tests.  :)
> 
> Well, I don't know what exactly my opinion is.  At a py.test user I
> don't have TestSuites for my tests.  I've argued py.test should load
> TestCase-based tests by default, but this is kind of the opposite.  I
> don't think it would be that hard to produce such suites; the test items
> that py.test collects could just be stuffed into a TestCase.
> 
> The result would be acceptable as a sort of "this package thinks it runs
> okay" test.  It's not the frontend I'd like to give to users.
> 
> That said, if "python setup.py test ..." was completely equivalent to
> "py.test ..." then that would be great, because though the interface
> would be different from projects that use unittest, the entry point
> would be the same (assuming py.test is installed).  I suppose a package
> could add an entry point that overrides the normal setuptools test
> command...?

+1 on the latter (make "python setup.py test ..." equivalent to "py.test
...") over the former.  I'd hate to see py.test have to conform to the
awkward conventions introduced by unittest.

Also, the MEMS folks have that Sancho thing, which I seem to recall is a
testing framework.  Might want to get their opinion on that matter, as
they use that testing convention for Durus, Quixote, and other packages.

-- 
Patrick K. O'Brien
Orbtech    http://www.orbtech.com
Schevo     http://www.schevo.org
Pypersyst  http://www.pypersyst.org


From pje at telecommunity.com  Fri Aug 12 22:50:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 12 Aug 2005 16:50:38 -0400
Subject: [Distutils] [py-dev] Re:  setuptools presentation
In-Reply-To: <42FD06E9.70405@colorstudy.com>
References: <20050812201022.GC11996@solar.trillke.net>
	<5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
	<42FCF9EE.6070401@colorstudy.com>
	<20050812201022.GC11996@solar.trillke.net>
Message-ID: <5.1.1.6.0.20050812164016.02937150@mail.telecommunity.com>

At 03:30 PM 8/12/2005 -0500, Ian Bicking wrote:
>However, I believe each command to setup.py is simply an "entry_point".  I 
>can't find documentation for that at the moment.  But it's something like 
>this in the setup.py file:
>
>   entry_points={
>     'distutils.commands': [
>       'test=test_module:test_command',
>     ],
>   }

Yes, that's exactly it.


>And then that test_command object (a class) has a specific interface 
>(based on the distutils.Command class).  So potentially a py.test-using 
>project could do:
>
>   entry_points={
>     'distutils.commands': [
>       'test=mypkg.commands:test']}
>
>   # in mypkg.commands:
>   from py.test.disutils import test
>
>
>I think that would override "setup.py test" for just that package.

Actually, it wouldn't, because the first 'test' command defined on sys.path 
will get used.  This doesn't mean that we can't have define test entry 
points, though, that the setuptools 'test' command would then use to run 
other testing frameworks.  For example, the builtin 'test' command could 
look up a 'test' command in the just-built package and run that.  The 
problem is that the master test command will have to define what options it 
takes, because distutils doesn't delegate option parsing to the commands.  :(

But setuptools does offer another solution.  In py.test's you could create 
a 'py.test' setup command, and projects that want to use it can do:

     setup(
         ...
         setup_requires=['py-test>=someversion'],
         project_args_for_py_test_here="something",
     )

They can also define a project alias in setup.cfg:

     [aliases]
     test = py.test

This will make "setup.py test" run the "py.test" command instead.

But to me, the simple and obvious thing to do is to make test suites.  If 
py.test exports a zero-argument function that finds tests in the current 
directory and returns a suite, then a py.test-using package need only set 
'test_suite="py.test.get_suite"' in order to use it, with no need to define 
an alias, nor would py.test need to define another distutils command.  And, 
for basic "does this work?" testing, users get a uniform interface for the 
'test' command.


From pje at telecommunity.com  Fri Aug 12 23:14:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 12 Aug 2005 17:14:38 -0400
Subject: [Distutils] [py-dev] Re:  setuptools presentation
In-Reply-To: <20050812201022.GC11996@solar.trillke.net>
References: <42FCF9EE.6070401@colorstudy.com>
	<5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com>
	<42FCF9EE.6070401@colorstudy.com>
Message-ID: <5.1.1.6.0.20050812163037.029a96f0@mail.telecommunity.com>

At 10:10 PM 8/12/2005 +0200, holger krekel wrote:
>I agree without having looked deeply into it.  Providing TestSuite
>instances that adapt running tests in a simple way with part
>of the py.test machinery should be quite feasible.  Would probably
>mean that some of py.test's execution features would be switched
>off (like stdout/stderr capturing).

You would just need to have each test object do its capturing between its 
calls to result.startTest and result.stopTest - I don't see that it would 
require switching anything off entirely.  Just treat each test as the place 
for setup/teardown of any special processing.

You might take a look at the doctest module, which provides similar 
wrappers.  Doctests of course capture stdout, but this is done within each 
"test" from the point of view of the unittest text or GUI test runners.


>However, as Ian hints at, py.test itself is already a project independent
>entry point for running tests in a given directory or for a given project.

Many projects distinguish functional or acceptance tests from their unit 
tests, so running *all* the tests isn't necessarily a good thing.  The 
point of the current "setup.py test" command in setuptools is to provide a 
way to define what the default on-installation tests are.  The test command 
also allows you to specify a different starting point as well, with the -s 
option, so that you can tell a user to run a specific test or suite.


> > That said, if "python setup.py test ..." was completely equivalent to
> > "py.test ..." then that would be great, because though the interface
> > would be different from projects that use unittest, the entry point
> > would be the same (assuming py.test is installed).
>
>would make sense, i think.
>
> > I suppose a package could add an entry point that overrides
> > the normal setuptools test command...?
>
>Yes, i guess something needs to be configurable there as far as i
>understand the situation.  I also presume that "setup.py test"
>would allow a custom test method to actually perform the execution
>of tests, not just provide a TestSuite.

It certainly could be done that way; my take on it is that both of the test 
frameworks in the stdlib support providing TestSuites, making it appear to 
be the One Obvious Way to do it.  Providing TestSuites means that your 
tests can be run in conjunction with unittest and doctest suites in any 
unittest-compatible test runner.


>If so, it should at best become a simple matter of how a
>package can signal to setuptools that it wants its tests to be
>handled by py.test in which case the work would be defered to
>the (neccessarily) installed py library when "setup.py test"
>is invoked.  It shouldn't be required from each application
>to provide this glue code as it should be generic.

If py.test exposes a zero-argument function returning a test suite 
constructed from the default py.test approach, then a user need only specify:

    test_suite = "py.test.default_suite"

in their setup() in order to use it.


>actually, how does "setup.py test" work today and which

It loads the suite named in setup() or via the '-s' command line option 
(using the default unittest test loader), and runs the unittest text-based 
test runner on it.  (Note that this means that project management tools 
(like an IDE) could also run a GUI test runner against the same suite -- 
something that's not trivially achievable with tests that aren't 
TestSuite-compatible.)


>applications/projects are integrated with it if i may ask?

The PEAK family of projects (including setuptools itself, PyProtocols, 
PEAK, RuleDispatch, etc.) have been using it as long as setuptools has 
existed (~18 months).

I'm willing to be flexible about the exact machinery invoked for tests, but 
I'd very much like to encourage "third-party" test frameworks to consider 
integrating with unittest (and thereby with all the other test frameworks 
that do), rather than encourage web-style proliferation of incompatible 
frameworks. :)  doctest already can be called from unittest, so it seems 
like a good example for other test frameworks to follow.  And I know that 
for me at least, no other framework is going to lure me away from unittest 
unless it allows me to run its tests as suites; I have way too many 
unittest-based tests I'm not going to go back and port.  (I never even 
bothered *trying* doctest until it was possible to integrate it with my 
existing unittest-based suites.)


From hawk78_it at yahoo.it  Sat Aug 13 18:57:29 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Sat, 13 Aug 2005 18:57:29 +0200
Subject: [Distutils] Dependencies
Message-ID: <200508131857.31174.hawk78_it@yahoo.it>

Hi,
Are dependencies taken into account inside PyPI records?
I mean both build dependencies and run dependencies.

If yes where can I find more info? How do setuptools manage it, how can I access this info?

Ciao
Vincenzo

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From pje at telecommunity.com  Sat Aug 13 21:08:14 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 13 Aug 2005 15:08:14 -0400
Subject: [Distutils] Dependencies
In-Reply-To: <200508131857.31174.hawk78_it@yahoo.it>
Message-ID: <5.1.1.6.0.20050813150548.025efe50@mail.telecommunity.com>

At 06:57 PM 8/13/2005 +0200, Vincenzo Di Massa wrote:
>Hi,
>Are dependencies taken into account inside PyPI records?

No.

>I mean both build dependencies and run dependencies.

Neither.


>If yes where can I find more info? How do setuptools manage it, how can I 
>access this info?

Setuptools lets you declare this information in the setup script; see the 
setuptools.txt file for details.  The "egg_info" command writes the data to 
a file, that is then read by other commands and also put into the built 
egg's metadata.

This information isn't something really suitable for PyPI, because the 
setup script is allowed to compute the information dynamically.  (Some 
dependencies might apply only to a particular platform, for example.)


From hawk78_it at yahoo.it  Sat Aug 13 23:18:25 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Sat, 13 Aug 2005 23:18:25 +0200
Subject: [Distutils] [Bulk] Re:  Dependencies
In-Reply-To: <5.1.1.6.0.20050813150548.025efe50@mail.telecommunity.com>
References: <5.1.1.6.0.20050813150548.025efe50@mail.telecommunity.com>
Message-ID: <200508132318.25874.hawk78_it@yahoo.it>

Alle 21:08, sabato 13 agosto 2005, Phillip J. Eby ha scritto:
> At 06:57 PM 8/13/2005 +0200, Vincenzo Di Massa wrote:
> >Hi,
> >Are dependencies taken into account inside PyPI records?
> 
> No.
> 
> >I mean both build dependencies and run dependencies.
> 
> Neither.
> 
> 
> >If yes where can I find more info? How do setuptools manage it, how can I 
> >access this info?
> 
> Setuptools lets you declare this information in the setup script; see the 
> setuptools.txt file for details.  The "egg_info" command writes the data to 
> a file, that is then read by other commands and also put into the built 
> egg's metadata.
> 
> This information isn't something really suitable for PyPI, because the 
> setup script is allowed to compute the information dynamically.  (Some 
> dependencies might apply only to a particular platform, for example.)

Is there a a way(API) to interact with this information?
My goal would be creating a database of correspondences between setuptools/PyPI(one day) dependency and debian package names.
The software I am coding could be used by other packaging systems(rpm?) too. To create a package of a python module (immagine an egg inside a deb or rpm) 
one could just use setuptools and my code. But I need to resolve dependencies: this is why I'll create a database of correspondences.

Ciao
Vincenzo

> 
> 

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From pje at telecommunity.com  Sun Aug 14 01:05:10 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 13 Aug 2005 19:05:10 -0400
Subject: [Distutils] [Bulk] Re:  Dependencies
In-Reply-To: <200508132318.25874.hawk78_it@yahoo.it>
References: <5.1.1.6.0.20050813150548.025efe50@mail.telecommunity.com>
	<5.1.1.6.0.20050813150548.025efe50@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050813185402.025df6b0@mail.telecommunity.com>

At 11:18 PM 8/13/2005 +0200, Vincenzo Di Massa wrote:
>Alle 21:08, sabato 13 agosto 2005, Phillip J. Eby ha scritto:
> > At 06:57 PM 8/13/2005 +0200, Vincenzo Di Massa wrote:
> > >If yes where can I find more info? How do setuptools manage it, how can I
> > >access this info?
> >
> > Setuptools lets you declare this information in the setup script; see the
> > setuptools.txt file for details.  The "egg_info" command writes the 
> data to
> > a file, that is then read by other commands and also put into the built
> > egg's metadata.
> >
> > This information isn't something really suitable for PyPI, because the
> > setup script is allowed to compute the information dynamically.  (Some
> > dependencies might apply only to a particular platform, for example.)
>
>Is there a a way(API) to interact with this information?
>My goal would be creating a database of correspondences between 
>setuptools/PyPI(one day) dependency and debian package names.
>The software I am coding could be used by other packaging systems(rpm?) 
>too. To create a package of a python module (immagine an egg inside a deb 
>or rpm)
>one could just use setuptools and my code. But I need to resolve 
>dependencies: this is why I'll create a database of correspondences.

The "requires()" method of pkg_resources.Distribution objects lets you 
obtain a sequence of pkg_resources.Requirement objects for a given physical 
distribution.  It can also be used to obtain information about optional 
requirements.

I'm in the process of creating a manual for pkg_resources that explains how 
to use many of these things; it's pkg_resources.txt in the setuptools CVS, 
but so far its information is still very sparse.


From pje at telecommunity.com  Sun Aug 14 19:57:24 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 13:57:24 -0400
Subject: [Distutils] pkg_resources API reference now available
Message-ID: <5.1.1.6.0.20050814134804.025dee30@mail.telecommunity.com>

I've just checked in the completion of a pkg_resources API reference.  In 
case you'd like to see the HTML version, or if SourceForge hasn't updated 
anonymous CVS yet, you can see it at:

     http://peak.telecommunity.com/DevCenter/PkgResources

Keep in mind that only the API reference is completed, and it turned out to 
be a much denser, far more interconnected document than I imagined it would 
be.  My plan is to write a short architecture overview and a 
tutorial/FAQ-style "developer's guide" to go with this, so that most people 
won't have to slog through the API reference just to learn how to use the 
basic features.  But, I wanted to start by writing the reference section so 
that I'd know what stuff I could then leave out of those other guides.  :)

However, if your goal is to extend pkg_resources or setuptools in some way, 
create new distribution formats, support custom importers, or create 
package management tools, you will probably benefit greatly from taking the 
time to read the full API reference.  (Especially since the API reference 
is less text to read than the pkg_resources source code!)


From hawk78_it at yahoo.it  Sun Aug 14 22:58:02 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Sun, 14 Aug 2005 22:58:02 +0200
Subject: [Distutils] Metadata.
Message-ID: <200508142258.02230.hawk78_it@yahoo.it>

Hi,
Thank you Phillip for your help and support!

I would like to access a package metadata before  it is installed:

I use the following class

####################CLASS
class Easy_deb:
    def __init__(self,req_string):
        self.requirement    = pkg_resources.Requirement.parse(req_string)
        self.index          = PackageIndex()
        self.found          = None
        self.dest           = None
        self.filename       = None
        self.source_dir     = None
        
    def download_source(self):
        self.index.find_packages(self.requirement)
        self.found=self.index.obtain(self.requirement)
        if self.found:
            self.dest='deb-pkg-'+self.found.key+self.found.version
        if os.path.isdir(self.dest):
            shutil.rmtree(self.dest)
        os.makedirs(self.dest)
        self.filename=self.index.download(self.found.location, self.dest)

    def unpack(self):
        unpack_archive(self.filename, self.dest)
        setups = glob.glob(os.path.join(self.dest, '*', 'setup.py'))
        self.source_dir = os.path.dirname(setups[0])

    def deb_src(self):
        debian_dir=os.path.join(self.source_dir,'debian')
        shutil.copytree('template',debian_dir)
        #file=os.path.join(debian_dir,"control")
        #print file
        sed(os.path.join(debian_dir,"control"),"%%PKGNAME%%",self.requirement.key)  <------ SED LINE
        os.chdir(self.source_dir)
        
        #inst_dir=os.path.join('./debian','tmp')
        #main(['-d']+[inst_dir,'.'])
            
    def create(self):
        self.download_source()
        self.unpack()
        self.deb_src()
####################END OF CLASS

to download modules and create a deb package.
I need to retrieve other metadata like description, author and license.
I nned it to put it into lines like SED LINE.

Thank you again
Vincenzo

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From pje at telecommunity.com  Sun Aug 14 23:13:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 17:13:47 -0400
Subject: [Distutils] Metadata.
In-Reply-To: <200508142258.02230.hawk78_it@yahoo.it>
Message-ID: <5.1.1.6.0.20050814170137.02809c68@mail.telecommunity.com>

At 10:58 PM 8/14/2005 +0200, Vincenzo Di Massa wrote:
>     def download_source(self):
>         self.index.find_packages(self.requirement)

The above line is redundant; obtain() will call find_packages() 
automatically.  Also, you may not want to use obtain anyway.  For what you 
are trying to do, I would suggest running "easy_install --editable 
--build_dir=somewhere requirement", because then it will handle all the 
details of finding and unpacking the source.  The source tree will then be 
"somewhere/projectname" (the project name will be all-lowercase).  You 
could use a temporary directory for "somewhere", if you like.  In any case 
this will take care of most of the things you're doing in this code outside 
of the deb_src() method.


>I need to retrieve other metadata like description, author and license.
>I nned it to put it into lines like SED LINE.

Well, you can always run the package's setup script, and in fact you will 
need to do so if you're working with an unbuilt source distribution.  Most 
of the PKG-INFO metadata items can be displayed using options to setup.py, 
however.  For example, running this in the setuptools directory:

    python setup.py --name --version --author --license --url

produces this output:

    setuptools
    0.6a0
    Phillip J. Eby
    PSF or ZPL
    http://peak.telecommunity.com/DevCenter/setuptools

Use setup.py --help to get a list of the available options.

Note that you may want to pass the name and version results through 
pkg_resources' safe_name() and safe_version() functions.


From pje at telecommunity.com  Sun Aug 14 23:35:51 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 17:35:51 -0400
Subject: [Distutils] Experimental non-empty namespace package support
Message-ID: <5.1.1.6.0.20050814171834.028847f0@mail.telecommunity.com>

I've just checked in experimental support for lazy, non-empty namespace 
packages.  And what is that?, you might well ask.

Well, it seems that livinglogic.de distributes certain Python packages 
using a distutils kludge that allows a kind of crude namespace package to 
exist, without using pkg_resources.  Specifically, their 'll-core' project 
distributes the 'll' package, and various other projects such as 'll-color' 
distribute modules for the 'll' package, but *without including an 
__init__.py*.  This allows the distutils to install the modules without 
overwriting or duplicating the single __init__.py file.

This is an interesting approach to addressing namespace packages in a 
pre-setuptools world, but until now it hasn't been really usable with 
setuptools, for several reasons:

   * pkg_resources only loaded one __init__ file for a given package

   * pkg_resources couldn't import modules from a zipfile or directory with 
no __init__.py

   * pkg_resources automatically imports registered namespace packages as 
soon as they're discovered, which is not a big deal for empty namespace 
packages, but could easily become quite problematic for ones like 'll' that 
contain actual code.

So, here's the solution:

    * pkg_resources now loads all __init__.py's from all the distributions 
that provide contents for the module.

    * the bdist_egg command automatically generates a dummy __init__.py for 
packages that don't have them.  The __init__.py file contains the single line:

      __import__('pkg_resources').declare_namespace(__name__)

      This ensures that the other package contents are importable, and that 
as soon as an import of the package occurs, it gets registered as a 
namespace package.

    * projects that use this approach to namespace packaging must *NOT* 
pass a 'namespace_packages' argument to setup(), because that would cause 
the package to be imported even when it isn't necessary.  They must also 
put a 'declare_namespace()' call in all of their __init__.py's, to ensure 
that the package will become a namespace no matter what the order of 
distributions on sys.path is.

Because of the complexity of this approach, I do not recommend it for new 
projects.  Namespace packages should not have code in any of their 
__init__.py files, as it eliminates all of these headaches.  So, this is an 
experimental feature intended to facilitate backward compatibility 
only.  Don't expect it to show up in the documentation any time soon!

Anyway, I was able to get this to work for ll-core and ll-color, as long as 
I added a 'declare_namespace()' call like the one shown above to ll-core's 
__init__.py, and made them both use setuptools.  I would appreciate any 
feedback that folks have on this.

By the way, because SourceForge CVS updating is slow, it may be a few hours 
before you can check out the version of setuptools that supports 
this.  Make sure you have pkg_resources.py revision 1.69 or higher, and 
setuptools/command/bdist_egg.py revision 1.28 or higher, if you want to 
experiment.  Thanks.


From pje at telecommunity.com  Mon Aug 15 01:06:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 19:06:38 -0400
Subject: [Distutils] File integrity checking and host blocking for
	EasyInstall
Message-ID: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>

After thinking over the last week's distutils-sig discussion about 
security, signatures, etc., I think I have a plan for handling basic file 
integrity checking and (non-cryptographic) trust management for 
EasyInstall.  It is not a high-security end-to-end solution, but I think it 
will allow security-conscious persons to take a more "locked down" approach 
if they want to, while providing everyone else with some baseline 
protection against corrupted files.

The first part of the plan is to add md5 digest checking to 
EasyInstall.  Because one of EasyInstall's design goals is to make it easy 
for anybody to publish links to packages, we need to be able to include the 
md5 signature in a package's URL.  I'm thinking we could achieve this via 
an '#md5=...' fragment identifier.  For example, a setuptools source 
archive URL might be:

http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md5=91f31a9058330174640a867cf5d4de57

The advantage of this approach is that it allows anyone to assert what the 
md5 of the targeted file is, and it can be asserted in any web page, just 
by pointing an HREF at the file.  EasyInstall could detect the '#md5=' 
marker, and then use this to verify the file during download.

The disadvantage, of course, is that PyPI doesn't currently support this; 
it creates a separate link to a page that displays the md5, and that URL 
doesn't contain anything that connects it back to the distribution file it 
refers to.  I could probably create some kind of parsing hack to fix that 
for PyPI, but it seems it might be worth adding the #md5 trick to PyPI to 
support this.

EasyInstall would also need to grow a --require-md5 option, which would 
refuse to install anything from a Subversion checkout or a distribution 
without a known md5 signature.

In addition to md5 support in EasyInstall, I propose to also add it to 
ez_setup; there, however, the md5 values for various distributions will be 
hardcoded into ez_setup.py itself.  (I'll make my "release" script append 
the md5 signatures for new distributions to the end of ez_setup.py.)  In 
this way, the bootstrap installation of setuptools can also be reasonably 
secured, as long as you trust a particular version of ez_setup.py.

The next part of the plan would be to add an --allow-hosts option to 
EasyInstall.  This would be a list of host wildcards that EasyInstall would 
be allowed to contact.  For example, --allow-hosts=*.python.org would let 
EasyInstall download or scan pages from PyPI or www.python.org, but not 
anywhere else.  The default, if not specified, would be '*', meaning that 
any host may be accessed.  If EasyInstall finds itself about to download a 
page or distribution from a host that isn't allowed, it will abort with a 
message explaining the problem.

This would allow folks like Paul Moore to configure a default --allow-hosts 
list in their pydistutils.cfg, to prevent EasyInstall from downloading 
things from just any old place on the Internet.  Once he's verified that he 
trusts a particular site, he can edit pydistutils.cfg and add it, or else 
manually download the blocked URL, publish it on a trusted intranet host, etc.

So, this is not a complete security solution, as it doesn't deal with 
end-to-end file integrity, and could easily be subverted by taking over a 
site somewhere in the middle (e.g. python.org).  But until we have more of 
the cryptographic infrastructure in place, I think this plan could provide 
us with a good starting point.  Comments, anyone?


From ianb at colorstudy.com  Mon Aug 15 03:38:31 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 14 Aug 2005 20:38:31 -0500
Subject: [Distutils] easy_install: link detection
Message-ID: <42FFF217.3040407@colorstudy.com>

easy_install doesn't work automatically with elementtree, because its 
download link leads to an interstitial HTML page.  So I thought I'd add 
the proper link to an index I'm keeping:
   http://effbot.org/downloads/elementtree-1.2.6-20050316.zip

But easy_install ignores it.  It looks fine to me?

Also, if I want to require a package that isn't locatable through PyPI, 
how should I deal with that?  I can add a find_links value to setup.cfg, 
then put the link there.  It'd be easier if I could put a URL to the 
package somewhere.  Or I guess if I could have a sort of local index. 
But if I put "./docs/packages.html" as a find_links value, easy_install 
can't find that.  So, what's the best way to deal with that?  I'm okay 
with the index page myself, but I think other people may not want to 
maintain such a thing.  (I'm thinking of writing a little app to create 
a index for broken PyPI entries, but that's another topic.)

And lastly on this topic, what should I do when it's likely a package 
will be installed without setuptools/easy_install?  For something like 
ElementTree there are lots of packages (RPM, deb, etc) that install the 
package; I can't really ask people to uninstall those and install the 
egg version.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 15 04:05:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 22:05:39 -0400
Subject: [Distutils] easy_install: link detection
In-Reply-To: <42FFF217.3040407@colorstudy.com>
Message-ID: <5.1.1.6.0.20050814215641.02861250@mail.telecommunity.com>

At 08:38 PM 8/14/2005 -0500, Ian Bicking wrote:
>easy_install doesn't work automatically with elementtree, because its
>download link leads to an interstitial HTML page.  So I thought I'd add
>the proper link to an index I'm keeping:
>    http://effbot.org/downloads/elementtree-1.2.6-20050316.zip
>
>But easy_install ignores it.  It looks fine to me?

Heck if I know.  I tried:

     easy_install.py 
-vnfhttp://effbot.org/downloads/elementtree-1.2.6-20050316.zip elementtree

and it worked just fine.  Got the url of a page I could use to 
test?  Perhaps it's something in the surrounding HTML that's at issue.


>Also, if I want to require a package that isn't locatable through PyPI,
>how should I deal with that?  I can add a find_links value to setup.cfg,
>then put the link there.  It'd be easier if I could put a URL to the
>package somewhere.  Or I guess if I could have a sort of local index.
>But if I put "./docs/packages.html" as a find_links value, easy_install
>can't find that.  So, what's the best way to deal with that?  I'm okay
>with the index page myself, but I think other people may not want to
>maintain such a thing.

You can just put all the links directly in the --find-links value, you 
know.  EasyInstall doesn't install links that it can tell are distributions.


>(I'm thinking of writing a little app to create
>a index for broken PyPI entries, but that's another topic.)

Make sure you add an option to send the owner of the broken entry an email 
every time you have to update your index.  :)


>And lastly on this topic, what should I do when it's likely a package
>will be installed without setuptools/easy_install?  For something like
>ElementTree there are lots of packages (RPM, deb, etc) that install the
>package; I can't really ask people to uninstall those and install the
>egg version.

I'm still looking at adding a feature to allow projects to support 
detecting "legacy" installations of packages.  There are still a few design 
kinks I want to figure out, so it probably won't be in 0.6a1 I'm afraid.

The basic idea is going to be that you should be able to add dummy 
Distribution objects to the global working_set to represent legacy 
installations.  But the devil is still in the details, mostly surrounding 
what the API should look like and how you get this stuff into a 
package.  For example, if you depend on ElementTree, there should be a way 
for you to include a code snippet that checks for its presence and makes 
the dummy Distribution.  But, you shouldn't have to code to the raw 
pkg_resources API to do that; instead, some helper functions are needed so 
that the part you write can be just a single API call for common cases 
(like importing a version from some module and checking what version it is).


From ianb at colorstudy.com  Mon Aug 15 04:16:28 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 14 Aug 2005 21:16:28 -0500
Subject: [Distutils] easy_install: link detection
In-Reply-To: <5.1.1.6.0.20050814215641.02861250@mail.telecommunity.com>
References: <5.1.1.6.0.20050814215641.02861250@mail.telecommunity.com>
Message-ID: <42FFFAFC.3090703@colorstudy.com>

Phillip J. Eby wrote:
> At 08:38 PM 8/14/2005 -0500, Ian Bicking wrote:
> 
>> easy_install doesn't work automatically with elementtree, because its
>> download link leads to an interstitial HTML page.  So I thought I'd add
>> the proper link to an index I'm keeping:
>>    http://effbot.org/downloads/elementtree-1.2.6-20050316.zip
>>
>> But easy_install ignores it.  It looks fine to me?
> 
> 
> Heck if I know.  I tried:
> 
>     easy_install.py 
> -vnfhttp://effbot.org/downloads/elementtree-1.2.6-20050316.zip elementtree
> 
> and it worked just fine.  Got the url of a page I could use to test?  
> Perhaps it's something in the surrounding HTML that's at issue.

It's at http://pythonpaste.org/package_index.html

>> Also, if I want to require a package that isn't locatable through PyPI,
>> how should I deal with that?  I can add a find_links value to setup.cfg,
>> then put the link there.  It'd be easier if I could put a URL to the
>> package somewhere.  Or I guess if I could have a sort of local index.
>> But if I put "./docs/packages.html" as a find_links value, easy_install
>> can't find that.  So, what's the best way to deal with that?  I'm okay
>> with the index page myself, but I think other people may not want to
>> maintain such a thing.
> 
> 
> You can just put all the links directly in the --find-links value, you 
> know.  EasyInstall doesn't install links that it can tell are 
> distributions.

OK, that'll do (probably putting the link in setup.cfg).

>> (I'm thinking of writing a little app to create
>> a index for broken PyPI entries, but that's another topic.)
> 
> 
> Make sure you add an option to send the owner of the broken entry an 
> email every time you have to update your index.  :)

Yeah... at the moment it's so many it's hard to be mad at them.  I 
assume in the ElementTree case it's an attempt to be helpful, even if 
personally it invokes a "just give the file already" reaction.

Personally I'm expecting to use "setup.py upload" for all my future 
distribution, which should make it easier.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From hawk78_it at yahoo.it  Mon Aug 15 11:54:43 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Mon, 15 Aug 2005 11:54:43 +0200
Subject: [Distutils] Python version.
Message-ID: <200508151154.44007.hawk78_it@yahoo.it>

Hi,
I have another question: how do I find what python version a module on pypi requires?
If I try to install module graph in python2.3 it just say that @proprty is invalid synatx.
How can I detect this?

Thank you

Vincenzo

		
___________________________________ 
Yahoo! Messenger: chiamate gratuite in tutto il mondo 
http://it.beta.messenger.yahoo.com

From bob at redivi.com  Mon Aug 15 12:45:36 2005
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 15 Aug 2005 00:45:36 -1000
Subject: [Distutils] Python version.
In-Reply-To: <200508151154.44007.hawk78_it@yahoo.it>
References: <200508151154.44007.hawk78_it@yahoo.it>
Message-ID: <D0D99835-1561-4CD5-96DE-0D3A5D6747FE@redivi.com>


On Aug 14, 2005, at 11:54 PM, Vincenzo Di Massa wrote:

> Hi,
> I have another question: how do I find what python version a module  
> on pypi requires?
> If I try to install module graph in python2.3 it just say that  
> @proprty is invalid synatx.
> How can I detect this?

I don't know if PyPI (Cheese Shop?) has that kind of metadata, but if  
something has @decorator syntax, then it needs Python 2.4.

-bob


From parlar at gmail.com  Tue Aug 16 15:28:58 2005
From: parlar at gmail.com (Jay Parlar)
Date: Tue, 16 Aug 2005 09:28:58 -0400
Subject: [Distutils] py2exe and eggs
Message-ID: <e15ccc4d05081606283b4934c2@mail.gmail.com>

I know this topic has briefly been covered before
(http://mail.python.org/pipermail/distutils-sig/2005-July/004697.html)
but I'm having some trouble with it.

I have an in-house app that uses RuleDispatch (and of course
PyProtocols). I used to just build them from cvs myself, but now I've
switched to using the eggs. Since I'm doing all the development on my
machine, and no one else will see my scripts, I don't worry about
pkg_resources.require(), I just simply import 'dispatch' and
'protocols'.

py2exe doesn't like this at all though. Whatever mechanism it uses for
pulling in dependencies doesn't work with eggs.

If there is a solution, I'd prefer *not* to have to do anything
specific in my setup.py, for each egg. As time goes on, I'd like to
move more and more of my site-packages to eggs, and I don't want to
have to modify my setup.py everytime I switch another library to an
egg.

Any solutions to this? I don't mind a solution that requires me to add
something to setup.py that can automatically find any of the eggs I
use, but I'd really like to avoid adding something to setup.py for
*each* egg I use.

Thanks in advance,
Jay P.

From pje at telecommunity.com  Tue Aug 16 16:47:12 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 16 Aug 2005 10:47:12 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d05081606283b4934c2@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050816104249.03079178@mail.telecommunity.com>

At 09:28 AM 8/16/2005 -0400, Jay Parlar wrote:
>Any solutions to this? I don't mind a solution that requires me to add
>something to setup.py that can automatically find any of the eggs I
>use, but I'd really like to avoid adding something to setup.py for
>*each* egg I use.

The require() function returns a list of all the distributions needed to 
satisfy the dependency or dependencies you gave it -- including recursive 
dependencies, even ones that were already on sys.path.  Distributions have 
a 'location' attribute that's their full filesystem path.  And, they have a 
'top_level.txt' metadata file (.get_metadata_lines('top_level.txt')) that 
lists the top-level module or package names exposed by that distribution.

Dunno if that helps you sufficiently or not.  I personally would love to 
see py2exe and py2app morph into distutils extensions using entry points to 
add their commands and setup() arguments, because then there'd be a minimum 
of monkeypatches dueling for control of the distutils.  :)


From dangoor at gmail.com  Tue Aug 16 17:05:37 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 16 Aug 2005 11:05:37 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d05081606283b4934c2@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
Message-ID: <3f085ecd0508160805e4500d3@mail.gmail.com>

On 8/16/05, Jay Parlar <parlar at gmail.com> wrote:
> Any solutions to this? I don't mind a solution that requires me to add
> something to setup.py that can automatically find any of the eggs I
> use, but I'd really like to avoid adding something to setup.py for
> *each* egg I use.

My solution to this is ugly, but it's not much code (about 50 lines of
trivial stuff) and didn't require delving into py2exe itself.

py2exe really tries to minimize your distribution size and get rid of
all unused modules. My method takes the eggs wholesale, so it's not
quite as space efficient.

Here's what I do:

1) get the list of Distributions using pkg_resources.require(), as
Phillip suggests

2) keep track of the packages in the egg (using the egg's
metadata.get_metadata_lines('top_level.txt'))

3) if the egg is pointing to a directory, add all of the files there
to data_files. If it's a zipped up egg, just add the .egg file to
data_files

4) finally, after building the exe, I create a new library.zip file
that doesn't contain the packages found in step 2. (So, it's not a
minimal set of packages and modules, but it at least isn't duplicating
packages!)

It's not pretty, but it does work. And, in my case, this process works
with both py2exe and py2app and was less work than modifying both of
those packages to do something with the eggs.

Kevin

From pje at telecommunity.com  Tue Aug 16 17:40:18 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 16 Aug 2005 11:40:18 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <3f085ecd0508160805e4500d3@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050816113925.03081858@mail.telecommunity.com>

At 11:05 AM 8/16/2005 -0400, Kevin Dangoor wrote:

>2) keep track of the packages in the egg (using the egg's
>metadata.get_metadata_lines('top_level.txt'))

FYI, you no longer need the 'metadata' attribute with the 0.6 series of 
setuptools; Distribution objects automatically delegate metadata requests 
to that attribute for you now.


From parlar at gmail.com  Tue Aug 16 18:23:47 2005
From: parlar at gmail.com (Jay Parlar)
Date: Tue, 16 Aug 2005 12:23:47 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <3f085ecd0508160805e4500d3@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
Message-ID: <e15ccc4d0508160923273ff4a1@mail.gmail.com>

I followed the steps, but I believe I'm missing something basic about
eggs in general.

The py2exe result now has the two egg files in it, but the only way
I'm actually able to import them is by doing:

sys.path.append('.')
pkg_resources.require("RuleDispatch")
pkg_resources.require("PyProtocols")


at the top of my main .py file. That doesn't seem like the right thing to do...

pkg_resources.find_distributions(".") can find the distibutions fine,
but how do I tell the runtime that it should look on "." for eggs,
without having to append to sys.path?

Thanks,
Jay P.

On 8/16/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> On 8/16/05, Jay Parlar <parlar at gmail.com> wrote:
> > Any solutions to this? I don't mind a solution that requires me to add
> > something to setup.py that can automatically find any of the eggs I
> > use, but I'd really like to avoid adding something to setup.py for
> > *each* egg I use.
> 
> My solution to this is ugly, but it's not much code (about 50 lines of
> trivial stuff) and didn't require delving into py2exe itself.
> 
> py2exe really tries to minimize your distribution size and get rid of
> all unused modules. My method takes the eggs wholesale, so it's not
> quite as space efficient.
> 
> Here's what I do:
> 
> 1) get the list of Distributions using pkg_resources.require(), as
> Phillip suggests
> 
> 2) keep track of the packages in the egg (using the egg's
> metadata.get_metadata_lines('top_level.txt'))
> 
> 3) if the egg is pointing to a directory, add all of the files there
> to data_files. If it's a zipped up egg, just add the .egg file to
> data_files
> 
> 4) finally, after building the exe, I create a new library.zip file
> that doesn't contain the packages found in step 2. (So, it's not a
> minimal set of packages and modules, but it at least isn't duplicating
> packages!)
> 
> It's not pretty, but it does work. And, in my case, this process works
> with both py2exe and py2app and was less work than modifying both of
> those packages to do something with the eggs.
> 
> Kevin
>

From dangoor at gmail.com  Tue Aug 16 18:36:02 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 16 Aug 2005 12:36:02 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d0508160923273ff4a1@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
Message-ID: <3f085ecd05081609364082e9f7@mail.gmail.com>

On 8/16/05, Jay Parlar <parlar at gmail.com> wrote:
> I followed the steps, but I believe I'm missing something basic about
> eggs in general.
> 
> The py2exe result now has the two egg files in it, but the only way
> I'm actually able to import them is by doing:
> 
> sys.path.append('.')
> pkg_resources.require("RuleDispatch")
> pkg_resources.require("PyProtocols")

Sounds like you're close, and it sounds like I left out an important step.

My code does actually have a pkg_resources.requre() statement in it.
But, looking back at my setup script, I neglected to mention this:

eggspth = open("build/eggs.pth", "w")
for egg in eggs:
    eggspth.write(os.path.basename(egg.path))
    eggspth.write("\n")

And then you add eggs.pth to your data_files. That should get rid of
your sys.path.append, and will quite possibly eliminate the need for
you to do pkg_resources.requre().

Kevin

From parlar at gmail.com  Tue Aug 16 19:10:28 2005
From: parlar at gmail.com (Jay Parlar)
Date: Tue, 16 Aug 2005 13:10:28 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <3f085ecd05081609364082e9f7@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
Message-ID: <e15ccc4d050816101045f5f109@mail.gmail.com>

Hmm... Even with eggs.pth in the resulting 'dist' directory, my
sys.path still doesn't show the eggs on it. Is there something you
have to do with py2exe to make it look for .pth files?

Jay P.

On 8/16/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> On 8/16/05, Jay Parlar <parlar at gmail.com> wrote:
> > I followed the steps, but I believe I'm missing something basic about
> > eggs in general.
> >
> > The py2exe result now has the two egg files in it, but the only way
> > I'm actually able to import them is by doing:
> >
> > sys.path.append('.')
> > pkg_resources.require("RuleDispatch")
> > pkg_resources.require("PyProtocols")
> 
> Sounds like you're close, and it sounds like I left out an important step.
> 
> My code does actually have a pkg_resources.requre() statement in it.
> But, looking back at my setup script, I neglected to mention this:
> 
> eggspth = open("build/eggs.pth", "w")
> for egg in eggs:
>     eggspth.write(os.path.basename(egg.path))
>     eggspth.write("\n")
> 
> And then you add eggs.pth to your data_files. That should get rid of
> your sys.path.append, and will quite possibly eliminate the need for
> you to do pkg_resources.requre().
> 
> Kevin
>

From pje at telecommunity.com  Tue Aug 16 19:25:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 16 Aug 2005 13:25:13 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d050816101045f5f109@mail.gmail.com>
References: <3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>

At 01:10 PM 8/16/2005 -0400, Jay Parlar wrote:
>Hmm... Even with eggs.pth in the resulting 'dist' directory, my
>sys.path still doesn't show the eggs on it. Is there something you
>have to do with py2exe to make it look for .pth files?

If all else fails, 'import site; addsitedir(".")' might do the trick.


From parlar at gmail.com  Tue Aug 16 19:48:38 2005
From: parlar at gmail.com (Jay Parlar)
Date: Tue, 16 Aug 2005 13:48:38 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
Message-ID: <e15ccc4d05081610487c6de2ed@mail.gmail.com>

On 8/16/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> Your eggs.pth should appear in the generated exe directory, right next
> to your exe file itself, and should have the names of the eggs without
> any other path on them (just the basename).

That's exactly what I've got, but it's a no-go. I'll keep trying.


On 8/16/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
> If all else fails, 'import site; addsitedir(".")' might do the trick.

That seems to work about as well as sys.path.append("."), but I still
need to do individual pkg_resources.require("...") calls in my main
file.

I guess until py2exe becomes egg aware, it's all going to be nasty hacks anyway.

Thanks for all the help.

Jay P.

From bai at kom.auc.dk  Thu Aug 18 14:04:12 2005
From: bai at kom.auc.dk (Bo Nygaard Bai)
Date: Thu, 18 Aug 2005 14:04:12 +0200
Subject: [Distutils] Hardlinks v copying
Message-ID: <4304793C.70403@kom.auc.dk>

I was trying to use the distutils sdist target on an AFS mounted
filesystem on Linux. It fails because it can't make hard links between
directories on AFS (you can inside a directory). I have not been able
to find any options to make it copy the files instead.

1) Is there a way for me to force it to copy the files into the
   temporary directory instead of using hard links?

2) Given that it is very hard to predict whether a particular usage of
   hard links is possible on any given OS/filesystem/mount combination.
   It seems to me that the best strategy would be to try to make the
   hard links, and if this fails, silently fall back to making a copy.

/Bo Bai

From cmlenz at gmx.de  Thu Aug 18 14:12:42 2005
From: cmlenz at gmx.de (Christopher Lenz)
Date: Thu, 18 Aug 2005 14:12:42 +0200
Subject: [Distutils] setuptools problem with --tag-svn-revision and
	localized svn messages
Message-ID: <8E734FB4-DCE3-4217-8DA7-022D40B21354@gmx.de>

Hi all,

I just tried to run the build of my project on a box that has  
localized SVN messages (German). As the tag_svn_revision option is  
enabled in the setup config, setuptools calls "svn info" and tries to  
parse the output, looking for "Last Changed Rev: (\d+)" string. In  
the german output this label is obviously different, so the parsing  
will fail and the build will abort.

I can temporarily switch subversion to use English messages for the  
build:

   $ export LC_MESSAGES=en_US
   $ ./setup.py test

But still, this should probably be fixed in setuptools.

Thanks,
Chris
--
Christopher Lenz
   cmlenz at gmx.de
   http://www.cmlenz.net/


From mal at egenix.com  Thu Aug 18 14:35:09 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 18 Aug 2005 14:35:09 +0200
Subject: [Distutils] Hardlinks v copying
In-Reply-To: <4304793C.70403@kom.auc.dk>
References: <4304793C.70403@kom.auc.dk>
Message-ID: <4304807D.6010102@egenix.com>

Bo Nygaard Bai wrote:
> I was trying to use the distutils sdist target on an AFS mounted
> filesystem on Linux. It fails because it can't make hard links between
> directories on AFS (you can inside a directory). I have not been able
> to find any options to make it copy the files instead.
> 
> 1) Is there a way for me to force it to copy the files into the
>    temporary directory instead of using hard links?
> 
> 2) Given that it is very hard to predict whether a particular usage of
>    hard links is possible on any given OS/filesystem/mount combination.
>    It seems to me that the best strategy would be to try to make the
>    hard links, and if this fails, silently fall back to making a copy.

How do you test whether a hard link works ? I suppose
this would have to be made based on the path since you
have mixed setups where some parts of the file system
tree allow hard links where others don't.

I don't find hard links all that useful. Disk space
is cheap and they often cause problems (e.g. in our
case with symlinks in the source tree).

Here's a quick hack:

import os
delattr(os, link)

This is more elegant, but requires extra work:

class mx_sdist(sdist):

    """ Build a source distribution.

        This version does not use hard links when preparing the source
        archive - hard links don't match well with symlinks which we
        use in our source repositories.

    """
    def make_release_tree (self, base_dir, files):

        # Prepare release dir
        self.mkpath(base_dir)
        self.distribution.metadata.write_pkg_info(base_dir)
        if not files:
            log.warn('no files found in release !')
            return

        # Create dir structure
        log.info('preparing release tree in %s...' % base_dir)
        create_tree(base_dir, files, dry_run=self.dry_run)
        for file in files:
            if not os.path.isfile(file):
                log.warn('%s is not a regular file -- skipping' % file)
            else:
                dest = os.path.join(base_dir, file)
                self.copy_file(file, dest, link=None)

and then register this command class as replacement for
'sdist'.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bai at kom.auc.dk  Thu Aug 18 15:31:46 2005
From: bai at kom.auc.dk (Bo Nygaard Bai)
Date: Thu, 18 Aug 2005 15:31:46 +0200
Subject: [Distutils] Hardlinks v copying
In-Reply-To: <4304807D.6010102@egenix.com>
References: <4304793C.70403@kom.auc.dk> <4304807D.6010102@egenix.com>
Message-ID: <43048DC2.3010302@kom.auc.dk>

M.-A. Lemburg wrote:

>Bo Nygaard Bai wrote:
>
>>I was trying to use the distutils sdist target on an AFS mounted
>>filesystem on Linux. It fails because it can't make hard links between
>>directories on AFS (you can inside a directory). I have not been able
>>to find any options to make it copy the files instead.
>>
>>1) Is there a way for me to force it to copy the files into the
>>   temporary directory instead of using hard links?
>>
>>2) Given that it is very hard to predict whether a particular usage of
>>   hard links is possible on any given OS/filesystem/mount combination.
>>   It seems to me that the best strategy would be to try to make the
>>   hard links, and if this fails, silently fall back to making a copy.
>
>
>How do you test whether a hard link works ? I suppose
>this would have to be made based on the path since you
>have mixed setups where some parts of the file system
>tree allow hard links where others don't.

I do not think it is possible (in the real world) to predict
when you can safely use a hard link. My suggestion was that
*if* hard links are to be used at all, the only reasonable way
to implement it would be to silently catch the exception when
creation of a link failed, and then try a copy instead.

>I don't find hard links all that useful. Disk space
>is cheap and they often cause problems (e.g. in our
>case with symlinks in the source tree).

I agree! In fact I would be quite happy with an implementation that
always did a copy, since copying is *much* more robust, and I doubt
that anybody has a python module that is big enough for the speed
difference to really matter.

> [... snip ... ]

Thanks for the plugin!

I must admit that I used the much less elegant solution of copying my
entire code base into /tmp to do the build there.

End result: Lots of copying to save a little copying ;-)

Would the maintainers consider not to use hard links?

It seems I am not the only one having problems because of them.

/Bo Bai


From pje at telecommunity.com  Thu Aug 18 18:39:15 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 12:39:15 -0400
Subject: [Distutils] setuptools problem with --tag-svn-revision and
 localized svn messages
In-Reply-To: <8E734FB4-DCE3-4217-8DA7-022D40B21354@gmx.de>
Message-ID: <5.1.1.6.0.20050818123824.01b34ce8@mail.telecommunity.com>

At 02:12 PM 8/18/2005 +0200, Christopher Lenz wrote:
>Hi all,
>
>I just tried to run the build of my project on a box that has
>localized SVN messages (German). As the tag_svn_revision option is
>enabled in the setup config, setuptools calls "svn info" and tries to
>parse the output, looking for "Last Changed Rev: (\d+)" string. In
>the german output this label is obviously different, so the parsing
>will fail and the build will abort.
>
>I can temporarily switch subversion to use English messages for the
>build:
>
>    $ export LC_MESSAGES=en_US
>    $ ./setup.py test
>
>But still, this should probably be fixed in setuptools.

So, I should set LC_MESSAGES to en_US when invoking svn?  Or is there 
something else I should be doing?


From cmlenz at gmx.de  Thu Aug 18 18:50:15 2005
From: cmlenz at gmx.de (Christopher Lenz)
Date: Thu, 18 Aug 2005 18:50:15 +0200
Subject: [Distutils] setuptools problem with --tag-svn-revision and
	localized svn messages
In-Reply-To: <5.1.1.6.0.20050818123824.01b34ce8@mail.telecommunity.com>
References: <5.1.1.6.0.20050818123824.01b34ce8@mail.telecommunity.com>
Message-ID: <42405192-C976-4D94-AFB0-02C59CBB895B@gmx.de>

Am 18.08.2005 um 18:39 schrieb Phillip J. Eby:
> At 02:12 PM 8/18/2005 +0200, Christopher Lenz wrote:
>> I just tried to run the build of my project on a box that has
>> localized SVN messages (German). As the tag_svn_revision option is
>> enabled in the setup config, setuptools calls "svn info" and tries to
>> parse the output, looking for "Last Changed Rev: (\d+)" string. In
>> the german output this label is obviously different, so the parsing
>> will fail and the build will abort.
>>
>> I can temporarily switch subversion to use English messages for the
>> build:
>>
>>    $ export LC_MESSAGES=en_US
>>    $ ./setup.py test
>>
>> But still, this should probably be fixed in setuptools.
>
> So, I should set LC_MESSAGES to en_US when invoking svn?  Or is  
> there something else I should be doing?

Yeah, that should do the trick.

Didn't find the time to attempt a patch yet, sorry. :-P

Cheers,
Chris
--
Christopher Lenz
   cmlenz at gmx.de
   http://www.cmlenz.net/


From theller at python.net  Thu Aug 18 19:08:49 2005
From: theller at python.net (Thomas Heller)
Date: Thu, 18 Aug 2005 19:08:49 +0200
Subject: [Distutils] setuptools test command on Windows
Message-ID: <zmrf6ula.fsf@python.net>

Seems I was late to try out setuptools.  Fantastic package, is my first
impression.  But I have still a lot to read and tryout.

But here is a first question: The 'test' command runs 'build_ext -i' and
then the tests.  Usually, I test my packages with several Python
versions.

Now, the problem is that the extension module is built in the source
directory, and is only compatible with exactly one python version.

This is on Windows, but I assume the problem would happen on other
platforms as well - the SF compilefarm uses the same home directory for
all the different platforms.  Binaries compiled for OSX Power PC are not
compatible with Linux AMD64, for example - but they all would have the
same filename.

This problem is not new, of course.

The workaround I have used for the ctypes project so far is a __path__
hack: the main module importing this extension tries to execfile() a
special file that is in CVS, but never distributed.  This file uses
distutils to determine the name of the build directory, inserts this
into sys.path.

Is there anything in setuptools to work around that problem, or are
there any plans or ideas how this could work?

Thomas


From pje at telecommunity.com  Thu Aug 18 19:25:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 13:25:58 -0400
Subject: [Distutils] setuptools test command on Windows
In-Reply-To: <zmrf6ula.fsf@python.net>
Message-ID: <5.1.1.6.0.20050818131711.01b29d18@mail.telecommunity.com>

At 07:08 PM 8/18/2005 +0200, Thomas Heller wrote:
>Seems I was late to try out setuptools.  Fantastic package, is my first
>impression.  But I have still a lot to read and tryout.
>
>But here is a first question: The 'test' command runs 'build_ext -i' and
>then the tests.

The "develop" command does this too.


>The workaround I have used for the ctypes project so far is a __path__
>hack: the main module importing this extension tries to execfile() a
>special file that is in CVS, but never distributed.  This file uses
>distutils to determine the name of the build directory, inserts this
>into sys.path.
>
>Is there anything in setuptools to work around that problem, or are
>there any plans or ideas how this could work?

Well, it occurs to me that it would be good for setuptools to track the 
last-built platform and Python version, and use "build_ext -if" if it 
differs from the current platform.  That wouldn't allow simultaneous use of 
different platforms in the same directory, but it would allow serial 
switching between Python versions or platforms (e.g. even cygwin vs. win32) 
to be relatively painless.


From theller at python.net  Thu Aug 18 21:57:50 2005
From: theller at python.net (Thomas Heller)
Date: Thu, 18 Aug 2005 21:57:50 +0200
Subject: [Distutils] setuptools test command on Windows
References: <zmrf6ula.fsf@python.net>
	<5.1.1.6.0.20050818131711.01b29d18@mail.telecommunity.com>
Message-ID: <u0hn2f29.fsf@python.net>

"Phillip J. Eby" <pje at telecommunity.com> writes:

> At 07:08 PM 8/18/2005 +0200, Thomas Heller wrote:
>>Seems I was late to try out setuptools.  Fantastic package, is my first
>>impression.  But I have still a lot to read and tryout.
>>
>>But here is a first question: The 'test' command runs 'build_ext -i' and
>>then the tests.
>
> The "develop" command does this too.
>
>
>>The workaround I have used for the ctypes project so far is a __path__
>>hack: the main module importing this extension tries to execfile() a
>>special file that is in CVS, but never distributed.  This file uses
>>distutils to determine the name of the build directory, inserts this
>>into sys.path.
>>
>>Is there anything in setuptools to work around that problem, or are
>>there any plans or ideas how this could work?
>
> Well, it occurs to me that it would be good for setuptools to track the 
> last-built platform and Python version, and use "build_ext -if" if it 
> differs from the current platform.  That wouldn't allow simultaneous use of 
> different platforms in the same directory, but it would allow serial 
> switching between Python versions or platforms (e.g. even cygwin vs. win32) 
> to be relatively painless.

build_ext -f rebuilds everything, which takes a long time for ctypes, at
least - because it reconfigures and rebuilds libffi also.
Since all the object files are still there, and in separate build
directories, deleting only the extensions would be enough.

Anyway, I was more thinking of a Python loader module in the current
directory, which would determine the platform-specific lib-directory,
and use imp to load the extension from there, and then update it's
globals from the extension.  But I cannot currently figure out how to do
that in a robust way.  FYI, here's what I've got so far.  It will
probably not work for extensions in packages anyway, and if you run a
build_ext -i, the extension module in the current directory will be used
instead of the loader module:

<snip>
# This module, when imported, imports an extension instead
# from the distutils' build directory with the same name.
#
# Must be placed in the same diectory as setup.py.
def __load__(globs):
    import sys, os, imp
    from distutils.util import get_platform

    modname = os.path.splitext(os.path.basename(__file__))[0]
    plat_specifier = ".%s-%s" % (get_platform(), sys.version[0:3])
    build_dir = os.path.join('build', 'lib' + plat_specifier)

    path = [os.path.abspath(os.path.join(os.path.dirname(__file__), build_dir))]

    fil, pathname, descr = imp.find_module(modname, path)
    mod = imp.load_module(modname, fil, pathname, descr)
    print "A"
    globs.update(mod.__dict__)

__load__(globals())
##del __load
<snip>

Thomas


From pje at telecommunity.com  Thu Aug 18 23:14:33 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 17:14:33 -0400
Subject: [Distutils] setuptools test command on Windows
In-Reply-To: <u0hn2f29.fsf@python.net>
References: <zmrf6ula.fsf@python.net>
	<5.1.1.6.0.20050818131711.01b29d18@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050818164511.01b40d88@mail.telecommunity.com>

At 09:57 PM 8/18/2005 +0200, Thomas Heller wrote:
>"Phillip J. Eby" <pje at telecommunity.com> writes:
> > At 07:08 PM 8/18/2005 +0200, Thomas Heller wrote:
> >>Is there anything in setuptools to work around that problem, or are
> >>there any plans or ideas how this could work?
> >
> > Well, it occurs to me that it would be good for setuptools to track the
> > last-built platform and Python version, and use "build_ext -if" if it
> > differs from the current platform.  That wouldn't allow simultaneous 
> use of
> > different platforms in the same directory, but it would allow serial
> > switching between Python versions or platforms (e.g. even cygwin vs. 
> win32)
> > to be relatively painless.
>
>build_ext -f rebuilds everything, which takes a long time for ctypes, at
>least - because it reconfigures and rebuilds libffi also.
>Since all the object files are still there, and in separate build
>directories, deleting only the extensions would be enough.

That might be doable.


>Anyway, I was more thinking of a Python loader module in the current
>directory, which would determine the platform-specific lib-directory,
>and use imp to load the extension from there, and then update it's
>globals from the extension.  But I cannot currently figure out how to do
>that in a robust way.

bdist_egg already creates stubs that do this for loading extensions from 
zip files.  The problem is that it has to generate .py files of the same 
name.  Putting them into the user's source tree seems wrong, however.

Maybe a better solution would be to always build in the normal build 
directories, and then have "test" and "develop" just copy or symlink the 
extensions from the build directories to the source directories.  Thus they 
would just run "build_ext" normally (without -i or -f), followed by some 
copying or linking operations.

I can see that you'd like to support simultaneous use of the same source 
tree from different Pythons, but the only way I can see to make that work 
is to put a stub loader directly.  Your solution isn't viable for "develop" 
however, aside from the fact that it doesn't support extensions in 
packages.  (Packages installed using "develop" don't necessarily have the 
setup directory on their path.)

I do have an idea as to how it could still be accomplished, but it's 
terribly complex and would add a lot of extra code to pkg_resources, and it 
wouldn't allow you to "develop" packages containing extensions without 
importing pkg_resources.  Really, putting the stub loader in the source 
tree is the only I can think of that would work in all cases.  But it's 
also really ugly because it will need to know the build base directory 
(which might not be "build", after all), and because it runs the risk of 
ending up in somebody's sdist, CVS or SVN tree.  There's also the 
complication of needing to overwrite it with a different stub loader during 
bdist_egg.

So, my current inclination is to build_ext in the build directory and then 
copy or symlink the extensions, since it leaves the source and bdist trees 
clean and should work with "develop" out of the box, as long as you rerun 
"develop" whenever you switch Pythons.  (It won't be an issue for the 
"test" command, as long as you don't simultaneously run tests with 
different Pythons.)


From parlar at gmail.com  Fri Aug 19 18:20:10 2005
From: parlar at gmail.com (Jay Parlar)
Date: Fri, 19 Aug 2005 12:20:10 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d05081610487c6de2ed@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
Message-ID: <e15ccc4d0508190920fed354c@mail.gmail.com>

Sorry to relaunch this topic, but I continue to have general problems
with eggs and py2exe.

Is there any way that easy_install can be used to install an egg, but
"un-egg" it, ie. install it into a directory in site-packages, as
would normally be done if just distutils were being used?

I love that RuleDispatch and PyProtocols are now distributed as eggs,
but if I could somehow install them without them being eggs, that'd be
very useful (to me, and some colleagues).

Or is my only option right now to check them out of SVN, modify
setup.py, and install like that?

Jay P.

On 8/16/05, Jay Parlar <parlar at gmail.com> wrote:
> On 8/16/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> > Your eggs.pth should appear in the generated exe directory, right next
> > to your exe file itself, and should have the names of the eggs without
> > any other path on them (just the basename).
> 
> That's exactly what I've got, but it's a no-go. I'll keep trying.
> 
> 
> On 8/16/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> >
> > If all else fails, 'import site; addsitedir(".")' might do the trick.
> 
> That seems to work about as well as sys.path.append("."), but I still
> need to do individual pkg_resources.require("...") calls in my main
> file.
> 
> I guess until py2exe becomes egg aware, it's all going to be nasty hacks anyway.
> 
> Thanks for all the help.
> 
> Jay P.
>

From pje at telecommunity.com  Fri Aug 19 18:32:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 12:32:34 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d0508190920fed354c@mail.gmail.com>
References: <e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>

At 12:20 PM 8/19/2005 -0400, Jay Parlar wrote:
>Sorry to relaunch this topic, but I continue to have general problems
>with eggs and py2exe.

I guess I'll add it to my list to see if I can create a py2exe wrapper for 
setuptools.  Or, since Thomas is now playing with setuptools also, perhaps 
he, Kevin, and I can work something out to make py2exe work as a setuptools 
extension.  Since setuptools provides hooks for such extensions to add 
setup() keywords and new commands, maybe we can get py2exe hooked in such 
that it just adds the required eggs to the zipfile in "basket" mode, and 
doesn't bother tracking imports to packages found in eggs.  (Except maybe 
for pkg_resources, which it should just always include in the zipfile.)


>Is there any way that easy_install can be used to install an egg, but
>"un-egg" it, ie. install it into a directory in site-packages, as
>would normally be done if just distutils were being used?

You can build a bdist_wininst from a source checkout, and then run the 
generated Windows installer.  Or you can use individual commands like 
install_lib, install_data, install_scripts, and so on.  Setuptools doesn't 
modify those commands, so they still have the ability to do non-egg installs.

I suppose I could probably add an option to force the "install" command not 
to install eggs either, and I'll probably have to if/when these features 
get folded back into distutils.


From parlar at gmail.com  Fri Aug 19 18:40:34 2005
From: parlar at gmail.com (Jay Parlar)
Date: Fri, 19 Aug 2005 12:40:34 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
Message-ID: <e15ccc4d0508190940659ffff8@mail.gmail.com>

On 8/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> I guess I'll add it to my list to see if I can create a py2exe wrapper for
> setuptools.  Or, since Thomas is now playing with setuptools also, perhaps
> he, Kevin, and I can work something out to make py2exe work as a setuptools
> extension.  Since setuptools provides hooks for such extensions to add
> setup() keywords and new commands, maybe we can get py2exe hooked in such
> that it just adds the required eggs to the zipfile in "basket" mode, and
> doesn't bother tracking imports to packages found in eggs.  (Except maybe
> for pkg_resources, which it should just always include in the zipfile.)

Well, the issue I just ran into was that the 'symbol' package wasn't 
being brought in by py2exe, because the only place it was imported was
in RuleDispatch (or maybe PyProtocols, can't remember).

So there'd still have to be some kind of import tracking done by
py2exe (or setuptools), because when you're running with py2exe, you
can't expect the entire stdlib to be available.

> You can build a bdist_wininst from a source checkout, and then run the
> generated Windows installer.  Or you can use individual commands like
> install_lib, install_data, install_scripts, and so on.  Setuptools doesn't
> modify those commands, so they still have the ability to do non-egg installs.

Didn't even think of that, thanks for the suggestion.


Thanks,
Jay P.

From pje at telecommunity.com  Fri Aug 19 19:12:51 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 13:12:51 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d0508190940659ffff8@mail.gmail.com>
References: <5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>

At 12:40 PM 8/19/2005 -0400, Jay Parlar wrote:
>On 8/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > I guess I'll add it to my list to see if I can create a py2exe wrapper for
> > setuptools.  Or, since Thomas is now playing with setuptools also, perhaps
> > he, Kevin, and I can work something out to make py2exe work as a setuptools
> > extension.  Since setuptools provides hooks for such extensions to add
> > setup() keywords and new commands, maybe we can get py2exe hooked in such
> > that it just adds the required eggs to the zipfile in "basket" mode, and
> > doesn't bother tracking imports to packages found in eggs.  (Except maybe
> > for pkg_resources, which it should just always include in the zipfile.)
>
>Well, the issue I just ran into was that the 'symbol' package wasn't
>being brought in by py2exe, because the only place it was imported was
>in RuleDispatch (or maybe PyProtocols, can't remember).

Did you have those packages (RuleDispatch & PyProtocols) installed in 
site-packages and easy_install.pth?  If not, they wouldn't be on sys.path, 
so I doubt py2exe would be able to find them.  You might need to add them 
to "setup_requires" to work with py2exe.  (Also, you should be importing 
py2exe *after* importing setuptools, and you should be using the CVS 
version of setuptools.)

If py2exe *still* can't find "symbol", I don't know what to tell you.

Makes me wonder if I should have setuptools create some kind of "stdlib 
imports" metadata file for eggs, so that it wouldn't be necessary to scan 
an egg to find its imports.  But that's a bit further down the road, maybe 
when I'm adding support for "legacy" dependencies.



From ianb at colorstudy.com  Fri Aug 19 19:21:31 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Aug 2005 12:21:31 -0500
Subject: [Distutils] setuptools: namespace packages
Message-ID: <4306151B.2090009@colorstudy.com>

When developing a namespace distribution, where should I put it?  Should 
each such distribution be a separate project in the repository (e.g., 
code goes in Paste/exceptions/trunk/paste/exceptions/)?  Can (or should) 
I put them all together and let setuptools break them up when I 
distribute?

And will I have to move modules out of the namespace package and into 
subpackages?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From parlar at gmail.com  Fri Aug 19 19:50:19 2005
From: parlar at gmail.com (Jay Parlar)
Date: Fri, 19 Aug 2005 13:50:19 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
Message-ID: <e15ccc4d0508191050645b6c63@mail.gmail.com>

> Did you have those packages (RuleDispatch & PyProtocols) installed in
> site-packages and easy_install.pth?  If not, they wouldn't be on sys.path,
> so I doubt py2exe would be able to find them.  You might need to add them
> to "setup_requires" to work with py2exe.  (Also, you should be importing
> py2exe *after* importing setuptools, and you should be using the CVS
> version of setuptools.)
> 
> If py2exe *still* can't find "symbol", I don't know what to tell you.

They were both in site-packages, and in easy_install.pth, they were
installed in the typical easy_install.py manner. But as the origins of
this email thread showed, I was having trouble getting py2exe to see
them even there.



I'll try again with the version currently in CVS, and see if I have
any more luck.

Thanks,
Jay P.



On 8/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:

> 

> 
> 
>

From pje at telecommunity.com  Fri Aug 19 19:54:01 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 13:54:01 -0400
Subject: [Distutils] setuptools: namespace packages
In-Reply-To: <4306151B.2090009@colorstudy.com>
Message-ID: <5.1.1.6.0.20050819133456.02f5f578@mail.telecommunity.com>

At 12:21 PM 8/19/2005 -0500, Ian Bicking wrote:
>When developing a namespace distribution, where should I put it?  Should
>each such distribution be a separate project in the repository (e.g.,
>code goes in Paste/exceptions/trunk/paste/exceptions/)?

If you want to distribute them separately, they need to be separate 
projects with their own setup.py and project name, like "Paste-Exceptions" 
or whatever.

For example, let's say I made "peak.util" a namespace package.  Then, I 
could create separate projects like "PyUUID", "DynamicImports", 
"IndentedStream", and so on, for the different modules in the "peak.util" 
package.  Each one would have to have its own, independently distributable 
project, something like:

     DynamicImports/
         setup.py
         peak/
             __init__.py
             util/
                 __init__.py
                 imports.py

With the setup.py listing 'peak.util' as a namespace package.  The 
__init__.py files will be empty.

The reason you need this layout is that you want "setup.py develop" to 
still work, and for "develop" and "test" to work, they needs the 
__init__.py files.  If you are only using "install" or you always run from 
built eggs, you can forego the __init__.py files and even the directory 
tree, slapping the module directly into the project directory, and 
setuptools will generate stub __init__.py's for you when it builds the 
egg.  (The livinglogic.de libraries like ll-color do this.)

But, if you want to be able to use the "develop" and "test" commands, you 
need a source layout that includes the package tree.

Anyway, the idea behind a namespace package is that you get to claim a 
package name like "paste", and then put all your modules and packages under 
it, without forcing somebody to install a single mega-distribution.  PEAK 
right now has a huge array of functionality under "peak.*" packages, and 
much of it could be used as independent pieces, if only you didn't have to 
install all of PEAK.  Over time (i.e. as soon as I have enough of 
setuptools finished to have some spare time!), I intend to start breaking 
out such pieces as separate distributions, but all the code that still 
imports e.g. "peak.util.imports" will still work, even though it will be in 
a different distribution.

It would be nice, by the way, if somebody who's new to this whole namespace 
package idea would write up an introduction to this for the setuptools 
docs, because the idea is so old hat to me that I doubt I can write a 
really good introduction to it.


>   Can (or should)
>I put them all together and let setuptools break them up when I
>distribute?

Setuptools actually can break them up, using the currently-undocumented 
"feature" facility.  It's undocumented because if you change feature 
options between builds, you may not get what you expect.  But I don't 
really recommend this approach, because then you have to deal with 
--with-X/--without-X options, and then the projects would have to have 
different names, etc.  I went through this thought process when thinking 
about splitting PEAK, as it seemed like it would be easier to leave 
everything together.  But, it's hard to still be compatible with 
EasyInstall that way.  Ultimately I think the "feature" approach is a dead end.



>And will I have to move modules out of the namespace package and into
>subpackages?

No.  A namespace package is a package that's a container.  The container 
can be added to by various distributions; each distribution can add 
modules, resources, or subpackages.  The only thing you shouldn't do is run 
code in the __init__ module, because it will get run as soon as 
pkg_resources is imported and any distribution for the package is on 
sys.path.  For normal site-packages installation, this means that as soon 
as pkg_resources is imported, all declared namespace package __init__.py's 
get run.

If you don't want them to get run right away, you can put this boilerplate 
into all __init__.py files for the package:

     __import__('pkg_resources').declare_namespace(__name__)

and then don't declare the namespace in setup.py.  The problem with this 
approach, however, is that you need to make absolutely certain you do it in 
the mynamespacepackage/__init__.py file of *each* distribution.  If you 
miss even one, and that missed one gets imported first, then the others 
will not get added to the namespace.  This is why it's preferable to use 
the documented approach of listing namespace packages in setup(), and 
always using empty __init__.py files for namespace packages.


From ianb at colorstudy.com  Fri Aug 19 20:16:34 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Aug 2005 13:16:34 -0500
Subject: [Distutils] setuptools: namespace packages
In-Reply-To: <5.1.1.6.0.20050819133456.02f5f578@mail.telecommunity.com>
References: <5.1.1.6.0.20050819133456.02f5f578@mail.telecommunity.com>
Message-ID: <43062202.6090203@colorstudy.com>

Phillip J. Eby wrote:
> At 12:21 PM 8/19/2005 -0500, Ian Bicking wrote:
> 
>> When developing a namespace distribution, where should I put it?  Should
>> each such distribution be a separate project in the repository (e.g.,
>> code goes in Paste/exceptions/trunk/paste/exceptions/)?
> 
> 
> If you want to distribute them separately, they need to be separate 
> projects with their own setup.py and project name, like 
> "Paste-Exceptions" or whatever.

If I have a package already, and I want to add a new subpackage, as a 
seperate distribution, what would that entail?  E.g., I have the paste 
package as it is, and I want to add paste.deploy as a separate 
distribution, but without splitting up anything else.  Do I have to 
modify the original Paste setup?


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Fri Aug 19 20:24:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 14:24:13 -0400
Subject: [Distutils] setuptools: namespace packages
In-Reply-To: <43062202.6090203@colorstudy.com>
References: <5.1.1.6.0.20050819133456.02f5f578@mail.telecommunity.com>
	<5.1.1.6.0.20050819133456.02f5f578@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050819142257.02873918@mail.telecommunity.com>

At 01:16 PM 8/19/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 12:21 PM 8/19/2005 -0500, Ian Bicking wrote:
>>
>>>When developing a namespace distribution, where should I put it?  Should
>>>each such distribution be a separate project in the repository (e.g.,
>>>code goes in Paste/exceptions/trunk/paste/exceptions/)?
>>
>>If you want to distribute them separately, they need to be separate 
>>projects with their own setup.py and project name, like 
>>"Paste-Exceptions" or whatever.
>
>If I have a package already, and I want to add a new subpackage, as a 
>seperate distribution, what would that entail?  E.g., I have the paste 
>package as it is, and I want to add paste.deploy as a separate 
>distribution, but without splitting up anything else.  Do I have to modify 
>the original Paste setup?

No, as long as the original project's paste.__init__ doesn't contain any 
code, and any new projects declare the namespace package in their setup().


From dangoor at gmail.com  Fri Aug 19 21:01:35 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Fri, 19 Aug 2005 15:01:35 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d0508191050645b6c63@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
	<e15ccc4d0508191050645b6c63@mail.gmail.com>
Message-ID: <3f085ecd0508191201384a19fe@mail.gmail.com>

On 8/19/05, Jay Parlar <parlar at gmail.com> wrote:
> They were both in site-packages, and in easy_install.pth, they were
> installed in the typical easy_install.py manner. But as the origins of
> this email thread showed, I was having trouble getting py2exe to see
> them even there.

Did you try installing them with the -Z (--always-unzip) option? Just
curious if that makes any difference to how py2exe works its
dependency magic.

I do remember encountering some of these stdlib problems myself, but
not since early on when I started playing with setuptools. I only have
a handful of the packages I use turned into eggs at this point,
though.

Kevin

From pje at telecommunity.com  Fri Aug 19 21:33:16 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 15:33:16 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <3f085ecd0508191201384a19fe@mail.gmail.com>
References: <e15ccc4d0508191050645b6c63@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
	<e15ccc4d0508191050645b6c63@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050819152205.01b29a58@mail.telecommunity.com>

At 03:01 PM 8/19/2005 -0400, Kevin Dangoor wrote:
>On 8/19/05, Jay Parlar <parlar at gmail.com> wrote:
> > They were both in site-packages, and in easy_install.pth, they were
> > installed in the typical easy_install.py manner. But as the origins of
> > this email thread showed, I was having trouble getting py2exe to see
> > them even there.
>
>Did you try installing them with the -Z (--always-unzip) option? Just
>curious if that makes any difference to how py2exe works its
>dependency magic.
>
>I do remember encountering some of these stdlib problems myself, but
>not since early on when I started playing with setuptools. I only have
>a handful of the packages I use turned into eggs at this point,
>though.

Yeah, I'm starting to think that the ultimate solution to this is going to 
be to build Bob Ippolito's modulegraph into setuptools, and put external 
import info into the egg metadata at build time.  Then tools like py2exe 
and py2app could just read the metadata for all the external packages 
you're using.  bdist_egg already has to scan all your modules if you don't 
specify a zip_safe flag, so having it build a dependencies list too 
probably isn't a big deal.

For that matter, maybe I should add a "basket" command that builds a nice 
library.zip with pkg_resources, the application, and all its eggs.  Then 
commands like py2exe and py2app could just focus on making the front-end 
wrappers, rather than on the grunt work of figuring out what modules the 
application needs.

I'm not sure how well that would work with all of e.g. py2exe's 
special-casing of various DLLs and whatnot, though.  Nonetheless, I'd like 
to hear Bob and Thomas' thoughts on the idea, as it would let us 
consolidate some of the infrastructure and maybe make cross-platform apps 
buildable from a single setup script.  (Especially if most of the 
py2exe-specific or py2app-specific options can be moved to setup.cfg, where 
they'll only be processed if running that particular command.)


From theller at python.net  Fri Aug 19 21:40:34 2005
From: theller at python.net (Thomas Heller)
Date: Fri, 19 Aug 2005 21:40:34 +0200
Subject: [Distutils] py2exe and eggs
References: <e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<3f085ecd0508160805e4500d3@mail.gmail.com>
	<e15ccc4d0508160923273ff4a1@mail.gmail.com>
	<3f085ecd05081609364082e9f7@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
Message-ID: <zmrdhg0d.fsf@python.net>

"Phillip J. Eby" <pje at telecommunity.com> writes:

> At 12:20 PM 8/19/2005 -0400, Jay Parlar wrote:
>>Sorry to relaunch this topic, but I continue to have general problems
>>with eggs and py2exe.
>
> I guess I'll add it to my list to see if I can create a py2exe wrapper for 
> setuptools.  Or, since Thomas is now playing with setuptools also, perhaps 
> he, Kevin, and I can work something out to make py2exe work as a setuptools 
> extension.  Since setuptools provides hooks for such extensions to add 
> setup() keywords and new commands, maybe we can get py2exe hooked in such 
> that it just adds the required eggs to the zipfile in "basket" mode, and 
> doesn't bother tracking imports to packages found in eggs.  (Except maybe 
> for pkg_resources, which it should just always include in the zipfile.)

Not sure if I can find the time for this - and I'm currently not using
eggs myself.

I would hope that py2exe works together with directory eggs (is that the
name?) but not zip-eggs.

py2exe uses modulefinder to find dependencies.  Modulefinder scans the
bytecode of scripts and modules for import statements, then uses the imp
module to locate the dependencies. Since imp does not find modules and
packages in zip.files, all hope is off for zip-eggs.

I think a workaround could be to let py2exe run do its work normally,
and then add the zipped egg files (if you need them) to the dist folder
manually or per setup script (as data-files, maybe).  Before that you
should in the main script add the eggs filenames to sys.path of the
frozen exeutable, maybe that should work.

Still, you have to track the dependencies of the eggs themselves
manually, and pass them to py2exe.

Thomas


From parlar at gmail.com  Fri Aug 19 21:52:37 2005
From: parlar at gmail.com (Jay Parlar)
Date: Fri, 19 Aug 2005 15:52:37 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <3f085ecd0508191201384a19fe@mail.gmail.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
	<e15ccc4d0508191050645b6c63@mail.gmail.com>
	<3f085ecd0508191201384a19fe@mail.gmail.com>
Message-ID: <e15ccc4d050819125259459673@mail.gmail.com>

On 8/19/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> Did you try installing them with the -Z (--always-unzip) option? Just
> curious if that makes any difference to how py2exe works its
> dependency magic.


Look at that, seems to work like a charm. I think that will be my new
solution until something real is in place.


Jay P.

From pje at telecommunity.com  Fri Aug 19 22:14:49 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 16:14:49 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <e15ccc4d050819125259459673@mail.gmail.com>
References: <3f085ecd0508191201384a19fe@mail.gmail.com>
	<e15ccc4d05081606283b4934c2@mail.gmail.com>
	<e15ccc4d050816101045f5f109@mail.gmail.com>
	<5.1.1.6.0.20050816132443.026eeab8@mail.telecommunity.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
	<e15ccc4d0508191050645b6c63@mail.gmail.com>
	<3f085ecd0508191201384a19fe@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050819160913.02c7ebc8@mail.telecommunity.com>

At 03:52 PM 8/19/2005 -0400, Jay Parlar wrote:
>On 8/19/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> > Did you try installing them with the -Z (--always-unzip) option? Just
> > curious if that makes any difference to how py2exe works its
> > dependency magic.
>
>
>Look at that, seems to work like a charm. I think that will be my new
>solution until something real is in place.

Okay, so are you doing anything else special?  Or are you able to use an 
otherwise-unchanged py2exe setup script if you install its dependencies 
using ---always-unzip?

If so, I'd suggest doing this in the setup.cfg of projects that will use 
py2exe:

     [easy_install]
     zip_ok = 0

This will make EasyInstall unzip any dependencies it builds as part of the 
"install" command; i.e., you'll need to run "setup.py install py2exe" in 
order to download the dependencies, install them to site-packages, and then 
run py2exe.  However, it will mysteriously break if a dependency was 
already installed as a zip.  :(

Kind of kludgy, but I guess it at least gets you going.


From parlar at gmail.com  Fri Aug 19 22:49:12 2005
From: parlar at gmail.com (Jay Parlar)
Date: Fri, 19 Aug 2005 16:49:12 -0400
Subject: [Distutils] py2exe and eggs
In-Reply-To: <5.1.1.6.0.20050819160913.02c7ebc8@mail.telecommunity.com>
References: <e15ccc4d05081606283b4934c2@mail.gmail.com>
	<e15ccc4d05081610487c6de2ed@mail.gmail.com>
	<e15ccc4d0508190920fed354c@mail.gmail.com>
	<5.1.1.6.0.20050819122354.031147f8@mail.telecommunity.com>
	<e15ccc4d0508190940659ffff8@mail.gmail.com>
	<5.1.1.6.0.20050819130837.01b2ac68@mail.telecommunity.com>
	<e15ccc4d0508191050645b6c63@mail.gmail.com>
	<3f085ecd0508191201384a19fe@mail.gmail.com>
	<e15ccc4d050819125259459673@mail.gmail.com>
	<5.1.1.6.0.20050819160913.02c7ebc8@mail.telecommunity.com>
Message-ID: <e15ccc4d05081913492af5040e@mail.gmail.com>

On 8/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> Okay, so are you doing anything else special?  Or are you able to use an
> otherwise-unchanged py2exe setup script if you install its dependencies
> using ---always-unzip?

I'm not doing anything special, it's my stock py2exe setup.py, and it
seems to be bringing in everything I want. You can even see the py2exe
output of byte-compiling the files inside the egg directories.


> If so, I'd suggest doing this in the setup.cfg of projects that will use
> py2exe:
> 
>      [easy_install]
>      zip_ok = 0
> 
> This will make EasyInstall unzip any dependencies it builds as part of the
> "install" command; i.e., you'll need to run "setup.py install py2exe" in
> order to download the dependencies, install them to site-packages, and then
> run py2exe.  However, it will mysteriously break if a dependency was
> already installed as a zip.  :(
> 
> Kind of kludgy, but I guess it at least gets you going.

Maybe I misunderstnd what you mean hear, but that didn't seem to do
much for me. If I have a project that I'm not putting anything special
into the setup.py, how would EasyInstall get brought in to do
anything?

I think that since I know that unzipped Eggs will work, I'll just make
sure I install them unzipped. Right now the only packages I have as
eggs are PyProtocols and RuleDispatch (at least, the only packages
that need to be part of my internal application), so that's the only
place py2exe will affect me.


Jay P.

From ianb at colorstudy.com  Sat Aug 20 01:00:05 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Aug 2005 18:00:05 -0500
Subject: [Distutils] testing with eggs
Message-ID: <43066475.4010401@colorstudy.com>

Just thought I'd pass this along, since it took me a little while to 
figure it out.  Maybe this can go in docs somewhere, but I'm not sure where.

So, I'm writing some code that uses egg plugins, and thus I need testing 
eggs.  These need to be path-independent (since the checkouts might live 
anywhere), with no setup commands (you shouldn't have to install the 
testing version of the eggs to run the tests), and the eggs should be 
available only when you are running the tests (no global installation).

I'm using py.test, so I add a conftest.py file which is loaded before 
any tests are imported.  It's important all this is done before 
pkg_resources is imported (maybe there are methods in pkg_resources that 
can fix things after it was imported, but pkg_resources uses sys.path 
when it is imported, so if you adjust the path later then pkg_resources 
won't notice it).

Anyway, here's the code I use:


import os
import sys
import glob

here = os.path.dirname(__file__)
base = os.path.dirname(here)
fake_packages = os.path.join(here, 'fake_packages')

for egg_info_dir in glob.glob('%s/*/*.egg-info' % fake_packages):
     sys.path.append(os.path.dirname(egg_info_dir))


At first I tried adding fake_packages to sys.path; didn't work at all. 
But if I do site.addsitedir(fake_packages) then it would work.  But this 
requires an .egg-link file in fake_packages, and that file has to have 
an absolute path (it can't be relative), but fake_packages could be 
anywhere.  So in the end, I just need to add all the necessary paths; 
this means I can't test the case when --multi-version is used to install 
an egg, but I guess I won't worry about that.

If you are curious about the base setup, I've checked in a minimal file 
layout of the whole thing into 
http://svn.pythonpaste.org/Paste/Deploy/trunk/

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Sat Aug 20 01:45:14 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 19 Aug 2005 19:45:14 -0400
Subject: [Distutils] testing with eggs
In-Reply-To: <43066475.4010401@colorstudy.com>
Message-ID: <5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>

At 06:00 PM 8/19/2005 -0500, Ian Bicking wrote:
>Just thought I'd pass this along, since it took me a little while to
>figure it out.  Maybe this can go in docs somewhere, but I'm not sure where.
>
>So, I'm writing some code that uses egg plugins, and thus I need testing
>eggs.  These need to be path-independent (since the checkouts might live
>anywhere), with no setup commands (you shouldn't have to install the
>testing version of the eggs to run the tests), and the eggs should be
>available only when you are running the tests (no global installation).

I'm kind of confused.  Maybe what you want is to create dummy Distribution 
objects and add them to a WorkingSet, then use the WorkingSet's APIs in 
your tests (such as iter_entry_points, require(), etc.).  It's not entirely 
clear to me what it is that you're testing, so I'm not sure.

In setuptools.tests.test_resources there's a simple Metadata class that's 
used to create dummy distributions that nonetheless have metadata; you 
could use something like that to create a dummy Distribution.

A WorkingSet is a collection of active Distributions, and it has methods 
like require(), iter_entry_points(), and so on.  There is a default global 
WorkingSet that's used for pkg_resources API, but you can create your own 
instances for testing purposes, and manipulating it won't affect 
sys.path.  The default working set has a listener subscribed to it that 
activates distributions on sys.path, but if you just create a WorkingSet 
for yourself it won't have this link.  The "api_tests.txt" file 
demonstrates some WorkingSet APIs, and "pkg_resources.txt" has complete 
reference docs for its methods.


>At first I tried adding fake_packages to sys.path; didn't work at all.

Because the subdirectories' names don't end in .egg; if they did and you 
just used EGG-INFO instead of .egg-info, it would work.


>But if I do site.addsitedir(fake_packages) then it would work.  But this
>requires an .egg-link file in fake_packages, and that file has to have
>an absolute path (it can't be relative), but fake_packages could be
>anywhere.  So in the end, I just need to add all the necessary paths;
>this means I can't test the case when --multi-version is used to install
>an egg, but I guess I won't worry about that.
>
>If you are curious about the base setup, I've checked in a minimal file
>layout of the whole thing into
>http://svn.pythonpaste.org/Paste/Deploy/trunk/

 From what little I can tell about what you're trying to do, I think you 
should just create dummy Distribution objects and add them a test-only 
WorkingSet, and not bother having anything in the filesystem, unless you're 
testing code that also does stuff with the filesystem.  In which case, you 
could just use pkg_resources' code to find and introspect packages, though, 
so as I said, I'm not sure what it is exactly you're testing.  :)


From ianb at colorstudy.com  Sat Aug 20 05:04:16 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Aug 2005 22:04:16 -0500
Subject: [Distutils] testing with eggs
In-Reply-To: <5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>
References: <5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>
Message-ID: <43069DB0.3090400@colorstudy.com>

Phillip J. Eby wrote:
> At 06:00 PM 8/19/2005 -0500, Ian Bicking wrote:
> 
>> Just thought I'd pass this along, since it took me a little while to
>> figure it out.  Maybe this can go in docs somewhere, but I'm not sure 
>> where.
>>
>> So, I'm writing some code that uses egg plugins, and thus I need testing
>> eggs.  These need to be path-independent (since the checkouts might live
>> anywhere), with no setup commands (you shouldn't have to install the
>> testing version of the eggs to run the tests), and the eggs should be
>> available only when you are running the tests (no global installation).
> 
> 
> I'm kind of confused.  Maybe what you want is to create dummy 
> Distribution objects and add them to a WorkingSet, then use the 
> WorkingSet's APIs in your tests (such as iter_entry_points, require(), 
> etc.).  It's not entirely clear to me what it is that you're testing, so 
> I'm not sure.

I'm testing loading WSGI apps out of eggs using entry points.  So there 
will be real code in those test packages, though just stubby code.  I'm 
not testing setuptools per se, so dummy Distributions probably won't be 
of much use.

>> At first I tried adding fake_packages to sys.path; didn't work at all.
> 
> 
> Because the subdirectories' names don't end in .egg; if they did and you 
> just used EGG-INFO instead of .egg-info, it would work.

OK, I'll try that.  I was surprised, though, that FakeApp.egg-link 
didn't work until I added the directory as addsitedir.  Though I was 
flailing about to get it working, so I don't know what exactly 
combination of things I ended up using.

I also realize it would be nice to refresh the egg_info before running 
the tests, but I'm not sure how well that will work if it happens in the 
same process as the tests themselves.  Or how I run a setup() command 
from Python.


-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From ianb at colorstudy.com  Sat Aug 20 05:44:57 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Aug 2005 22:44:57 -0500
Subject: [Distutils] testing with eggs
In-Reply-To: <43069DB0.3090400@colorstudy.com>
References: <5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>
	<43069DB0.3090400@colorstudy.com>
Message-ID: <4306A739.5040305@colorstudy.com>

Ian Bicking wrote:
>>>At first I tried adding fake_packages to sys.path; didn't work at all.
>>
>>
>>Because the subdirectories' names don't end in .egg; if they did and you 
>>just used EGG-INFO instead of .egg-info, it would work.
> 
> 
> OK, I'll try that. 

That worked; as long as fake_packages is in the path (before I import 
pkg_resources), and if I rename FakeApp to FakeApp.egg, and symlink 
EGG-INFO to FakeApp.egg-info, then it works.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Sat Aug 20 06:37:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 20 Aug 2005 00:37:38 -0400
Subject: [Distutils] testing with eggs
In-Reply-To: <43069DB0.3090400@colorstudy.com>
References: <5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>
	<5.1.1.6.0.20050819193505.01b2ba18@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050819233456.01b2dcb8@mail.telecommunity.com>

At 10:04 PM 8/19/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>I'm kind of confused.  Maybe what you want is to create dummy 
>>Distribution objects and add them to a WorkingSet, then use the 
>>WorkingSet's APIs in your tests (such as iter_entry_points, require(), 
>>etc.).  It's not entirely clear to me what it is that you're testing, so 
>>I'm not sure.
>
>I'm testing loading WSGI apps out of eggs using entry points.  So there 
>will be real code in those test packages, though just stubby code.  I'm 
>not testing setuptools per se, so dummy Distributions probably won't be of 
>much use.

Note that entry points don't have to point into the egg; they just get 
imported.  Thus, you might as well point the entry points back into your 
test code, with no need to create a separate package.  All you really need 
are Distribution objects with dummy "entry_points.txt" metadata resources.


>>>At first I tried adding fake_packages to sys.path; didn't work at all.
>>
>>Because the subdirectories' names don't end in .egg; if they did and you 
>>just used EGG-INFO instead of .egg-info, it would work.
>
>OK, I'll try that.  I was surprised, though, that FakeApp.egg-link didn't 
>work until I added the directory as addsitedir.  Though I was flailing 
>about to get it working, so I don't know what exactly combination of 
>things I ended up using.
>
>I also realize it would be nice to refresh the egg_info before running the 
>tests, but I'm not sure how well that will work if it happens in the same 
>process as the tests themselves.  Or how I run a setup() command from Python.

That's why I think you'll have an easier time just creating dummy 
Distributions with entry point metadata that links back into your test 
code; then you don't have to even have real packages inside the eggs, nor 
do you have to manipulate sys.path any.  Since you're not testing 
setuptools, that should work fine for your purposes, if I understand them 
correctly.  All you really want to test is that your code makes the right 
calls on a working set and invokes your stub entry points.

Just make your code take a WorkingSet as a parameter, and default to 
pkg_resources.working_set (the global WorkingSet).  Your test code will 
pass in its own WorkingSet instances, which will not affect sys.path or the 
global pkg_resources state.  But your code will be able to call require(), 
resolve(), iter_entry_points(), and all the other standard APIs, because 
these are WorkingSet methods anyway.  The global working_set's methods are 
exported by pkg_resources as module-level APIs, but code like yours should 
operate on a WorkingSet instance and just default to the global one.


From webmaster at smallco.com  Sun Aug 21 19:25:24 2005
From: webmaster at smallco.com (Webmaster)
Date: Sun, 21 Aug 2005 13:25:24 -0400 (EDT)
Subject: [Distutils] Automatic response to your mail (Error)
Message-ID: <m2107/13/04M24.BBDAB26BEE@onyx.bcentralhost.com>

The automatic reply to this e-mail which you should have
received in response to your e-mail to webmaster at smallco.com has not been defined.

Please contact postmaster at smallco.com for assistance.

From pje at telecommunity.com  Mon Aug 22 06:03:10 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 00:03:10 -0400
Subject: [Distutils] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
Message-ID: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>

This thing's been in limbo too long, so I've made a prerelease 
snapshot.  You can get it with:

     easy_install -f http://peak.telecommunity.com/snapshots/ setuptools

I have fixed all reported bugs, and implemented quite a few new features 
like MD5 checking (which works even with PyPI's md5 data) and an enhanced 
"build_ext --inplace" that lets you switch between different Python 
interpreters without having to recompile your extensions.  (The extensions 
are built in build/, then copied to the source tree whenever you 
"build_ext" using a different interpreter than the last.)

If you have any projects using setuptools, I encourage you to give this 
snapshot a try, so I can find out if there are still any lurking bugs.  If 
there are no bug reports by next weekend, this will get pushed out as 0.6a1.

If you want to look at the documentation, use CVS or browse the 
EasyInstall.txt, setuptools.txt, and pkg_resources.txt files found at:

    http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/

(I just made some changes to these, however, that may take a few hours to 
update in SourceForge's anonymous CVS.)


From richardjones at optushome.com.au  Mon Aug 22 06:30:09 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 22 Aug 2005 14:30:09 +1000
Subject: [Distutils] [Catalog-sig] setuptools/EasyInstall 0.6a1
	candidate 1 snapshot
In-Reply-To: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <200508221430.10167.richardjones@optushome.com.au>

On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> I have fixed all reported bugs, and implemented quite a few new features
> like MD5 checking (which works even with PyPI's md5 data)

Glad you could get things going in spite of PyPI :)

I'm going to be busy with PyWeek for the next few weeks. Once that's out of 
the way, I can do some PyPI stuff if need be (of course, anyone else is free 
to work on PyPI too... :)


   Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050822/bd2ba436/attachment.pgp

From cmlenz at gmx.de  Mon Aug 22 15:09:18 2005
From: cmlenz at gmx.de (Christopher Lenz)
Date: Mon, 22 Aug 2005 15:09:18 +0200
Subject: [Distutils] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <A4E8C239-2D22-4DE6-A984-C53B5C595B9E@gmx.de>

Hi Phillip,

Am 22.08.2005 um 06:03 schrieb Phillip J. Eby:
> If you have any projects using setuptools, I encourage you to give  
> this
> snapshot a try, so I can find out if there are still any lurking  
> bugs.  If
> there are no bug reports by next weekend, this will get pushed out  
> as 0.6a1.

I've upgraded to 0.6a1 on one machine, and am now getting the  
following error for a fresh checkout of my project [1]:

   $ ./setup.py build
   Traceback (most recent call last):
     File "./setup.py", line 23, in ?
       ez_setup.use_setuptools()
     File "/usr/local/src/bitten/trunk/ez_setup/__init__.py", line  
72, in use_setuptools
       pkg_resources.require("setuptools>="+version)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 503, in require
       needed = self.resolve(parse_requirements(requirements))
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 479, in resolve
       env = Environment(self.entries)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 558, in __init__
       self.scan(search_path)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 588, in scan
       self.add(dist)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 610, in add
       if dist not in dists:
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 1760, in __cmp__
       def __cmp__(self, other): return cmp(self.hashcmp, other)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 1760, in __cmp__
       def __cmp__(self, other): return cmp(self.hashcmp, other)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 1755, in <lambda>
       lambda self: (
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 1782, in parsed_version
       self._parsed_version = pv = parse_version(self.version)
     File "/usr/lib/python2.3/site-packages/setuptools-0.6a1- 
py2.3.egg/pkg_resources.py", line 1797, in version
       raise ValueError(
   ValueError: ("Missing 'Version:' header and/or PKG-INFO file",  
bitten [unknown version] (/usr/local/src/bitten/trunk))

Previously I've been using 0.5a13. The project includes ez_setup as  
svn:external.
Is this a problem in (or incompatibility with) ez_setup?

Thanks,
Chris

  [1] http://bitten.cmlenz.net/
--
Christopher Lenz
   cmlenz at gmx.de
   http://www.cmlenz.net/


From pje at telecommunity.com  Mon Aug 22 15:55:29 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 09:55:29 -0400
Subject: [Distutils] setuptools/EasyInstall 0.6a1 candidate 1  snapshot
In-Reply-To: <A4E8C239-2D22-4DE6-A984-C53B5C595B9E@gmx.de>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822095148.01b0f750@mail.telecommunity.com>

At 03:09 PM 8/22/2005 +0200, Christopher Lenz wrote:
>I've upgraded to 0.6a1 on one machine, and am now getting the
>following error for a fresh checkout of my project [1]:
>...
>       raise ValueError(
>   ValueError: ("Missing 'Version:' header and/or PKG-INFO file",
>bitten [unknown version] (/usr/local/src/bitten/trunk))

Is there a /usr/local/src/bitten/trunk/bitten.egg-info/PKG-INFO file, with 
a 'Version:' header in it?  If not, then the error message is correct, and 
you should go to /usr/local/src/bitten/trunk and run "setup.py egg_info" to 
rebuild the project's egg-info.


>Previously I've been using 0.5a13. The project includes ez_setup as
>svn:external.
>Is this a problem in (or incompatibility with) ez_setup?

No; it just looks like a straightforward error to me.  Older versions of 
pkg_resources (like the 0.5 versions) sometimes masked this error, though, 
as it used to be an AttributeError instead of a ValueError, and it could 
thus get suppressed in certain situations.


From cmlenz at gmx.de  Mon Aug 22 16:30:36 2005
From: cmlenz at gmx.de (Christopher Lenz)
Date: Mon, 22 Aug 2005 16:30:36 +0200
Subject: [Distutils] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <5.1.1.6.0.20050822095148.01b0f750@mail.telecommunity.com>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050822095148.01b0f750@mail.telecommunity.com>
Message-ID: <607D48C5-8577-41F9-8276-EC661FE53D13@gmx.de>

Am 22.08.2005 um 15:55 schrieb Phillip J. Eby:
> At 03:09 PM 8/22/2005 +0200, Christopher Lenz wrote:
>
>> I've upgraded to 0.6a1 on one machine, and am now getting the
>> following error for a fresh checkout of my project [1]:
>> ...
>>       raise ValueError(
>>   ValueError: ("Missing 'Version:' header and/or PKG-INFO file",
>> bitten [unknown version] (/usr/local/src/bitten/trunk))
>>
>
> Is there a /usr/local/src/bitten/trunk/bitten.egg-info/PKG-INFO  
> file, with a 'Version:' header in it?  If not, then the error  
> message is correct, and you should go to /usr/local/src/bitten/ 
> trunk and run "setup.py egg_info" to rebuild the project's egg-info.

Running "setup.py egg_info" results in exactly the error I posted. A  
chicken and egg problem, I suppose. ;-)

Note that the error is raised from the "ez_setup.use_setuptools()"  
call in setup.py. If I just import setuptools without going through  
ez_setup, everything works fine. Am I missing something?

Thanks,
Chris
--
Christopher Lenz
   cmlenz at gmx.de
   http://www.cmlenz.net/


From pje at telecommunity.com  Mon Aug 22 18:17:08 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 12:17:08 -0400
Subject: [Distutils] setuptools/EasyInstall 0.6a1 candidate 1  snapshot
In-Reply-To: <607D48C5-8577-41F9-8276-EC661FE53D13@gmx.de>
References: <5.1.1.6.0.20050822095148.01b0f750@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050822095148.01b0f750@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822121457.01b214b0@mail.telecommunity.com>

At 04:30 PM 8/22/2005 +0200, Christopher Lenz wrote:
>Am 22.08.2005 um 15:55 schrieb Phillip J. Eby:
>>At 03:09 PM 8/22/2005 +0200, Christopher Lenz wrote:
>>
>>>I've upgraded to 0.6a1 on one machine, and am now getting the
>>>following error for a fresh checkout of my project [1]:
>>>...
>>>       raise ValueError(
>>>   ValueError: ("Missing 'Version:' header and/or PKG-INFO file",
>>>bitten [unknown version] (/usr/local/src/bitten/trunk))
>>
>>Is there a /usr/local/src/bitten/trunk/bitten.egg-info/PKG-INFO
>>file, with a 'Version:' header in it?  If not, then the error
>>message is correct, and you should go to /usr/local/src/bitten/ trunk and 
>>run "setup.py egg_info" to rebuild the project's egg-info.
>
>Running "setup.py egg_info" results in exactly the error I posted. A
>chicken and egg problem, I suppose. ;-)
>
>Note that the error is raised from the "ez_setup.use_setuptools()"
>call in setup.py. If I just import setuptools without going through
>ez_setup, everything works fine. Am I missing something?

No, it probably just means that I need to change the above code to issue a 
warning instead of an error and treat the version as '' in that case.  When 
it was trapped as an AttributeError, there was no way to find the problem, 
but this makes it clear that *not* trapping the error is also a 
problem.  So I guess I'll have to change it to issue a warning instead.


From ianb at colorstudy.com  Mon Aug 22 20:26:27 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 22 Aug 2005 13:26:27 -0500
Subject: [Distutils] setuptools or PyPI problem...?
Message-ID: <430A18D3.5040700@colorstudy.com>

I don't know which side this belongs to, but I had a problem when I 
tried to create a package with a "-" in it ("Paste-Deploy").  setup.py 
register worked fine, and created a "Paste-Deploy" project; however, 
when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and 
PyPI wouldn't accept it, I believe because it thought it belonged to the 
(nonexistant) Paste_Deploy project.  In the end I just smushed the words 
together, but I figure this really should work.  I don't know if it's 
setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI 
that is mismatching projects.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Mon Aug 22 20:29:21 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 22 Aug 2005 13:29:21 -0500
Subject: [Distutils] setuptools: Plugins
Message-ID: <430A1981.5020706@colorstudy.com>

I'm looking for some advice on using setuptools/eggs/entry_points for 
plugins.  Specifically, how do I find distributions tht provide the 
entry points I'm interested in?  For instance, can I find all the 
distributions that provide "paste.script_command" entry points?  Or does 
there need to be some way for these distributions to tell my package 
they exist?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 22 20:43:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 14:43:44 -0400
Subject: [Distutils] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
Message-ID: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>

At 01:26 PM 8/22/2005 -0500, Ian Bicking wrote:
>I don't know which side this belongs to, but I had a problem when I
>tried to create a package with a "-" in it ("Paste-Deploy").  setup.py
>register worked fine, and created a "Paste-Deploy" project; however,
>when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and
>PyPI wouldn't accept it, I believe because it thought it belonged to the
>(nonexistant) Paste_Deploy project.  In the end I just smushed the words
>together, but I figure this really should work.  I don't know if it's
>setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI
>that is mismatching projects.

Try the latest setuptools; you shouldn't have this problem as I changed it 
not to create anything other than eggs with escaped '-' characters.

However, it would be very nice to have PyPI support escaped 
nonalphanumerics in filenames, as they are a bitch to deal with 
otherwise.  Eggs absolutely have to have an unambiguously parseable 
filename, and the only way to do that is by escaping '-' to '_'.  This 
means that you can't upload eggs for a project named e.g. 'Paste-Deploy', 
if PyPI rejects a Paste_Deploy-whatever.egg file.

On the broader scope of things, I'd like to see PyPI smash all 
non-alphanumeric runs in project names to a single '-', and use 
case-insensitive project name comparison.  I'd attempt to try my hand at 
PyPI patches but at the moment don't have any obvious way to test them.


From pje at telecommunity.com  Mon Aug 22 20:48:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 14:48:53 -0400
Subject: [Distutils] setuptools: Plugins
In-Reply-To: <430A1981.5020706@colorstudy.com>
Message-ID: <5.1.1.6.0.20050822144347.01b0cc60@mail.telecommunity.com>

At 01:29 PM 8/22/2005 -0500, Ian Bicking wrote:
>I'm looking for some advice on using setuptools/eggs/entry_points for
>plugins.  Specifically, how do I find distributions tht provide the
>entry points I'm interested in?  For instance, can I find all the
>distributions that provide "paste.script_command" entry points?  Or does
>there need to be some way for these distributions to tell my package
>they exist?

def distributions_with_commands(ws=pkg_resources.working_set):
     for dist in ws:
         if dist.get_entry_map('paste.script_command'):
             yield dist


Distribution.get_entry_map() returns a dictionary mapping entry point names 
to EntryPoint objects, so this code yields the distributions on sys.path 
(by default) that contain any entry points in the 'paste.script_command' group.

Complete documentation on the entry point API can be found at:

http://peak.telecommunity.com/DevCenter/PkgResources#entry-points


From martin at v.loewis.de  Tue Aug 23 00:01:00 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 23 Aug 2005 00:01:00 +0200
Subject: [Distutils] [Catalog-sig] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
References: <430A18D3.5040700@colorstudy.com>
Message-ID: <430A4B1C.8020900@v.loewis.de>

Ian Bicking wrote:
> I don't know which side this belongs to, but I had a problem when I 
> tried to create a package with a "-" in it ("Paste-Deploy").  setup.py 
> register worked fine, and created a "Paste-Deploy" project; however, 
> when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and 
> PyPI wouldn't accept it, I believe because it thought it belonged to the 
> (nonexistant) Paste_Deploy project.  In the end I just smushed the words 
> together, but I figure this really should work.  I don't know if it's 
> setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI 
> that is mismatching projects.

Sounds like a PyPI problem to me.

Regards,
Martin

From Smayham at manh.com  Wed Aug 24 00:14:24 2005
From: Smayham at manh.com (Scott Mayham)
Date: Tue, 23 Aug 2005 18:14:24 -0400
Subject: [Distutils] Opening question
Message-ID: <CC5B93677E8E5347B0E156A7D830ED32466D67@Ma-atl58.us.manh.com>


Well, I just now joined this SIG.  I'm an abject Python newbie.  My role is that of Installation developer for various Windows products produced by my company.  Therein lies my need for help.

We are rushing out the door, a new version of one of our main products.  Part of what's new about this release is that some parts of it are written in Python (2.4, I think).  These parts also use some bits for the PyWin package.  I have been chartered with exploring whether we can bundle the Python Windows installer (the MSI that the link on the "downloads" page points to) along with our product, and whether it (the Python installer) can be launched as a silent sub-install, in much the same way as we presently launch the Microsoft MSXML4.msi.  

As usual, the pointy-haired boss doesn't understand that the install developer doesn't (yet) know squat about Python.  I've just gotta get it installed so it can support our product.  I know, Disutils is supposed to make this unnecessary, but (I'm told) we use some stuff in PyWin which uses some stuff in Python which is not normally packaged that way.  If I've been told wrong, tell me what I need to know/do in order to package up our spiffy new written-in-Python software.

Any help, hints, even sympathy will be greatly appreciated.

Regards,
T. Scott Mayham
Sr. Analyst
Manhattan Associates
smayham at manh.com
678-597-6874

"A government which attempts more than it ought, will perform less."  ...Thomas Babington Macaulay


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050823/8532f4af/attachment.htm

From dangoor at gmail.com  Wed Aug 24 03:53:06 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 23 Aug 2005 21:53:06 -0400
Subject: [Distutils] Opening question
In-Reply-To: <CC5B93677E8E5347B0E156A7D830ED32466D67@Ma-atl58.us.manh.com>
References: <CC5B93677E8E5347B0E156A7D830ED32466D67@Ma-atl58.us.manh.com>
Message-ID: <3f085ecd05082318535b061cf0@mail.gmail.com>

Hey, welcome to Python!

I'll answer what I can below... maybe some others can amplify.

On 8/23/05, Scott Mayham <Smayham at manh.com> wrote:
> We are rushing out the door, a new version of one of our main products. 
> Part of what's new about this release is that some parts of it are written
> in Python (2.4, I think).  These parts also use some bits for the PyWin
> package.  I have been chartered with exploring whether we can bundle the
> Python Windows installer (the MSI that the link on the "downloads" page
> points to) along with our product, and whether it (the Python installer) can
> be launched as a silent sub-install, in much the same way as we presently
> launch the Microsoft MSXML4.msi.  

You *can* bundle the Python Windows installer (as in: it is legal to
do so). I don't know anything about MSI installers, so I can't comment
on the silent sub-install part.

> As usual, the pointy-haired boss doesn't understand that the install
> developer doesn't (yet) know squat about Python.  I've just gotta get it
> installed so it can support our product.  I know, Disutils is supposed to
> make this unnecessary, but (I'm told) we use some stuff in PyWin which uses
> some stuff in Python which is not normally packaged that way.  If I've been
> told wrong, tell me what I need to know/do in order to package up our spiffy
> new written-in-Python software. 

Here's how I bundle up my Windows software: use py2exe:

http://starship.python.net/crew/theller/py2exe/

which should grab everything you need (including a Python interpreter)
and drop it in a directory.

I use InnoSetup for the installer myself, but that's just personal preference.

If you have multiple standalone python scripts or there is some other
need you have that eliminates creating a single exe and using py2exe,
then including a full Python interpreter may indeed make sense.

Kevin

From antti.korvenoja at helsinki.fi  Wed Aug 24 09:01:08 2005
From: antti.korvenoja at helsinki.fi (Antti Korvenoja)
Date: Wed, 24 Aug 2005 10:01:08 +0300
Subject: [Distutils] Installing data files
Message-ID: <1124866868.430c1b34c3346@www2.helsinki.fi>

Hello!

I would like to ask for advice how to install data files that is used by the
modules or scripts in a package. I need to provide the path where some
template image files are installed, also in the case that the package is not
installed under sys.prefix. Studying the documents did not provide me an
answer and searching previous discussions about this issue left me with an
impression that this might not be supported in current distutils.

Is it possible to extract somehow within distutils whether an alternate
installation location was specified or do I need to resort to parsing the
commandline in my setup.py?

Any advice is highly appreciated,

Antti Korvenoja





From ianb at colorstudy.com  Wed Aug 24 18:05:44 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 24 Aug 2005 11:05:44 -0500
Subject: [Distutils] SQLObject and Python 2.2
In-Reply-To: <5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
References: <20050824133826.GB20338@phd.pp.ru>
	<20050824133826.GB20338@phd.pp.ru>
	<5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
Message-ID: <430C9AD8.70609@colorstudy.com>

Phillip J. Eby wrote:
> At 10:16 AM 8/24/2005 -0500, Ian Bicking wrote:
> 
>> Oleg Broytmann wrote:
>>
>>> Hello, Ian. Have we stopped supporting Python 2.2?
>>> $ python2.2 setup.py install -O2
>>> Traceback (most recent call last):
>>>   File "setup.py", line 2, in ?
>>>     use_setuptools()
>>>   File "ez_setup/__init__.py", line 66, in use_setuptools
>>>     egg = download_setuptools(version, download_base, to_dir)
>>>   File "ez_setup/__init__.py", line 100, in download_setuptools
>>>     from distutils import log
>>> ImportError: cannot import name log
>>
>>
>> No, not really (though I admit I don't test it frequently).  ez_setup 
>> comes from setuptools -- Phillip, should 2.2 be supported?  SQLObject 
>> is bringing ez_setup in as an external.
> 
> 
> Sorry; I've moved on from 2.2, even if Google hasn't.  If you want to 
> provide backward compatibility patches, I'll certainly look at them, but 
> I can't guarantee not to accidentally break them in some future release.

Well, it's broken in a couple ways.  One is that a setuptools egg for 
py2.2 doesn't exist.

But anyway, I've attached a patch that fixes two problems I've 
encountered in ez_setup.py -- distutils.log, and tempfile.mkdtemp (doing 
crude clones of both).  I don't know what issues setuptools itself might 
still have.  This doesn't make it functional, but I guess it's closer.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ez_setup_py22.diff
Type: text/x-patch
Size: 1944 bytes
Desc: not available
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050824/fb3704c2/ez_setup_py22.bin

From pje at telecommunity.com  Wed Aug 24 18:35:09 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 24 Aug 2005 12:35:09 -0400
Subject: [Distutils] SQLObject and Python 2.2
In-Reply-To: <430C9AD8.70609@colorstudy.com>
References: <5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
	<20050824133826.GB20338@phd.pp.ru>
	<20050824133826.GB20338@phd.pp.ru>
	<5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050824120926.028423f8@mail.telecommunity.com>

At 11:05 AM 8/24/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 10:16 AM 8/24/2005 -0500, Ian Bicking wrote:
>>
>>>Oleg Broytmann wrote:
>>>
>>>>Hello, Ian. Have we stopped supporting Python 2.2?
>>>>$ python2.2 setup.py install -O2
>>>>Traceback (most recent call last):
>>>>   File "setup.py", line 2, in ?
>>>>     use_setuptools()
>>>>   File "ez_setup/__init__.py", line 66, in use_setuptools
>>>>     egg = download_setuptools(version, download_base, to_dir)
>>>>   File "ez_setup/__init__.py", line 100, in download_setuptools
>>>>     from distutils import log
>>>>ImportError: cannot import name log
>>>
>>>
>>>No, not really (though I admit I don't test it frequently).  ez_setup 
>>>comes from setuptools -- Phillip, should 2.2 be supported?  SQLObject is 
>>>bringing ez_setup in as an external.
>>
>>Sorry; I've moved on from 2.2, even if Google hasn't.  If you want to 
>>provide backward compatibility patches, I'll certainly look at them, but 
>>I can't guarantee not to accidentally break them in some future release.
>
>Well, it's broken in a couple ways.  One is that a setuptools egg for 
>py2.2 doesn't exist.

Yep.  Trying to build one should tell you what else is missing - like 
zipimport, for example.  Just for the heck of it, I tried building a 
setuptools egg for 2.2 and quickly discovered why I decided to go with 2.3 
only for setuptools: zipimport isn't in Python 2.2, and can't be backported 
because it's based on PEP 302.  And pkg_resources is PEP 302-based right 
down to the bone.

But even if you comment out the zipimport stuff, and stub out the PEP 302 
hooks so they only do filesystem stuff, you then need to include the 2.3 
'sets' module, because Requirement objects use ImmutableSet internally to 
handle comparisons.  And comment out 'classifiers' in setup.py because 
that's not in 2.3.  And all the 'imp.acquire_lock()' calls in 
pkg_resources.  And add "from __future__ import generators" to every 
freaking module.  And then you need to backport distutils.log.  And then 
fix up the dict.fromkeys() calls and the extended list slicing.

And that's as far as I got in 15 minutes or so, with bdist_egg only getting 
far enough along to try to write the toplevel.txt file and failing on a 
dict.fromkeys() call.  Still, in some ways that's pretty good, since to get 
that far it had to boot pkg_resources and load several entry point groups, 
albeit all from one unpacked egg.

When you get far enough to successfully build a Python 2.2 egg for 
setuptools, it'd be interesting to take a look at your patch.  You'll also 
need to add code to ez_setup to *extract* the egg, because Python 2.2 can't 
import from a zipfile.  This also means that setuptools and EasyInstall 
running under Python 2.2 should always mark all eggs as not zip safe, and 
always extract them.


From ianb at colorstudy.com  Wed Aug 24 18:59:34 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 24 Aug 2005 11:59:34 -0500
Subject: [Distutils] SQLObject and Python 2.2
In-Reply-To: <5.1.1.6.0.20050824120926.028423f8@mail.telecommunity.com>
References: <5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
	<20050824133826.GB20338@phd.pp.ru>
	<20050824133826.GB20338@phd.pp.ru>
	<5.1.1.6.0.20050824113542.01b24dc0@mail.telecommunity.com>
	<5.1.1.6.0.20050824120926.028423f8@mail.telecommunity.com>
Message-ID: <430CA776.3060904@colorstudy.com>

Phillip J. Eby wrote:
> Yep.  Trying to build one should tell you what else is missing - like 
> zipimport, for example.  Just for the heck of it, I tried building a 
> setuptools egg for 2.2 and quickly discovered why I decided to go with 
> 2.3 only for setuptools: zipimport isn't in Python 2.2, and can't be 
> backported because it's based on PEP 302.  And pkg_resources is PEP 
> 302-based right down to the bone.

Maybe it'll be easier then to do:

import sys
if sys.version_info < (2, 3):
     from distutils.core import setup
else:
     from ez_setup import use_setuptools
     use_setuptools()
     from setuptools import setup

I'll also also have to patch distutils to accept the extra keyword 
parameters, but that's not a big deal.  That just means I can't use 
find_packages, and of course installation under Python 2.2 will be a bit 
different, but that's fine.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From hawk78_it at yahoo.it  Thu Aug 25 23:17:57 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Thu, 25 Aug 2005 23:17:57 +0200
Subject: [Distutils] PythonModulePackaging
Message-ID: <200508252317.57762.hawk78_it@yahoo.it>

I have just put on pipy the first version of easy-deb: a tool to make source debs out of standard python modules.
I would really like to thank everyone who helped me. A special thank to "Phillip J. Eby" for setuptools.
I someone woul like to test it and tell if how it is working... I will be really HAPPY :-)

Vincenzo

I attach the same docs:

==========
=easy-deb=
==========

FEATURES
========

* Packages python modules
   * from source distribution files
   * from urls
   * from PyPI records
* can package it self
* handles dependencies
* intstalls modules into eggs (from setuptools)
* can activate or deactivate an installed egg (add or remove from sys.path)
* handles a database of dependencies

=======================
=easy_deb instructions=
=======================

SYNOPSIS
========
easy-deb:
^^^^^^^^^^
   Type 
   $ easy-deb -h 
   for usage and options synopsis:
usage: easy-deb [options] (pypi-modulename | archive-file-name| url)

options:
  -h, --help            show this help message and exit
  -v VERSIONS, --python-versions=VERSIONS
                        Coma separated list of python versions to package for.
                        E.g.: -v 2.3,2.4
  -d DEPS, --debian-deps=DEPS
                        Standard debian dependency string. %v is replaced with
                        python version
  -b BDEPS, --debian-build-deps=BDEPS
                        Standard debian dependency string. %v is replaced with
                        python version
  -f FIND_LINKS, --find-links=FIND_LINKS
                        Additionnal links to scan
  -D DEST_DIR, --dest-dir=DEST_DIR
                        Distribution downloaded into file
  -c, --common-dir      Install to commond dir from where tree linking is done
  -a, --arch-dep        Build arch dep packages
  -u, --update-database
                        Update the database of python modules options

update-pypi:
^^^^^^^^^^^^^
   Type 
   $ update-pypi -h 
   for usage and options synopsis:
usage: update-pypi [options] (module-name | show | updatew)

options:
  -h, --help    show this help message and exit
  -a, --add     Add egg to sys.path
  -r, --remove  Add egg to sys.path



USAGE
=====
easy-deb: this tool will create a debian source package into a deb-pkg-<modulename> directory.
To compile the source package use the standard debian tools (debuild or "fakeroot debian/rules binary")

update-pypi: this script is used to enable and disable installed egg (you can enable/disable all eggs 
ing the removing the All package). Enabled eggs are on sys.path, disabled eggs must be pkg_resource.require() before you can import from them.

EXAMPLE
=======
To install module "graph":

$ easy-deb -v2.4 graph -D /tmp/
$ cd /tmp/deb-pkg-graph0.4/graph-0.4
$ fakeroot debian/rules binary
$ sudo dpkg -i ../*.deb

$ update-pypi -a graph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050825/80d56b7f/attachment.htm

From hancock at anansispaceworks.com  Fri Aug 26 03:13:40 2005
From: hancock at anansispaceworks.com (Terry Hancock)
Date: Thu, 25 Aug 2005 20:13:40 -0500
Subject: [Distutils] PythonModulePackaging
In-Reply-To: <200508252317.57762.hawk78_it@yahoo.it>
References: <200508252317.57762.hawk78_it@yahoo.it>
Message-ID: <200508252013.41212.hancock@anansispaceworks.com>

On Thursday 25 August 2005 04:17 pm, Vincenzo Di Massa wrote:
> I have just put on pipy the first version of easy-deb: a tool to make source debs out of standard python modules.
> I would really like to thank everyone who helped me. A special thank to "Phillip J. Eby" for setuptools.
> I someone woul like to test it and tell if how it is working... I will be really HAPPY :-)

Searching "PyPI", which now calls itself "Cheese Shop", does what one
might expect a Monty Python Cheese Shop to do when asked for
easy-deb:

Cheese Shop: Index of Packages
There were no matches.

Is there perhaps another source?

I admit I'm not a regular PyPI user, most of the packages I'm looking for
turn out to already be in Debian.   But I'd like to try this out.

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks  http://www.anansispaceworks.com


From pje at telecommunity.com  Fri Aug 26 04:21:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 25 Aug 2005 22:21:36 -0400
Subject: [Distutils] PythonModulePackaging
In-Reply-To: <200508252013.41212.hancock@anansispaceworks.com>
References: <200508252317.57762.hawk78_it@yahoo.it>
	<200508252317.57762.hawk78_it@yahoo.it>
Message-ID: <5.1.1.6.0.20050825222027.01b415a0@mail.telecommunity.com>

At 08:13 PM 8/25/2005 -0500, Terry Hancock wrote:
>On Thursday 25 August 2005 04:17 pm, Vincenzo Di Massa wrote:
> > I have just put on pipy the first version of easy-deb: a tool to make 
> source debs out of standard python modules.
> > I would really like to thank everyone who helped me. A special thank to 
> "Phillip J. Eby" for setuptools.
> > I someone woul like to test it and tell if how it is working... I will 
> be really HAPPY :-)
>
>Searching "PyPI", which now calls itself "Cheese Shop", does what one
>might expect a Monty Python Cheese Shop to do when asked for
>easy-deb:
>
>Cheese Shop: Index of Packages
>There were no matches.

It appears that Vincenzo has actually named the package "easydeb", at least 
on PyPI.  This is probably due to the '-' vs. '_' issue in distutils 
distribution names.


From ianb at colorstudy.com  Fri Aug 26 07:06:14 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 26 Aug 2005 00:06:14 -0500
Subject: [Distutils] svn tagging setuptools command
Message-ID: <430EA346.8040600@colorstudy.com>

So, I made my first setuptools command; I don't know where it really 
belongs, but I've dumped it here for now: 
http://svn.pythonpaste.org/Paste/Deploy/trunk/paste/deploy/tag.py

Then you do "python setup.py tag --version=0.3" to copy the trunk to a 
tag (in a normally-laid-out subversion repository) and adjust setup.py 
and setup.cfg.  It's not particularly robust, but I thought I'd note 
that it's there.

I can vaguely imagine other svn based setup.py commands for project 
management; maybe a few thrown together could become a package.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From hawk78_it at yahoo.it  Fri Aug 26 16:45:26 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Fri, 26 Aug 2005 16:45:26 +0200
Subject: [Distutils] setuptools deps
Message-ID: <200508261645.26512.hawk78_it@yahoo.it>

Hello,
As you know I'm developing easy-deb using setuptools.
I need a way to query a module dependencies, but I don't know how to achieve this.

I need to know dependencies because when I install an egg into a package I don't want the dependencies to go into the same package.
So I detect if setup.py uses setuptools (grep setuptools setup.py) and then:
if the module uses setuptools:
 I call setup.py bdist_egg -d dest/pkg/dir
else:
 I call easy_install.py -d dest/pkg/dir . # in the setup.py containing dir

Is there a way to tell easy_install "don't install dependencies but tell me what they are".

Better, I would like to be able to detect dependencies from inside python.

Thank you

Ciao
Vincenzo

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From pje at telecommunity.com  Fri Aug 26 17:43:31 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 26 Aug 2005 11:43:31 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <430EA346.8040600@colorstudy.com>
Message-ID: <5.1.1.6.0.20050826114135.01b421a0@mail.telecommunity.com>

At 12:06 AM 8/26/2005 -0500, Ian Bicking wrote:
>So, I made my first setuptools command; I don't know where it really
>belongs, but I've dumped it here for now:
>http://svn.pythonpaste.org/Paste/Deploy/trunk/paste/deploy/tag.py
>
>Then you do "python setup.py tag --version=0.3" to copy the trunk to a
>tag (in a normally-laid-out subversion repository) and adjust setup.py
>and setup.cfg.  It's not particularly robust, but I thought I'd note
>that it's there.

FYI, your setup.cfg changes could probably be done by calling the 
'edit_config' function in setuptools' "setopt" module.


From martin at v.loewis.de  Sat Aug 27 21:15:01 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 27 Aug 2005 21:15:01 +0200
Subject: [Distutils] Announcing bdist_msi 1.0b1
Message-ID: <4310BBB5.2020109@v.loewis.de>

I'm happy to announce the first beta release of my long-planned
distutils command bdist_msi, at

http://www.dcl.hpi.uni-potsdam.de/home/loewis/bdist_msi/
http://www.dcl.hpi.uni-potsdam.de/home/loewis/bdist_msi/bdist_msi-1.0b1.win32-py2.4.msi
http://www.dcl.hpi.uni-potsdam.de/home/loewis/bdist_msi/bdist_msi-1.0b1.zip

Please let me know what you think; it may take some weeks before I can
respond, though.

Regards,
Martin

From ianb at colorstudy.com  Sat Aug 27 21:26:55 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sat, 27 Aug 2005 14:26:55 -0500
Subject: [Distutils] Fixing the stdlib
Message-ID: <4310BE7F.90109@colorstudy.com>

Vaguely off-topic, but I figured someone here might know...

Anyway, I want to provide backported stdlib modules to older versions of 
Python.  Most of these are trivial to backport, so it's just a matter of 
accumulating them in one place.

So I'm wondering how best I should put this in place.  One way would be 
to import the backported modules, then modify sys.modules.  But this 
requires importing everything ahead of time, which is no good.  At the 
same time, I'd rather have the process Instantly Seem Upgraded (assuming 
the program isn't using any language features not supported by the 
process).  I could actually distribute something that included the 
top-level modules; this would work for something like optparse or 
subprocess, where the module simply didn't exist before.  But this 
doesn't work when providing updated modules that existed before but have 
been enhanced, like doctest.

I'm also not sure when I should monkeypatch the existing module, or 
provide a new one.  For instance, tempfile added a couple functions in 
2.4, and UserDict added a useful class (DictMixin).

Thoughts?

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From hancock at anansispaceworks.com  Sun Aug 28 09:54:23 2005
From: hancock at anansispaceworks.com (Terry Hancock)
Date: Sun, 28 Aug 2005 02:54:23 -0500
Subject: [Distutils] PythonModulePackaging
In-Reply-To: <5.1.1.6.0.20050825222027.01b415a0@mail.telecommunity.com>
References: <200508252317.57762.hawk78_it@yahoo.it>
	<5.1.1.6.0.20050825222027.01b415a0@mail.telecommunity.com>
Message-ID: <200508280254.23619.hancock@anansispaceworks.com>

On Thursday 25 August 2005 09:21 pm, Phillip J. Eby wrote:
> It appears that Vincenzo has actually named the package "easydeb", at least 
> on PyPI.  This is probably due to the '-' vs. '_' issue in distutils 
> distribution names.

Thanks!  I see it now.

Cheers,
Terry

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks  http://www.anansispaceworks.com


From ianb at colorstudy.com  Sun Aug 28 22:07:48 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 28 Aug 2005 15:07:48 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <430EA346.8040600@colorstudy.com>
References: <430EA346.8040600@colorstudy.com>
Message-ID: <43121994.6020503@colorstudy.com>

Ian Bicking wrote:
> So, I made my first setuptools command; I don't know where it really 
> belongs, but I've dumped it here for now: 
> http://svn.pythonpaste.org/Paste/Deploy/trunk/paste/deploy/tag.py

I had kind of forgotten about buildutils 
(http://buildutils.lesscode.org/) -- maybe this should go there?

Looking through buildtools, there's also some overlap with some of the 
other things I'm thinking about in terms of providing management 
frontends to applications.  I've started just a small portion of this in 
PasteScript: http://svn.pythonpaste.org/Paste/Script/trunk/

But I've been debating how the system should work; setuptools/distutils 
commands are extended globally, which sometimes is perfect.  But other 
times I want them local to a project.  For instance, I want a command to 
create a servlet -- but only in my WebKit/Wareweb projects, of course. 
buildtools pytest command is nice, but only if I'm using py.test -- if 
I'm not using py.test there's not much point.  In contrast, buildtools 
stats and flakes commands are applicable to any project.

In PasteScript I've been setting up a system where plugins are 
specifically enabled (this is actually about all I've really written 
yet).  paste.script.command 
(http://svn.pythonpaste.org/Paste/Script/trunk/paste/script/command.py) 
basically looks for a special file in the egg-info dir 
(paster_plugins.txt) and loads those plugins (extra commands as entry 
points, recursively checking for extra plugins), as well as any plugins 
on the command-line.  A command I have not yet written would allow you 
to add plugins (without editing that file), so you might add a plugin 
for your web framework (e.g., that create servlet command), your 
persistence framework (e.g., SQLObject has tools for doing management on 
databases), your development environment (e.g., the svn tag tool), your 
documentation framework (pudge, epydoc, etc), and so on.  Most of these 
are orthogonal to each other.

I don't know if setup.py is the right frontend for all of these.  One 
specific issue is that one command creates the basic framework of a 
package, including a setup.py file.  But maybe setuptools could be 
extended to support two kinds of entry points: one that is applied 
globally (what it currently has), and one that is applied only on demand 
(for framework-specific commands).

One option for the creation of an initial framework would be something 
like "python -m buildtools.createpackage" or something; it's not a 
command you need that often.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 29 00:01:46 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 28 Aug 2005 18:01:46 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <43121994.6020503@colorstudy.com>
References: <430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
Message-ID: <5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>

At 03:07 PM 8/28/2005 -0500, Ian Bicking wrote:
>I don't know if setup.py is the right frontend for all of these.  One
>specific issue is that one command creates the basic framework of a
>package, including a setup.py file.  But maybe setuptools could be
>extended to support two kinds of entry points: one that is applied
>globally (what it currently has), and one that is applied only on demand
>(for framework-specific commands).

My thought is that at some point (0.7?) setuptools will have a 'nest' 
script that runs general-purpose commands, and it will have its own entry 
point group for commands to install, list, uninstall, etc.


From pje at telecommunity.com  Mon Aug 29 00:26:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 28 Aug 2005 18:26:13 -0400
Subject: [Distutils] setuptools deps
In-Reply-To: <200508261645.26512.hawk78_it@yahoo.it>
Message-ID: <5.1.1.6.0.20050828182106.01b96de0@mail.telecommunity.com>

At 04:45 PM 8/26/2005 +0200, Vincenzo Di Massa wrote:
>Hello,
>As you know I'm developing easy-deb using setuptools.
>I need a way to query a module dependencies, but I don't know how to 
>achieve this.
>
>I need to know dependencies because when I install an egg into a package I 
>don't want the dependencies to go into the same package.
>So I detect if setup.py uses setuptools (grep setuptools setup.py) and then:
>if the module uses setuptools:
>  I call setup.py bdist_egg -d dest/pkg/dir
>else:
>  I call easy_install.py -d dest/pkg/dir . # in the setup.py containing dir
>
>Is there a way to tell easy_install "don't install dependencies but tell 
>me what they are".
>
>Better, I would like to be able to detect dependencies from inside python.


The way that easy_install does this sort of thing is to run the package's 
setup script and execute a distutils command.  The distutils command then 
has access to the distribution metadata.

So, your high-level script could:

1. import setuptools
2. invoke easy_install --editable to download and extract the target package
3. use setuptools.sandbox.run_setup() to run the setup script, telling it 
to run a 'bdist_easydeb' command or some such
4. Have the command access metadata directly from the setup() call, e.g. 
the 'install_requires' attribute.  Be prepared for the possibility that the 
attribute(s) you're looking for might be None.

This is the easiest way to get information from a setup script, and with 
the latest version of setuptools it should work fine even if the setup 
script does not import setuptools itself.


From ianb at colorstudy.com  Mon Aug 29 01:32:50 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 28 Aug 2005 18:32:50 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
References: <430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
Message-ID: <431249A2.9050101@colorstudy.com>

Phillip J. Eby wrote:
> At 03:07 PM 8/28/2005 -0500, Ian Bicking wrote:
> 
>> I don't know if setup.py is the right frontend for all of these.  One
>> specific issue is that one command creates the basic framework of a
>> package, including a setup.py file.  But maybe setuptools could be
>> extended to support two kinds of entry points: one that is applied
>> globally (what it currently has), and one that is applied only on demand
>> (for framework-specific commands).
> 
> 
> My thought is that at some point (0.7?) setuptools will have a 'nest' 
> script that runs general-purpose commands, and it will have its own 
> entry point group for commands to install, list, uninstall, etc.

I suppose this would be a good place for a command to setup a new 
project (since that's kind of package management -- at least 
chronologically); would it be an appropriate entry point for other more 
project-specific commands?  It doesn't seem like it to me; but then 
setup.py seems appropriate for those commands.

I think for the command plugins, I'd imagine:

setup(
   ....
   setup_plugins=['spec1', 'spec2', ...])

Which writes to .egg-info/setup_plugins.txt (one line per requirement). 
  Then it includes any 'distutils.extra_commands' entry points from 
those eggs, which act exactly like distutils.commands.

Or maybe it could be something in setup.cfg?  I'd actually find this 
sort of thing quite awkward in setup.py, since setup.py can't be 
(reasonably) written to.

Incidentally, is it possible to put setup.cfg in the .egg-info directory 
or somewhere?  It messes up tab completion.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 29 01:44:21 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 28 Aug 2005 19:44:21 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <431249A2.9050101@colorstudy.com>
References: <5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>

At 06:32 PM 8/28/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>At 03:07 PM 8/28/2005 -0500, Ian Bicking wrote:
>>
>>>I don't know if setup.py is the right frontend for all of these.  One
>>>specific issue is that one command creates the basic framework of a
>>>package, including a setup.py file.  But maybe setuptools could be
>>>extended to support two kinds of entry points: one that is applied
>>>globally (what it currently has), and one that is applied only on demand
>>>(for framework-specific commands).
>>
>>My thought is that at some point (0.7?) setuptools will have a 'nest' 
>>script that runs general-purpose commands, and it will have its own entry 
>>point group for commands to install, list, uninstall, etc.
>
>I suppose this would be a good place for a command to setup a new project 
>(since that's kind of package management -- at least chronologically); 
>would it be an appropriate entry point for other more project-specific 
>commands?  It doesn't seem like it to me; but then setup.py seems 
>appropriate for those commands.

Yep.


>I think for the command plugins, I'd imagine:
>
>setup(
>   ....
>   setup_plugins=['spec1', 'spec2', ...])
>
>Which writes to .egg-info/setup_plugins.txt (one line per 
>requirement).  Then it includes any 'distutils.extra_commands' entry 
>points from those eggs, which act exactly like distutils.commands.

I'm confused.  If you want to add custom commands to a particular project's 
setup.py, you just supply the standard distutils 'cmdclass' argument to 
setup, e.g.:

     setup(
         ...
         cmdclass = {'mycommand':mycommand}
     )

Conversely, if what you mean is that you'd like to ensure that certain 
distutils extensions are available when setup.py runs, then you should use 
the 'setup_requires' argument (see the setuptools docs).


>Or maybe it could be something in setup.cfg?  I'd actually find this sort 
>of thing quite awkward in setup.py, since setup.py can't be (reasonably) 
>written to.

I'm still not following you.  Configuration for custom commands of course 
can always be put in setup.cfg.


>Incidentally, is it possible to put setup.cfg in the .egg-info directory 
>or somewhere?

No; that file belongs to the distutils, and the distutils decree that it's 
found in the setup.py directory.


From ianb at colorstudy.com  Mon Aug 29 01:58:37 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 28 Aug 2005 18:58:37 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
References: <5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
Message-ID: <43124FAD.20402@colorstudy.com>

Phillip J. Eby wrote:
>> I think for the command plugins, I'd imagine:
>>
>> setup(
>>   ....
>>   setup_plugins=['spec1', 'spec2', ...])
>>
>> Which writes to .egg-info/setup_plugins.txt (one line per 
>> requirement).  Then it includes any 'distutils.extra_commands' entry 
>> points from those eggs, which act exactly like distutils.commands.
> 
> 
> I'm confused.  If you want to add custom commands to a particular 
> project's setup.py, you just supply the standard distutils 'cmdclass' 
> argument to setup, e.g.:
> 
>     setup(
>         ...
>         cmdclass = {'mycommand':mycommand}
>     )
> 
> Conversely, if what you mean is that you'd like to ensure that certain 
> distutils extensions are available when setup.py runs, then you should 
> use the 'setup_requires' argument (see the setuptools docs).

I'm thinking of framework-specific commands, not purely project-specific.

For instance, if I have a project that uses SQLObject, I would like to 
make the SQLObject admin commands available.  This way I could do things 
like "python setup.py sqlcreate", "python setup.py sqlstatus", etc.  I 
want a single setting in setup.py to add all the commands from 
SQLObject.  But I also don't want every setup.py on the system to have 
these same commands, because they mean nothing for a project that 
doesn't use SQLObject.


-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 29 08:45:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 29 Aug 2005 02:45:53 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <43124FAD.20402@colorstudy.com>
References: <5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>

At 06:58 PM 8/28/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>Conversely, if what you mean is that you'd like to ensure that certain 
>>distutils extensions are available when setup.py runs, then you should 
>>use the 'setup_requires' argument (see the setuptools docs).
>
>I'm thinking of framework-specific commands, not purely project-specific.

Then use setup_requires to list the eggs that offer those commands.


>For instance, if I have a project that uses SQLObject, I would like to 
>make the SQLObject admin commands available.  This way I could do things 
>like "python setup.py sqlcreate", "python setup.py sqlstatus", etc.  I 
>want a single setting in setup.py to add all the commands from 
>SQLObject.  But I also don't want every setup.py on the system to have 
>these same commands, because they mean nothing for a project that doesn't 
>use SQLObject.

Yep, this is exactly what the setup_requires keyword is for.  If SQLObject 
is the egg with entry points for those commands, then 
setup_requires=['SQLObject>=0.7b1'] or whatever will do the trick, to the 
point of downloading SQLObject to the setup directory and installing it 
there if need be.

(Of course, if SQLObject is already on sys.path due to the easy_install.pth 
file, then you will get those commands in every project anyway, but oh well.)


From ianb at colorstudy.com  Mon Aug 29 17:38:25 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 10:38:25 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
References: <5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
Message-ID: <43132BF1.2070207@colorstudy.com>

Phillip J. Eby wrote:
>> For instance, if I have a project that uses SQLObject, I would like to 
>> make the SQLObject admin commands available.  This way I could do 
>> things like "python setup.py sqlcreate", "python setup.py sqlstatus", 
>> etc.  I want a single setting in setup.py to add all the commands from 
>> SQLObject.  But I also don't want every setup.py on the system to have 
>> these same commands, because they mean nothing for a project that 
>> doesn't use SQLObject.
> 
> 
> Yep, this is exactly what the setup_requires keyword is for.  If 
> SQLObject is the egg with entry points for those commands, then 
> setup_requires=['SQLObject>=0.7b1'] or whatever will do the trick, to 
> the point of downloading SQLObject to the setup directory and installing 
> it there if need be.
> 
> (Of course, if SQLObject is already on sys.path due to the 
> easy_install.pth file, then you will get those commands in every project 
> anyway, but oh well.)

That last part is exactly what I'm trying to avoid.  If every framework 
starts publishing 1-5 distutils commands -- and I think most frameworks 
have that potential -- it's going to be a bit overwhelming.  And I don't 
like that installing SQLObject will needlessly change the entire 
system's setup.py commands.

If packages listed in setup_requires had extra commands loaded, that 
would work fine.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 29 17:47:52 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 29 Aug 2005 11:47:52 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <43132BF1.2070207@colorstudy.com>
References: <5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>

At 10:38 AM 8/29/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>>For instance, if I have a project that uses SQLObject, I would like to 
>>>make the SQLObject admin commands available.  This way I could do things 
>>>like "python setup.py sqlcreate", "python setup.py sqlstatus", etc.  I 
>>>want a single setting in setup.py to add all the commands from 
>>>SQLObject.  But I also don't want every setup.py on the system to have 
>>>these same commands, because they mean nothing for a project that 
>>>doesn't use SQLObject.
>>
>>Yep, this is exactly what the setup_requires keyword is for.  If 
>>SQLObject is the egg with entry points for those commands, then 
>>setup_requires=['SQLObject>=0.7b1'] or whatever will do the trick, to the 
>>point of downloading SQLObject to the setup directory and installing it 
>>there if need be.
>>(Of course, if SQLObject is already on sys.path due to the 
>>easy_install.pth file, then you will get those commands in every project 
>>anyway, but oh well.)
>
>That last part is exactly what I'm trying to avoid.  If every framework 
>starts publishing 1-5 distutils commands -- and I think most frameworks 
>have that potential -- it's going to be a bit overwhelming.  And I don't 
>like that installing SQLObject will needlessly change the entire system's 
>setup.py commands.
>
>If packages listed in setup_requires had extra commands loaded, that would 
>work fine.

Personally, I think the simple way to deal with this is to package 
distutils extensions as  separate packages; e.g. SQLObjectSetup or some such.



From ianb at colorstudy.com  Mon Aug 29 18:57:51 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 11:57:51 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
References: <5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<430EA346.8040600@colorstudy.com> <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
Message-ID: <43133E8F.5060708@colorstudy.com>

Phillip J. Eby wrote:
>> If packages listed in setup_requires had extra commands loaded, that 
>> would work fine.
> 
> 
> Personally, I think the simple way to deal with this is to package 
> distutils extensions as  separate packages; e.g. SQLObjectSetup or some 
> such.

Is there a way to force packages to be installed multiversion (from the 
package metadata itself)?  I'm fine putting it in a seperate package, 
but I don't want to require people to install that package in a special 
way -- especially since it's kind of hard to do with easy_install 
anyway, when a requirement is installed implicitly.  Maybe something in 
setup.cfg will do it?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Mon Aug 29 19:01:34 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 12:01:34 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <43133E8F.5060708@colorstudy.com>
References: <5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>	<430EA346.8040600@colorstudy.com>
	<430EA346.8040600@colorstudy.com>	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<43133E8F.5060708@colorstudy.com>
Message-ID: <43133F6E.9040902@colorstudy.com>

Ian Bicking wrote:
> Phillip J. Eby wrote:
> 
>>>If packages listed in setup_requires had extra commands loaded, that 
>>>would work fine.
>>
>>
>>Personally, I think the simple way to deal with this is to package 
>>distutils extensions as  separate packages; e.g. SQLObjectSetup or some 
>>such.
> 
> 
> Is there a way to force packages to be installed multiversion (from the 
> package metadata itself)?  I'm fine putting it in a seperate package, 
> but I don't want to require people to install that package in a special 
> way -- especially since it's kind of hard to do with easy_install 
> anyway, when a requirement is installed implicitly.  Maybe something in 
> setup.cfg will do it?

Though I'll note this is also somewhat awkward -- for instance, looking 
at buildtools,I think some of those commands should be globally 
installed, and some shouldn't.  But changing that means refactoring the 
commands into two distributions; inter-distribution refactoring feels 
very heavy to me, especially for something I'd consider a UI adjustment.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From dangoor at gmail.com  Mon Aug 29 19:41:48 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Mon, 29 Aug 2005 13:41:48 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
References: <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<43132BF1.2070207@colorstudy.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
Message-ID: <3f085ecd0508291041237a86f8@mail.gmail.com>

On 8/29/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> Personally, I think the simple way to deal with this is to package
> distutils extensions as  separate packages; e.g. SQLObjectSetup or some such.

Could this type of thing be done with a "feature"? That way, it's
still ultimately part of SQLObject, but people can choose to leave it
out if they wish...

Kevin

From ianb at colorstudy.com  Mon Aug 29 21:04:29 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 14:04:29 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <3f085ecd0508291041237a86f8@mail.gmail.com>
References: <430EA346.8040600@colorstudy.com>	
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>	
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>	
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>	
	<43132BF1.2070207@colorstudy.com>	
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<3f085ecd0508291041237a86f8@mail.gmail.com>
Message-ID: <43135C3D.30807@colorstudy.com>

Kevin Dangoor wrote:
> On 8/29/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
>>Personally, I think the simple way to deal with this is to package
>>distutils extensions as  separate packages; e.g. SQLObjectSetup or some such.
> 
> 
> Could this type of thing be done with a "feature"? That way, it's
> still ultimately part of SQLObject, but people can choose to leave it
> out if they wish...

Entry points can require features, but AFAIK they can't be 
feature-specific.  Unless if an entry point depended on a feature, and 
that feature wasn't loaded (which they aren't by default, even when the 
egg is installed for implicit use without require()), and the entry 
point wasn't loaded as a result.

However, I think the entry point will be found either way, and then the 
feature required when the entry point is actually used.

If there was a way to say that an entry point is only available when a 
feature is loaded (as opposed to saying that a feature must be loaded 
before using the entry point), then that would work.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Tue Aug 30 04:24:52 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 21:24:52 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <43135C3D.30807@colorstudy.com>
References: <430EA346.8040600@colorstudy.com>		<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>		<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>		<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>		<43132BF1.2070207@colorstudy.com>		<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>	<3f085ecd0508291041237a86f8@mail.gmail.com>
	<43135C3D.30807@colorstudy.com>
Message-ID: <4313C374.2040605@colorstudy.com>

Upon further thought, I think I'm forcing this into distutils.commands 
entry points unnecessarily.  --command-packages can be applied through 
the commandline or through the setup.cfg file.  I have to figure out the 
exact details, but as I remember it's not particularly difficult.

I expect I'll still want a entry point for all these that acts like 
distutils.commands, but under a different name.  Then a command-packages 
managing command (that's hard to parse ;) could use that to list 
available command packages.  The command-packages managing command 
itself would be installed globally.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From dangoor at gmail.com  Tue Aug 30 04:49:54 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Mon, 29 Aug 2005 22:49:54 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <4313C374.2040605@colorstudy.com>
References: <430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<43132BF1.2070207@colorstudy.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<3f085ecd0508291041237a86f8@mail.gmail.com>
	<43135C3D.30807@colorstudy.com> <4313C374.2040605@colorstudy.com>
Message-ID: <3f085ecd0508291949688462aa@mail.gmail.com>

On 8/29/05, Ian Bicking <ianb at colorstudy.com> wrote:
> I expect I'll still want a entry point for all these that acts like
> distutils.commands, but under a different name.  Then a command-packages
> managing command (that's hard to parse ;) could use that to list
> available command packages.  The command-packages managing command
> itself would be installed globally.

I have to wonder, though, how many commands is too many? And how many
people will likely install enough packages to have that many commands?

I, personally, was really happy with how easy and transparent the use
of an EntryPoint was for allowing people to use Testido's distutils
command. I find it hard to imagine that there would be a truly
unmanageable set of commands in a typical person's installation.

Kevin

From ianb at colorstudy.com  Tue Aug 30 05:04:16 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 29 Aug 2005 22:04:16 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <3f085ecd0508291949688462aa@mail.gmail.com>
References: <430EA346.8040600@colorstudy.com>	
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>	
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>	
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>	
	<43132BF1.2070207@colorstudy.com>	
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>	
	<3f085ecd0508291041237a86f8@mail.gmail.com>	
	<43135C3D.30807@colorstudy.com> <4313C374.2040605@colorstudy.com>
	<3f085ecd0508291949688462aa@mail.gmail.com>
Message-ID: <4313CCB0.604@colorstudy.com>

Kevin Dangoor wrote:
> On 8/29/05, Ian Bicking <ianb at colorstudy.com> wrote:
> 
>>I expect I'll still want a entry point for all these that acts like
>>distutils.commands, but under a different name.  Then a command-packages
>>managing command (that's hard to parse ;) could use that to list
>>available command packages.  The command-packages managing command
>>itself would be installed globally.
> 
> 
> I have to wonder, though, how many commands is too many? And how many
> people will likely install enough packages to have that many commands?
> 
> I, personally, was really happy with how easy and transparent the use
> of an EntryPoint was for allowing people to use Testido's distutils
> command. I find it hard to imagine that there would be a truly
> unmanageable set of commands in a typical person's installation.

Personally I think there's too many already.  And SQLObject, for 
instance, would add 8.  Plus a constrained number of commands makes a 
package understandable through --help-commands.  If many (most?) of the 
commands are applicable to a package, package maintainers will have to 
keep a separate document just listing the relevent commands.

The other option might be to allow commands to opt out of being 
listed/available.  For instance, if build_clib noticed there were no C 
libraries.

My vision of how it would work with command-packages would be like:

   python setup.py plugins --list
   # get a list of installed and uninstalled plugins, using entry points
   # to discover commands
   python setup.py plugins --remove=EggName
   python setup.py plugins --add=EggName
   # these change setup.cfg

I'm not sure whether you'd specify eggs, eggs+entry points, modules, or 
what.  Or any of the above.  I still have to remind myself how the 
underlying stuff works.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From pje at telecommunity.com  Tue Aug 30 05:56:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 29 Aug 2005 23:56:17 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <4313CCB0.604@colorstudy.com>
References: <3f085ecd0508291949688462aa@mail.gmail.com>
	<430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<43132BF1.2070207@colorstudy.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<3f085ecd0508291041237a86f8@mail.gmail.com>
	<43135C3D.30807@colorstudy.com> <4313C374.2040605@colorstudy.com>
	<3f085ecd0508291949688462aa@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050829233217.01b20210@mail.telecommunity.com>

At 10:04 PM 8/29/2005 -0500, Ian Bicking wrote:
>My vision of how it would work with command-packages would be like:
>
>    python setup.py plugins --list
>    # get a list of installed and uninstalled plugins, using entry points
>    # to discover commands
>    python setup.py plugins --remove=EggName
>    python setup.py plugins --add=EggName
>    # these change setup.cfg
>
>I'm not sure whether you'd specify eggs, eggs+entry points, modules, or
>what.  Or any of the above.  I still have to remind myself how the
>underlying stuff works.

I'm not clear on what you're trying to do exactly.  However, keep in mind 
that distutils commands running from setup.py aren't the only way to skin 
this cat.  You can always define entry points in a different group than 
'distutils.commands'.

For example, when I create the 'nest' script, I'll probably have a 
'nest.commands' entry point group that will be searched for commands.  Such 
"non-setup" scripts can always use setuptools.sandbox to run a setup script 
and thus get access to setup metadata if they need to.  So, if you need 
some kind of "dbadmin" tool for SQLObject, you can always have it use its 
own command entry points stuff, and then invoke setup.py with e.g. the 
--command-packages option.

Conversely, if you want to enable certain setup.py commands based on the 
availability of specific setup() metadata, there's already a facility for 
that.  Register an argument validator in 'distutils.setup_keywords', and if 
the metadata you want is present, it will be called.  Your hook can then do:

     dist.cmdclass.setdefault('cmdname',command_class)

to register the additional command(s).  Argument hooks are called while the 
distribution object is being created, so this will be before the distutils 
has done anything with the commands.

Note that you can't use this technique to *remove* commands that already 
exist in the distutils or setuptools, only to optionally-add them.

Anyway, for commands that depend on non-standard setup metadata, this is an 
ideal way to hook them in, since you'll need a distutils.setup_keywords 
entry point to do argument validation anyway.  So if the custom metadata 
passes validation, just register the additional command class(es).

Thus, if you have setup() arguments for say, a DB schema, then you could 
automatically enable dbcreate/dbdrop commands for the setup script in that 
case.  If the script doesn't supply the keywords for schema information, 
your keyword entry point won't get imported, and so the commands won't get 
added to the list for the setup script.


From dangoor at gmail.com  Tue Aug 30 13:07:41 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 30 Aug 2005 07:07:41 -0400
Subject: [Distutils] listening for EntryPoints
Message-ID: <3f085ecd05083004076cdac50b@mail.gmail.com>

I'm planning to use the EntryPoint facility for plugins in my
application, and I was just thinking about what it would take to load
a new plugin. My best guess is that, right now, it would take a
restart... which is not bad, but not ideal either.

In order to dynamically add a plugin at runtime, it seems like I would need

1) a way to get a new egg loaded
2) a notification to go out to the parts of the code that care about
the entry points for a particular name or group.

Does anyone else care about being able to add plugins without a
restart, or is it just me?

Kevin

From pje at telecommunity.com  Tue Aug 30 17:15:55 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 30 Aug 2005 11:15:55 -0400
Subject: [Distutils] listening for EntryPoints
In-Reply-To: <3f085ecd05083004076cdac50b@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050830111328.01b2f4c0@mail.telecommunity.com>

At 07:07 AM 8/30/2005 -0400, Kevin Dangoor wrote:
>I'm planning to use the EntryPoint facility for plugins in my
>application, and I was just thinking about what it would take to load
>a new plugin. My best guess is that, right now, it would take a
>restart... which is not bad, but not ideal either.
>
>In order to dynamically add a plugin at runtime, it seems like I would need
>
>1) a way to get a new egg loaded

require(), or working_set.add()


>2) a notification to go out to the parts of the code that care about
>the entry points for a particular name or group.

working_set.subscribe() adds a callback that will be invoked with a 
Distribution object whenever one is added to the working set.  You can then 
call the distribution object's entry point introspection methods.  Note 
that the function will be called for all distributions, including those 
that are already in the working set when you call subscribe(), so you 
shouldn't iterate over the existing entry points if you're going to use 
subscribe.  Just subscribe, and your callback will get invoked for 
everything, as if it had been there before anything was added.


>Does anyone else care about being able to add plugins without a
>restart, or is it just me?

Adding plugins without a restart isn't a problem.  It's removing or 
upgrading them that's not practical.


From dangoor at gmail.com  Tue Aug 30 17:57:05 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Tue, 30 Aug 2005 11:57:05 -0400
Subject: [Distutils] listening for EntryPoints
In-Reply-To: <5.1.1.6.0.20050830111328.01b2f4c0@mail.telecommunity.com>
References: <3f085ecd05083004076cdac50b@mail.gmail.com>
	<5.1.1.6.0.20050830111328.01b2f4c0@mail.telecommunity.com>
Message-ID: <3f085ecd050830085727784ac2@mail.gmail.com>

On 8/30/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 07:07 AM 8/30/2005 -0400, Kevin Dangoor wrote:
> >I'm planning to use the EntryPoint facility for plugins in my
> >application, and I was just thinking about what it would take to load
> >a new plugin. My best guess is that, right now, it would take a
> >restart... which is not bad, but not ideal either.
> >
> >In order to dynamically add a plugin at runtime, it seems like I would need
> >
> >1) a way to get a new egg loaded
> 
> require(), or working_set.add()
> 
> 
> >2) a notification to go out to the parts of the code that care about
> >the entry points for a particular name or group.
> 
> working_set.subscribe() adds a callback that will be invoked with a
> Distribution object whenever one is added to the working set. 

Thanks, Phillip! I'm not surprised to hear that you've already worked this out!

> >Does anyone else care about being able to add plugins without a
> >restart, or is it just me?
> 
> Adding plugins without a restart isn't a problem.  It's removing or
> upgrading them that's not practical.

Yeah, I've been using Python long enough to know about the intracacies
of trying to reload something. That's a less critical use case than
being able to add something seamlessly.

Kevin

From ianb at colorstudy.com  Tue Aug 30 19:22:46 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 30 Aug 2005 12:22:46 -0500
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <5.1.1.6.0.20050829233217.01b20210@mail.telecommunity.com>
References: <3f085ecd0508291949688462aa@mail.gmail.com>
	<430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<43132BF1.2070207@colorstudy.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<3f085ecd0508291041237a86f8@mail.gmail.com>
	<43135C3D.30807@colorstudy.com> <4313C374.2040605@colorstudy.com>
	<3f085ecd0508291949688462aa@mail.gmail.com>
	<5.1.1.6.0.20050829233217.01b20210@mail.telecommunity.com>
Message-ID: <431495E6.8070609@colorstudy.com>

Phillip J. Eby wrote:
> At 10:04 PM 8/29/2005 -0500, Ian Bicking wrote:
> 
>> My vision of how it would work with command-packages would be like:
>>
>>    python setup.py plugins --list
>>    # get a list of installed and uninstalled plugins, using entry points
>>    # to discover commands
>>    python setup.py plugins --remove=EggName
>>    python setup.py plugins --add=EggName
>>    # these change setup.cfg
>>
>> I'm not sure whether you'd specify eggs, eggs+entry points, modules, or
>> what.  Or any of the above.  I still have to remind myself how the
>> underlying stuff works.
> 
> 
> I'm not clear on what you're trying to do exactly.  However, keep in 
> mind that distutils commands running from setup.py aren't the only way 
> to skin this cat.  You can always define entry points in a different 
> group than 'distutils.commands'.
> 
> For example, when I create the 'nest' script, I'll probably have a 
> 'nest.commands' entry point group that will be searched for commands.  
> Such "non-setup" scripts can always use setuptools.sandbox to run a 
> setup script and thus get access to setup metadata if they need to.  So, 
> if you need some kind of "dbadmin" tool for SQLObject, you can always 
> have it use its own command entry points stuff, and then invoke setup.py 
> with e.g. the --command-packages option.

Indeed, this was my original plan; well, SQLObject had a sqlobject-admin 
script, and I was creating a paster script that would be more general 
and driven by entry points.

However, in an effort to Reduce The Total Number Of Things, I was 
thinking about moving these all into setup.py.  I would be okay with 
just one other script, not pushing everything into setup.py, but in 
practice I suspect that only I would use that one other script, and 
every project would have their own.

.onversely, if you want to enable certain setup.py commands based on the
> availability of specific setup() metadata, there's already a facility 
> for that.  Register an argument validator in 'distutils.setup_keywords', 
> and if the metadata you want is present, it will be called.  Your hook 
> can then do:
> 
>     dist.cmdclass.setdefault('cmdname',command_class)
> 
> to register the additional command(s).  Argument hooks are called while 
> the distribution object is being created, so this will be before the 
> distutils has done anything with the commands.
> 
> Note that you can't use this technique to *remove* commands that already 
> exist in the distutils or setuptools, only to optionally-add them.
> 
> Anyway, for commands that depend on non-standard setup metadata, this is 
> an ideal way to hook them in, since you'll need a 
> distutils.setup_keywords entry point to do argument validation anyway.  
> So if the custom metadata passes validation, just register the 
> additional command class(es).

Interesting, I'll have to try that out.  So I'm thinking I might be able 
to have someone add this to setup.cfg:

   [sqlobject]
   dbmodule = mypackage.db

And use that to kind of implicitly enable the SQLObject commands.  At 
least, assuming I can access arbitrary sections of the config from the 
setup (since sqlobject would imply several commands).  Or alterately in 
setup.py.  Hrm... maybe it has to be in setup.py, though, since 
setup.cfg is only used after the command is invoked, so it can't keep 
the command from being available?  Well, I suppose experimentation is in 
order.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Tue Aug 30 19:52:56 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 30 Aug 2005 13:52:56 -0400
Subject: [Distutils] svn tagging setuptools command
In-Reply-To: <431495E6.8070609@colorstudy.com>
References: <5.1.1.6.0.20050829233217.01b20210@mail.telecommunity.com>
	<3f085ecd0508291949688462aa@mail.gmail.com>
	<430EA346.8040600@colorstudy.com>
	<5.1.1.6.0.20050828175958.01b9f488@mail.telecommunity.com>
	<5.1.1.6.0.20050828194105.01b2e880@mail.telecommunity.com>
	<5.1.1.6.0.20050829024201.01b2e5e0@mail.telecommunity.com>
	<43132BF1.2070207@colorstudy.com>
	<5.1.1.6.0.20050829114614.01b31e30@mail.telecommunity.com>
	<3f085ecd0508291041237a86f8@mail.gmail.com>
	<43135C3D.30807@colorstudy.com> <4313C374.2040605@colorstudy.com>
	<3f085ecd0508291949688462aa@mail.gmail.com>
	<5.1.1.6.0.20050829233217.01b20210@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050830134621.01b22210@mail.telecommunity.com>

At 12:22 PM 8/30/2005 -0500, Ian Bicking wrote:
>Interesting, I'll have to try that out.  So I'm thinking I might be able 
>to have someone add this to setup.cfg:
>
>   [sqlobject]
>   dbmodule = mypackage.db
>
>And use that to kind of implicitly enable the SQLObject commands.  At 
>least, assuming I can access arbitrary sections of the config from the 
>setup (since sqlobject would imply several commands).  Or alterately in 
>setup.py.  Hrm... maybe it has to be in setup.py, though, since setup.cfg 
>is only used after the command is invoked, so it can't keep the command 
>from being available?  Well, I suppose experimentation is in order.

setup.cfg is loaded and parsed before the command line is, but *after* 
argument processing.  So there isn't really a way to hook into any of that, 
short of loading the config file yourself from a keyword argument 
processor.  It'd be better for you to define a setup() keyword like 
'sqlobject_dbmodule'.  See the CVS version of setuptools.txt under "Adding 
``setup()`` Arguments" for detailed info.

The setup() would also have to list 'setup_requires="SQLObject>=0.7b1"' or 
whatever in order to ensure that the keyword argument is recognized, 
though.  So, the setup might look like:

     setup(
         ...
         setup_requires=["SQLObject>=0.7b1"],
         sqlobject_dbmodule = "mypackage.db",
     )

And there you go.  Your "sqlobject_dbmodule" entry point in the 
"distutils.setup_keywords" group will then define a validation function to 
check that sqlobject_dbmodule has the right type/syntax, and then insert 
any useful command classes into the dist.cmdclass dictionary, making them 
available before command line parsing takes place.


From dangoor at gmail.com  Wed Aug 31 21:09:22 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Wed, 31 Aug 2005 15:09:22 -0400
Subject: [Distutils] announce not working for me
Message-ID: <3f085ecd050831120967130021@mail.gmail.com>

I'm doing something wrong, and it's probably very obvious...

from distutils.core import setup

from distutils.core import Command

class FooBar(Command):
    description = "foo. no! Bar!"
    user_options = []
    def initialize_options(self):
        pass

    def finalize_options(self):
        pass

    def run(self):
        print "am I here?"
        self.announce("Run, I say!")

setup(
    cmdclass = { "foobar" : FooBar }
)

Running "python setup.py foobar" prints the "am I here?" line, but not
the "Run, I say!" line. This is true even with the -v flag.

*Should* this work?

Kevin

From pje at telecommunity.com  Wed Aug 31 21:16:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 31 Aug 2005 15:16:35 -0400
Subject: [Distutils] announce not working for me
In-Reply-To: <3f085ecd050831120967130021@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050831151556.01b43788@mail.telecommunity.com>

At 03:09 PM 8/31/2005 -0400, Kevin Dangoor wrote:
>I'm doing something wrong, and it's probably very obvious...
>
>from distutils.core import setup
>
>from distutils.core import Command
>
>class FooBar(Command):
>     description = "foo. no! Bar!"
>     user_options = []
>     def initialize_options(self):
>         pass
>
>     def finalize_options(self):
>         pass
>
>     def run(self):
>         print "am I here?"
>         self.announce("Run, I say!")
>
>setup(
>     cmdclass = { "foobar" : FooBar }
>)
>
>Running "python setup.py foobar" prints the "am I here?" line, but not
>the "Run, I say!" line. This is true even with the -v flag.
>
>*Should* this work?

Dunno.  But you should probably be using distutils.log.info() (or 
debug/warn/etc.) instead anyway.


From dangoor at gmail.com  Wed Aug 31 21:32:59 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Wed, 31 Aug 2005 15:32:59 -0400
Subject: [Distutils] announce not working for me
In-Reply-To: <5.1.1.6.0.20050831151556.01b43788@mail.telecommunity.com>
References: <3f085ecd050831120967130021@mail.gmail.com>
	<5.1.1.6.0.20050831151556.01b43788@mail.telecommunity.com>
Message-ID: <3f085ecd05083112327cd780d7@mail.gmail.com>

On 8/31/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
> Dunno.  But you should probably be using distutils.log.info() (or
> debug/warn/etc.) instead anyway.

The funny thing is that I started using self.announce when I saw it in
setuptools/command/test.py :)

distutils.log.info works fine.

Thanks!
Kevin