From pje at telecommunity.com  Thu Jun  1 17:56:08 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 01 Jun 2006 11:56:08 -0400
Subject: [Distutils] setuptools 0.6b2 released
Message-ID: <5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>

This is a bugfix release to fix a few issues with 0.6b1.  Please upgrade 
and test if you can, I'd like to release an 0.6 final soon.  There are 
still a couple of known minor issues that I haven't addressed yet (error 
messages from ez_setup/easy_isntall reported here by Nathan Yergler), but I 
wanted to get this intermediate release out because of one serious bug that 
affects you if you install a package whose name begins with the word 
"import" on Windows -- e.g., my own "Importing" package, published recently 
on PyPI.

Anyway, the next release of 0.6 should be a release candidate, so please 
beat on this one hard to find any remaining bugs.  I'd like to get started 
on 0.7 (and the "nest" tool suite) as soon as practical.

(By the way: setuptools 0.6 is probably not going in the 2.5 stdlib; after 
lots of discussion on Python-Dev, it's fairly clear that some 0.7 release 
should be the earliest version to go in, once the "nest" tools are 
done.  Practically speaking, that probably means waiting until Python 2.6, 
due to the lead time that would be needed to get adequate field testing of 
the "nest" tools once they're written.)


From dangoor at gmail.com  Thu Jun  1 20:16:54 2006
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 1 Jun 2006 14:16:54 -0400
Subject: [Distutils] setuptools 0.6b2 released
In-Reply-To: <5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>
References: <5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>
Message-ID: <3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>

On 6/1/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> (By the way: setuptools 0.6 is probably not going in the 2.5 stdlib; after
> lots of discussion on Python-Dev, it's fairly clear that some 0.7 release
> should be the earliest version to go in, once the "nest" tools are
> done.  Practically speaking, that probably means waiting until Python 2.6,
> due to the lead time that would be needed to get adequate field testing of
> the "nest" tools once they're written.)

That's unfortunate, though I can certainly understand the rationale.
Based on feedback I've seen on the TG list, ability to easily manage
eggs is a fairly common desire.

Kevin

From jjl at pobox.com  Thu Jun  1 22:16:37 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 1 Jun 2006 20:16:37 +0000 (UTC)
Subject: [Distutils] setuptools 0.6b2 released
In-Reply-To: <3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>
References: <5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>
	<3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0606012012170.8440@localhost>

On Thu, 1 Jun 2006, Kevin Dangoor wrote:

> On 6/1/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>> (By the way: setuptools 0.6 is probably not going in the 2.5 stdlib; after
>> lots of discussion on Python-Dev, it's fairly clear that some 0.7 release
>> should be the earliest version to go in, once the "nest" tools are
>> done.  Practically speaking, that probably means waiting until Python 2.6,
>> due to the lead time that would be needed to get adequate field testing of
>> the "nest" tools once they're written.)
>
> That's unfortunate, though I can certainly understand the rationale.
> Based on feedback I've seen on the TG list, ability to easily manage
> eggs is a fairly common desire.

Heh -- that may have been part of the rationale, but it wasn't the cause. 
Several core Pythonistas objected in strong terms on python-dev to various 
aspects of it.  Still, Guido clearly wants it, so seems pretty sure thing 
for 2.6 still.

No doubt PJE made the decision to pull it for a combination of reasons, 
though.

Best of luck with TG


John


From jjl at pobox.com  Thu Jun  1 22:22:32 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 1 Jun 2006 20:22:32 +0000 (UTC)
Subject: [Distutils] setuptools 0.6b2 released
In-Reply-To: <Pine.LNX.4.64.0606012012170.8440@localhost>
References: <5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>
	<3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>
	<Pine.LNX.4.64.0606012012170.8440@localhost>
Message-ID: <Pine.LNX.4.64.0606012020490.8440@localhost>

[...]
> Heh -- that may have been part of the rationale, but it wasn't the cause.
> Several core Pythonistas objected in strong terms on python-dev to various
[...]

Oh sugar, didn't mean to post that, and risk stiring this up again, yawn 
(I only meant to email it to KD).

Please ignore


John


From pje at telecommunity.com  Thu Jun  1 22:36:30 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 01 Jun 2006 16:36:30 -0400
Subject: [Distutils] setuptools 0.6b2 released
In-Reply-To: <Pine.LNX.4.64.0606012012170.8440@localhost>
References: <3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>
	<5.1.1.6.0.20060601114636.01f3bfa8@mail.telecommunity.com>
	<3f085ecd0606011116g35bd9141w6d253c895fa2a7b@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060601162954.01f65f40@mail.telecommunity.com>

At 08:16 PM 6/1/2006 +0000, John J Lee wrote:
>Heh -- that may have been part of the rationale, but it wasn't the cause.
>Several core Pythonistas objected in strong terms on python-dev to various
>aspects of it.

Actually, three people objected, and one of them actually had some good 
points that led me to pull back.  Fredrik raised the issue of support, 
especially with respect to the sometimes confusing and baroque command-line 
interface to easy_install.  I agree that it's not reasonable to expect the 
core Python team to support all its intricacies at the present time.


>   Still, Guido clearly wants it,

It was only on the table because of Guido's request in the first place; in 
a private email I originally told him I wasn't sure about doing an 0.6 
unless it could be upgraded on-the-fly (which it can, now) but I wasn't 
100% sure anyway.  The support issues, the short timeline, and lack of 
volunteers basically sealed its fate for 2.5, though.


From jim at zope.com  Sun Jun  4 20:13:29 2006
From: jim at zope.com (Jim Fulton)
Date: Sun, 4 Jun 2006 14:13:29 -0400
Subject: [Distutils] Bug in easy_install?
Message-ID: <07E91BC7-42BF-41A8-8174-B228AB547CCD@zope.com>


If I give easy_install

- an ordinary spec (e.g. "demo")

- an -f option that points to a local directory containing a bunch of
   eggs and that is not on sys,path,

- ask it to install to some directory (-d), and

- use the -m option

it doesn't copy eggs found to the install directory even though
the source directory isn't on sys.path.   The logic for deciding whether
to copy seems to be:

         # Installation is also needed if file in tmpdir or is not an  
egg
         install_needed = install_needed or os.path.dirname(download)  
== tmpdir
         install_needed = install_needed or not download.endswith 
('.egg')

where install_needed starts out as false in this case because the
spec is not a url.  The second part has no effect because the
value if the download variable ends in .egg.

The --always-copy documentation hints that distributions will be
copied unless they are found in a directory sys.path, but I see no
check.

Why are the eggs not copied?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From ronaldoussoren at mac.com  Tue Jun  6 15:46:07 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 6 Jun 2006 15:46:07 +0200
Subject: [Distutils] setuptools & python2.5
Message-ID: <249455FA-0FE5-4D69-B970-1D9792CD63F8@mac.com>

Hi,

I tried to install setuptools 0.6b2 in my python2.5 installation, but  
that doesn't seem to work. The ez_setup.py installation method  
doesn't work because there is no egg for python2.5, therefore I  
downloaded the sources and ran 'python2.5 setup.py install'.

This results in this following output at the bottom of this message.  
It looks like python2.5 & setuptools don't mix very well at the moment.

Ronald


setuptools-0.6b2 ronald$ python2.5 setup.py install
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ 
distutils/dist.py:263: UserWarning: Unknown distribution option:  
'entry_points'
   warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ 
distutils/dist.py:263: UserWarning: Unknown distribution option:  
'zip_safe'
   warnings.warn(msg)
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ 
distutils/dist.py:263: UserWarning: Unknown distribution option:  
'test_suite'
   warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib
copying pkg_resources.py -> build/lib
copying easy_install.py -> build/lib
copying site.py -> build/lib
creating build/lib/setuptools
copying setuptools/__init__.py -> build/lib/setuptools
copying setuptools/archive_util.py -> build/lib/setuptools
copying setuptools/depends.py -> build/lib/setuptools
copying setuptools/dist.py -> build/lib/setuptools
copying setuptools/extension.py -> build/lib/setuptools
copying setuptools/package_index.py -> build/lib/setuptools
copying setuptools/sandbox.py -> build/lib/setuptools
creating build/lib/setuptools/command
copying setuptools/command/__init__.py -> build/lib/setuptools/command
copying setuptools/command/alias.py -> build/lib/setuptools/command
copying setuptools/command/bdist_egg.py -> build/lib/setuptools/command
copying setuptools/command/bdist_rpm.py -> build/lib/setuptools/command
copying setuptools/command/build_ext.py -> build/lib/setuptools/command
copying setuptools/command/build_py.py -> build/lib/setuptools/command
copying setuptools/command/develop.py -> build/lib/setuptools/command
copying setuptools/command/easy_install.py -> build/lib/setuptools/ 
command
copying setuptools/command/egg_info.py -> build/lib/setuptools/command
copying setuptools/command/install.py -> build/lib/setuptools/command
copying setuptools/command/install_egg_info.py -> build/lib/ 
setuptools/command
copying setuptools/command/install_lib.py -> build/lib/setuptools/ 
command
copying setuptools/command/install_scripts.py -> build/lib/setuptools/ 
command
copying setuptools/command/rotate.py -> build/lib/setuptools/command
copying setuptools/command/saveopts.py -> build/lib/setuptools/command
copying setuptools/command/sdist.py -> build/lib/setuptools/command
copying setuptools/command/setopt.py -> build/lib/setuptools/command
copying setuptools/command/test.py -> build/lib/setuptools/command
copying setuptools/command/upload.py -> build/lib/setuptools/command
creating build/lib/setuptools/tests
copying setuptools/tests/__init__.py -> build/lib/setuptools/tests
copying setuptools/tests/doctest.py -> build/lib/setuptools/tests
copying setuptools/tests/test_resources.py -> build/lib/setuptools/tests
copying setuptools/cli.exe -> build/lib/setuptools
copying setuptools/gui.exe -> build/lib/setuptools
running build_scripts
creating build/scripts-2.5
copying and adjusting easy_install.py -> build/scripts-2.5
changing mode of build/scripts-2.5/easy_install.py from 644 to 755
running install_lib
running install_scripts
copying build/scripts-2.5/easy_install.py -> /Library/Frameworks/ 
Python.framework/Versions/2.5/bin
changing mode of /Library/Frameworks/Python.framework/Versions/2.5/ 
bin/easy_install.py to 755
running install_egg_info
Removing /Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/site-packages/setuptools-0.6b2-py2.5.egg-info
Writing /Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/site-packages/setuptools-0.6b2-py2.5.egg-info



From pje at telecommunity.com  Tue Jun  6 18:00:29 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 06 Jun 2006 12:00:29 -0400
Subject: [Distutils] setuptools & python2.5
In-Reply-To: <249455FA-0FE5-4D69-B970-1D9792CD63F8@mac.com>
Message-ID: <5.1.1.6.0.20060606115829.04661008@mail.telecommunity.com>

At 03:46 PM 6/6/2006 +0200, Ronald Oussoren wrote:
>Hi,
>
>I tried to install setuptools 0.6b2 in my python2.5 installation, but
>that doesn't seem to work.

0.6b2 isn't compatible with 2.5.  Try the development version of 0.7a1, 
which has had numerous changes for Python 2.5 support.


From pje at telecommunity.com  Tue Jun  6 18:03:25 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 06 Jun 2006 12:03:25 -0400
Subject: [Distutils] Bug in easy_install?
In-Reply-To: <07E91BC7-42BF-41A8-8174-B228AB547CCD@zope.com>
Message-ID: <5.1.1.6.0.20060606120035.044ea1d0@mail.telecommunity.com>

At 02:13 PM 6/4/2006 -0400, Jim Fulton wrote:

>If I give easy_install
>
>- an ordinary spec (e.g. "demo")
>
>- an -f option that points to a local directory containing a bunch of
>    eggs and that is not on sys,path,
>
>- ask it to install to some directory (-d), and
>
>- use the -m option
>
>it doesn't copy eggs found to the install directory even though
>the source directory isn't on sys.path.   The logic for deciding whether
>to copy seems to be:
>
>          # Installation is also needed if file in tmpdir or is not an
>egg
>          install_needed = install_needed or os.path.dirname(download)
>== tmpdir
>          install_needed = install_needed or not download.endswith
>('.egg')
>
>where install_needed starts out as false in this case because the
>spec is not a url.  The second part has no effect because the
>value if the download variable ends in .egg.
>
>The --always-copy documentation hints that distributions will be
>copied unless they are found in a directory sys.path, but I see no
>check.
>
>Why are the eggs not copied?

Because I'm an idiot, that's why.  :(  That logic predates --find-links 
being able to support local files without using file:// URLs.  At one 
point, it worked correctly because file:// URLs caused easy_install to copy 
them to a temporary directory.  Later, when I made that more efficient, it 
broke the install_needed logic.  :(

Too bad you didn't find this one *before* I released 0.6b2.  Oh well.  :)


From risky.martin at gmail.com  Wed Jun  7 16:59:33 2006
From: risky.martin at gmail.com (Risky Martin)
Date: Wed, 7 Jun 2006 22:59:33 +0800
Subject: [Distutils] unrecognised swig options ?
Message-ID: <186780900606070759m8314ae2vdea0c9373f739fc9@mail.gmail.com>

Hi,


I am trying to use disutils's features with 'SWIG'... but hit
a rather strange problem.

To reproduce it:

setup.py
------------------------------------------------------
from distutils.core import setup, Extension

e = Extension("_foo",
              sources = ["foo.i", ],
              swig_opts = ["-outdir .", ])


setup(name="foo",
      ext_modules=[e, ])
-------------------------------------------------------

building fails like:

> python setup.py build
running build
running build_ext
building '_foo' extension
swigging foo.i to foo_wrap.c
swig -python -outdir . -o foo_wrap.c foo.i
swig error : Unrecognized option -outdir .
Use 'swig -help' for available options.
error: command 'swig' failed with exit status 1

but a copy/paste of the swig command shows that '-outdir' *is* recognised...
> swig -python -outdir . -o foo_wrap.c foo.i
Unable to find file 'foo.i'.

Are the permitted options for SWIG hardcoded in 'distutils' ?



RiMa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20060607/3a876c77/attachment.html 

From uche at ogbuji.net  Wed Jun  7 20:35:32 2006
From: uche at ogbuji.net (Uche Ogbuji)
Date: Wed, 07 Jun 2006 12:35:32 -0600
Subject: [Distutils] Bug in setuptools-0.6a11-py2.4 ?
Message-ID: <44871C74.4090906@ogbuji.net>

I'm trying to port 4Suite and Amara to setuptools.  I thought 4Suite
would be the hard part, but I have it basically functioning now.  No
such luck with Amara, though.

I can't get setuptools to handle the fact that Amara has all its code in
a subdir named lib, such that it used to work fine in plain distutils as
follows:

setup(package_dir = {'amara': 'lib'}, packages = ['amara'],...)

This same invocation with setuptools does not give an error message, but
 no packages are found or processed.

I also tried:

setup(package_dir = {'amara': 'lib'},
      packages = find_packages(),

setup(package_dir = {'amara': 'lib'},
      packages = find_packages('lib'),

setup(package_dir = {'amara': 'lib'},
      packages = find_packages(),
      packages = ['amara'],

and many other desperate attempts, with no luck.

It feels to me like a bug in setuptools with such a layout, but I'm
really surprised, because it's so common, and even covered explicitly by
the distutils and setuptools docs.

If anyone else wants to try you can do so as follows:

cvs -d:pserver:anonymous at cvs.4suite.org:/var/local/cvsroot login
cvs -d:pserver:anonymous at cvs.4suite.org:/var/local/cvsroot get 4Suite -r
EASYINSTALL-branch
cvs -d:pserver:anonymous at cvs.4suite.org:/var/local/cvsroot get Amara

This CVS is also accessible via viewcvs:

http://cvs.4suite.org/viewcvs/

I'd appreciate any help.

One last thing: is there an issue tracker for setuptools?


-- 
Uche Ogbuji                               Fourthought, Inc.
http://uche.ogbuji.net                    http://fourthought.com
http://copia.ogbuji.net                   http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/

From pje at telecommunity.com  Wed Jun  7 21:19:50 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 07 Jun 2006 15:19:50 -0400
Subject: [Distutils] Bug in setuptools-0.6a11-py2.4 ?
In-Reply-To: <44871C74.4090906@ogbuji.net>
Message-ID: <5.1.1.6.0.20060607150716.01e8fe88@mail.telecommunity.com>

At 12:35 PM 6/7/2006 -0600, Uche Ogbuji wrote:
>I can't get setuptools to handle the fact that Amara has all its code in
>a subdir named lib, such that it used to work fine in plain distutils as
>follows:
>
>setup(package_dir = {'amara': 'lib'}, packages = ['amara'],...)
>
>This same invocation with setuptools does not give an error message, but
>  no packages are found or processed.
>
>I also tried:
>
>setup(package_dir = {'amara': 'lib'},
>       packages = find_packages(),
>
>setup(package_dir = {'amara': 'lib'},
>       packages = find_packages('lib'),
>
>setup(package_dir = {'amara': 'lib'},
>       packages = find_packages(),
>       packages = ['amara'],
>
>and many other desperate attempts, with no luck.
>
>It feels to me like a bug in setuptools with such a layout, but I'm
>really surprised, because it's so common,

Actually, it's pretty *un*common, and it's not a good layout to use for 
many reasons, including non-importability during development, user 
confusion in understanding the layout, and inconvenience (i.e., a more 
natural layout wouldn't require setting package_dir at all).

Merits of the layout aside, it should nonetheless work properly with 
setuptools, and I actually didn't have any problems with it.  (see further 
below)


>  and even covered explicitly by
>the distutils and setuptools docs.

Actually, setuptools only mentions creating a *parent* package, e.g. 
package_dir={'':'lib'}, where the 'amara' package directory would then be a 
subdirectory of lib.

Again, however, it *should* work, and as far as I can tell, it does.


>cvs -d:pserver:anonymous at cvs.4suite.org:/var/local/cvsroot get Amara

I checked this out, and it builds an egg just fine for me under both 2.3 
and 2.4.  What problem are you having, exactly?


>One last thing: is there an issue tracker for setuptools?

You're speaking to him now.  :)


From uche at ogbuji.net  Thu Jun  8 00:31:56 2006
From: uche at ogbuji.net (Uche Ogbuji)
Date: Wed, 07 Jun 2006 16:31:56 -0600
Subject: [Distutils] Bug in setuptools-0.6a11-py2.4 ?
In-Reply-To: <5.1.1.6.0.20060607150716.01e8fe88@mail.telecommunity.com>
References: <5.1.1.6.0.20060607150716.01e8fe88@mail.telecommunity.com>
Message-ID: <448753DC.708@ogbuji.net>

Phillip J. Eby wrote:
> At 12:35 PM 6/7/2006 -0600, Uche Ogbuji wrote:
>> I can't get setuptools to handle the fact that Amara has all its code in
>> a subdir named lib, such that it used to work fine in plain distutils as
>> follows:
>>
>> setup(package_dir = {'amara': 'lib'}, packages = ['amara'],...)
>>
>> This same invocation with setuptools does not give an error message, but
>>  no packages are found or processed.
>>
>> I also tried:
>>
>> setup(package_dir = {'amara': 'lib'},
>>       packages = find_packages(),
>>
>> setup(package_dir = {'amara': 'lib'},
>>       packages = find_packages('lib'),
>>
>> setup(package_dir = {'amara': 'lib'},
>>       packages = find_packages(),
>>       packages = ['amara'],
>>
>> and many other desperate attempts, with no luck.
>>
>> It feels to me like a bug in setuptools with such a layout, but I'm
>> really surprised, because it's so common,

I think this issue report turned out to be a bogon, as I'll elaborate
below, so I hate to launch into a debate that should never have been
inspired in the first place, but some of the things you said puzzle me.

> Actually, it's pretty *un*common, and it's not a good layout to use for
> many reasons,

Based on the projects I see in my Python src directory, it is indeed
common.  Sometime the code dir is called "lib", sometimes "src", and
occasionally something else.  I guess we just seem to use a disjoint set
of Python packages.

> including non-importability during development,

I don't understand why that's bad.  setuptools seems to go out of its
way to make it so you can do this.  That's fine with me, but I don't see
it as an important feature by any means.

> user confusion in understanding the layout,

Again I don't see your point.  Why do you think a user would be
confused?  This is the first I've heard such a claim in a long time
managing project layouts.

Personally, I don't have an objection to any layout as long as there's a
README to explain it.

> and inconvenience (i.e., a more
> natural layout wouldn't require setting package_dir at all).

Why is adding one dict to the setup call an inconvenience?  As I said,
it's mentioned right in the distutils manual.

> Merits of the layout aside, it should nonetheless work properly with
> setuptools, and I actually didn't have any problems with it.  (see
> further below)
> 
> 
>>  and even covered explicitly by
>> the distutils and setuptools docs.
> 
> Actually, setuptools only mentions creating a *parent* package, e.g.
> package_dir={'':'lib'}, where the 'amara' package directory would then
> be a subdirectory of lib.

That's true.  But to me the fact that it's a prominent pattern in base
distutils docs is enough, especially since setuptools bills itself as a
drop-in replacement.

> Again, however, it *should* work, and as far as I can tell, it does.

And here I think you have me.  I was working within my CVS local
repository, where I do all my development play.  After your success
report, I checked out clean versions of everything to a new directory,
and all went well.  I can only imagine that something in my previous
working directories was confusing things.

I should have done this before posting to the list, but I thought I'd
received corroboration of the problem from someone who would have
checked out a fresh repository already.  Sorry for all the noise, and
thanks for trying things out.

I'm just happy that it looks as if I'll be able to provide
setuptools-enabled versions of 4Suite and Amara soonish.


-- 
Uche Ogbuji                               Fourthought, Inc.
http://uche.ogbuji.net                    http://fourthought.com
http://copia.ogbuji.net                   http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/

From pje at telecommunity.com  Thu Jun  8 03:31:18 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 07 Jun 2006 21:31:18 -0400
Subject: [Distutils] Bug in setuptools-0.6a11-py2.4 ?
In-Reply-To: <448753DC.708@ogbuji.net>
References: <5.1.1.6.0.20060607150716.01e8fe88@mail.telecommunity.com>
	<5.1.1.6.0.20060607150716.01e8fe88@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20060607211739.0307e420@mail.telecommunity.com>

At 04:31 PM 6/7/2006 -0600, Uche Ogbuji wrote:
>Based on the projects I see in my Python src directory, it is indeed
>common.  Sometime the code dir is called "lib", sometimes "src", and
>occasionally something else.  I guess we just seem to use a disjoint set
>of Python packages.

But that's not the same thing at all.  The common layout in such a case is 
to have

     package_dir = {'':'src'}  # or 'lib'

and that is quite a different layout from what you're doing.

I would be very curious to see what packages are doing as you are; at the 
moment I know of one that actually uses a {'foo':''} convention (which has 
similar problems), and I think I've seen one other besides yours that uses 
{'foo':'lib'}.

But to my knowledge, the majority of 'src' and 'lib' dirs out there are 
following the {'':'src'} or {'':'lib'} convention in setup.py, not the 
{'foo':'lib'} one.


> > user confusion in understanding the layout,
>
>Again I don't see your point.  Why do you think a user would be
>confused?  This is the first I've heard such a claim in a long time
>managing project layouts.

Because there is no directory with the package's name, so simple browsing 
of the directory layout will not tell you what packages are in it.  The 
src/ or lib/ prefix isn't the obstacle, it's the absence of disambiguating 
information.


> > Actually, setuptools only mentions creating a *parent* package, e.g.
> > package_dir={'':'lib'}, where the 'amara' package directory would then
> > be a subdirectory of lib.
>
>That's true.  But to me the fact that it's a prominent pattern in base
>distutils docs

I'd dispute that, as the distutils docs themselves *literally* describe the 
layout with packages in the setup.py directory as being the "most 
obvious".  See:

    http://python.org/doc/2.4.1/dist/listing-packages.html

The {'':'lib'} pattern is listed as "a different convention", and the 
{'foo':'lib'} option is listed last, as "another possible convention" -- 
hardly a stellar recommendation.  :)

Setuptools supports it, of course, because there *are* packages in the wild 
that use it.  Nonetheless, I won't pass up an opportunity to encourage 
people to use the "most obvious" approach instead.  :)  I'm personally 
migrating away from the habit of using the {'':'src'} convention myself.


>I'm just happy that it looks as if I'll be able to provide
>setuptools-enabled versions of 4Suite and Amara soonish.

I'm glad it's working for you.  Please don't let any of this discourage you 
from reporting bugs; I'm just as happy when a reported bug can be 
considered fixed without actually making any code changes.  ;-)


From pje at telecommunity.com  Fri Jun  9 20:00:35 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 09 Jun 2006 14:00:35 -0400
Subject: [Distutils] error: None
In-Reply-To: <4471DFE1.6060602@yergler.net>
Message-ID: <5.1.1.6.0.20060609135559.032aed90@mail.telecommunity.com>

At 11:59 AM 5/22/2006 -0400, Nathan R. Yergler wrote:
>Thanks to Phillip for the insight into the interaction between
>command-line arguments and options.  So another question regarding error
>reporting.  When I so something like:
>
>$ PYTHONPATH=lib/ python ez_setup.py --install-dir lib/ --script-dir
>bin/ --site-dirs lib/ zope.testing
>
>setuptools is bootstrapped correctly, and zope.testing and its
>dependencies are downloaded and installed.  However, at the end of the
>run you get a message like:
>
>Installed /home/nathan/Projects/zope.i18n/lib/setuptools-0.6b1-py2.4.egg
>Processing dependencies for setuptools==0.6b1
>error: None
>
>Does anyone have any insights into this?

Okay, I finally tracked this down.  It's actually the same error as the:

"error: No urls, filenames, or requirements specified (see --help)"

message you were getting before.  Or rather, they're both caused by a 
freaky code fallthrough bug in ez_setup.py.

If, and only if, you are installing setuptools when it is not available on 
the local system, ez_setup tries to run easy_install *twice* with more or 
less the same arguments.  If you do *not* have any non-option arguments, 
you will get the first message.  If you *do*, then easy_install gets an 
IOError because it tries to read from the temporarily-downloaded 
setuptools.egg that has already been deleted.  The distutils sees this 
IOError and unhelpfully (not to mention idiotically) outputs "error: None" 
and aborts!

So I've fixed the source of both problems in the SVN trunk, and the 0.6b3 
release (probably today) will have an ez_setup.py with the fix.


From pje at telecommunity.com  Fri Jun  9 21:08:56 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 09 Jun 2006 15:08:56 -0400
Subject: [Distutils] setuptools 0.6b3 released
Message-ID: <5.1.1.6.0.20060609143635.0339b1f0@mail.telecommunity.com>

This is a bugfix release, to fix problems found by Jim Fulton, Markku 
Mielityinen, Nathan Yergler, and others in 0.6b2 or earlier versions.


From nad at acm.org  Sat Jun 10 02:22:15 2006
From: nad at acm.org (Ned Deily)
Date: Fri, 09 Jun 2006 17:22:15 -0700
Subject: [Distutils] setuptools: mishandles install of non-python scripts
Message-ID: <nad-332EEB.17221509062006@sea.gmane.org>

easy_install doesn't properly handle non-python scripts in a 
distribution.

The problem shows up when using easy_install to install mercurial:
   <http://cheeseshop.python.org/pypi/Mercurial/>

The mercurial distribution includes two scripts, one a python script, 
the other a shell script:
hg - #!/usr/bin/env python
hgmerge - #!/bin/sh

Using easy_install, both scripts get installed with a python wrapper 
script. Execution of hgmerge fails when the wrapper tries to run the 
shell script under python:

$ hgmerge
Traceback (most recent call last):
  File "/usr/local/python/bin/hgmerge", line 5, in ?
    pkg_resources.run_script('mercurial==dbeaa4369121', 'hgmerge')
  File 
"/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/setuptools-0.6b3-py2.4.egg/pkg_resources.py", line 407, in 
run_script
    self.require(requires)[0].run_script(script_name, ns)
  File 
"/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/setuptools-0.6b3-py2.4.egg/pkg_resources.py", line 1084, in 
run_script
    execfile(script_filename, namespace, namespace)
  File 
"/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-pac
kages/mercurial-dbeaa4369121-py2.4-macosx-10.4-fat.egg/EGG-INFO/scripts/h
gmerge", line 20
    if [ -z "$EDITOR" ]; then
                    ^
SyntaxError: invalid syntax

If mercurial is installed using distutils directly;
   python setup.py install
both scripts work properly.

In build_scripts.py in distutils.command, distuitils uses the results of 
first_line_re to decide whether to adjust the python value in the shbang 
line of the script: if no "python" is found, distutils installs the 
script without modification.  easy_install.py uses the first_line_re 
test from distutils but seems to unconditionally add a wrapper to the 
script during installation regardless of the script's type.
-- 
 Ned Deily,
 nad at acm.org


From risky.martin at gmail.com  Sat Jun 10 09:49:04 2006
From: risky.martin at gmail.com (Risky Martin)
Date: Sat, 10 Jun 2006 15:49:04 +0800
Subject: [Distutils] unrecognised swig options ?
In-Reply-To: <186780900606070759m8314ae2vdea0c9373f739fc9@mail.gmail.com>
References: <186780900606070759m8314ae2vdea0c9373f739fc9@mail.gmail.com>
Message-ID: <186780900606100049k7921eb15g4618bd9507854f70@mail.gmail.com>

Well, well... after some time crawling after information on distutils,
SWIG, and C++, it seems that disutils has some problems with
the combo SWIG/C++ (and bug/patches found across found on
mailing list never included in distutils :/ ).

So setuptools seems more alive, and I gave it a go:
Same example as below, replacing

from distutils.core import setup, Extension

by

from setuptools import setup, Extension

.

No luck. Same results.

Anyone with an idea ?


--thanks.



2006/6/7, Risky Martin <risky.martin at gmail.com>:
>
> Hi,
>
>
> I am trying to use disutils's features with 'SWIG'... but hit
> a rather strange problem.
>
> To reproduce it:
>
> setup.py
> ------------------------------------------------------
> from distutils.core import setup, Extension
>
> e = Extension("_foo",
>               sources = ["foo.i", ],
>               swig_opts = ["-outdir .", ])
>
>
> setup(name="foo",
>       ext_modules=[e, ])
> -------------------------------------------------------
>
> building fails like:
>
> > python setup.py build
> running build
> running build_ext
> building '_foo' extension
> swigging foo.i to foo_wrap.c
> swig -python -outdir . -o foo_wrap.c foo.i
> swig error : Unrecognized option -outdir .
> Use 'swig -help' for available options.
> error: command 'swig' failed with exit status 1
>
> but a copy/paste of the swig command shows that '-outdir' *is*
> recognised...
> > swig -python -outdir . -o foo_wrap.c foo.i
> Unable to find file 'foo.i'.
>
> Are the permitted options for SWIG hardcoded in 'distutils' ?
>
>
>
> RiMa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20060610/951ddd6a/attachment.html 

From jerome.bouat at wanadoo.fr  Sun Jun 11 10:18:15 2006
From: jerome.bouat at wanadoo.fr (=?ISO-8859-1?Q?J=E9r=F4me?= Bouat)
Date: Sun, 11 Jun 2006 10:18:15 +0200
Subject: [Distutils] bad files extensions of text digests in list archive
Message-ID: <1150013895.6125.8.camel@localhost>

I browsed the list archive in:
http://mail.python.org/pipermail/distutils-sig/

Some text links are tagged as *.txt.gz
but there are simply plain text.


From chris at kateandchris.net  Mon Jun 12 17:05:04 2006
From: chris at kateandchris.net (Chris Lambacher)
Date: Mon, 12 Jun 2006 11:05:04 -0400
Subject: [Distutils] bad files extensions of text digests in list archive
In-Reply-To: <1150013895.6125.8.camel@localhost>
References: <1150013895.6125.8.camel@localhost>
Message-ID: <20060612150504.GB1673@kateandchris.net>

On Sun, Jun 11, 2006 at 10:18:15AM +0200, J?r?me Bouat wrote:
> I browsed the list archive in:
> http://mail.python.org/pipermail/distutils-sig/
> 
> Some text links are tagged as *.txt.gz
> but there are simply plain text.
Your browser is being helpful and unziping it for you.  If you right click on
the link and do save target as, you will get a zipped copy which you can then
unzip with gunzip or winzip.

-Chris
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

From pje at telecommunity.com  Mon Jun 12 18:52:49 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Jun 2006 12:52:49 -0400
Subject: [Distutils] setuptools: mishandles install of non-python scripts
In-Reply-To: <nad-332EEB.17221509062006@sea.gmane.org>
Message-ID: <5.1.1.6.0.20060612124800.01ead038@sparrow.telecommunity.com>

At 05:22 PM 6/9/2006 -0700, Ned Deily wrote:
>In build_scripts.py in distutils.command, distuitils uses the results of
>first_line_re to decide whether to adjust the python value in the shbang
>line of the script: if no "python" is found, distutils installs the
>script without modification.  easy_install.py uses the first_line_re
>test from distutils but seems to unconditionally add a wrapper to the
>script during installation regardless of the script's type.

Who needs shell scripts when you have Python?  ;)

I can (sort of) fix this, but there are some potentially-significant 
limitations.  Notably, if the shell script runs some Python code, it's not 
guaranteed that the library version it accesses will be the same one that 
the script was installed with, or that it will be able to access the 
library at all (e.g., if the user installed it with --multi-version).  I'll 
have to add some kind of warning message when processing non-Python scripts 
in --multi-version mode.


From jerome.bouat at wanadoo.fr  Mon Jun 12 21:25:56 2006
From: jerome.bouat at wanadoo.fr (=?ISO-8859-1?Q?J=E9r=F4me?= Bouat)
Date: Mon, 12 Jun 2006 21:25:56 +0200
Subject: [Distutils] bad files extensions of text digests in list	archive
In-Reply-To: <20060612150504.GB1673@kateandchris.net>
References: <1150013895.6125.8.camel@localhost>
	<20060612150504.GB1673@kateandchris.net>
Message-ID: <1150140356.5549.1.camel@localhost>

Le lundi 12 juin 2006 ? 11:05 -0400, Chris Lambacher a ?crit :
> On Sun, Jun 11, 2006 at 10:18:15AM +0200, J?r?me Bouat wrote:
> > I browsed the list archive in:
> > http://mail.python.org/pipermail/distutils-sig/
> > 
> > Some text links are tagged as *.txt.gz
> > but there are simply plain text.
> Your browser is being helpful and unziping it for you.  If you right click on
> the link and do save target as, you will get a zipped copy which you can then
> unzip with gunzip or winzip.
> 
> -Chris
> > 
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > http://mail.python.org/mailman/listinfo/distutils-sig
> 

You're right. Sorry.


From Jens.Engel at marconi.com  Wed Jun 14 12:11:50 2006
From: Jens.Engel at marconi.com (Jens Engel)
Date: Wed, 14 Jun 2006 12:11:50 +0200
Subject: [Distutils] Minor bug in PythonEggs setuptools-0.6b3 and earlier)?!
Message-ID: <OF1A9CCF93.1F7F1306-ONC125718D.00375988@uk.marconicomms.com>

Hello,

PythonEggs seem to have a problem access the docstring of the root 
package.
Note that there is no problem accessing docstrings of contained modules 
and packages with Python's help().

The following session gives an example what I mean by using (and 
installing) the funkload package.

SYNDROME: Install PythonEgg and verify that help() does no longer work => 
SYNDROME
shell> cd funkload-1.5.0
shell> python setup.py install
...
# -- INSTALLS PythonEgg ...
# TEST: To access docstring of egg-root package -> FAILS.
shell> cd ..
shell> python
>>> import funkload
>>> help(funkload)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File 
"/usr/lib/python2.4/site-packages/setuptools-0.6b3-py2.4.egg/site.py", 
line 328, in __call__

  File "/usr/lib/python2.4/pydoc.py", line 1647, in __call__
    self.help(request)
  File "/usr/lib/python2.4/pydoc.py", line 1691, in help
    else: doc(request, 'Help on %s:')
  File "/usr/lib/python2.4/pydoc.py", line 1475, in doc
    pager(title % desc + '\n\n' + text.document(object, name))
  File "/usr/lib/python2.4/pydoc.py", line 295, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "/usr/lib/python2.4/pydoc.py", line 1052, in docmodule
    for file in os.listdir(object.__path__[0]):
OSError: [Errno 2] No such file or directory: 
'/usr/lib/python2.4/site-packages/funkload-1.5.0-py2.4.egg/funkload'


Ciao,
Jens Engel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20060614/ea85bc3f/attachment.htm 

From jerome.bouat at wanadoo.fr  Wed Jun 14 15:59:28 2006
From: jerome.bouat at wanadoo.fr (=?ISO-8859-1?Q?J=E9r=F4me?= Bouat)
Date: Wed, 14 Jun 2006 15:59:28 +0200
Subject: [Distutils] Why using distutils.core.setup requires Python sources ?
Message-ID: <1150293569.8913.5.camel@localhost>

I don't understand why distutils.core.setup needs Python sources (a
missing Makefile on my distro).

I did not found in the mail archives.

Could you explain me ?

Thanks.


From pje at telecommunity.com  Wed Jun 14 16:42:52 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Jun 2006 10:42:52 -0400
Subject: [Distutils] Minor bug in PythonEggs setuptools-0.6b3 and
 earlier)?!
In-Reply-To: <OF1A9CCF93.1F7F1306-ONC125718D.00375988@uk.marconicomms.co
 m>
Message-ID: <5.1.1.6.0.20060614103757.01e99050@sparrow.telecommunity.com>

At 12:11 PM 6/14/2006 +0200, Jens Engel wrote:

>Hello,
>
>PythonEggs seem to have a problem access the docstring of the root package.
>Note that there is no problem accessing docstrings of contained modules 
>and packages with Python's help().

The pydoc module (which implements the help() function) doesn't support zip 
files when listing the contents of a package.  (That is, it's not the 
docstring that there's a problem with, it's help() trying to list the 
modules from the package.)

This is a Python bug that is fixed in Python 2.5.  I plan to release a 
hotfix package for older versions of Python that fixes the bug.

In the meantime, you can work around it by using the --always-unzip option 
when easy_install-ing a package; the package will then be installed as a 
directory instead of a zipfile.  But be sure you need it, because egg 
directories slow down startup and importing much more than zipfiles do.

Alternatively, you can just grab the Python 2.5 version of the pydoc and 
pkgutil modules and drop them on your PYTHONPATH ahead of the standard 
library versions.  (The 2.5 version of pydoc uses utilities in pkgutil to 
access packages' module lists.)


From pje at telecommunity.com  Thu Jun 15 21:02:19 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Jun 2006 15:02:19 -0400
Subject: [Distutils] setuptools: mishandles install of non-python scripts
In-Reply-To: <nad-332EEB.17221509062006@sea.gmane.org>
Message-ID: <5.1.1.6.0.20060615150131.03347c70@sparrow.telecommunity.com>

At 05:22 PM 6/9/2006 -0700, Ned Deily wrote:
>easy_install doesn't properly handle non-python scripts in a
>distribution.
>...
>In build_scripts.py in distutils.command, distuitils uses the results of
>first_line_re to decide whether to adjust the python value in the shbang
>line of the script: if no "python" is found, distutils installs the
>script without modification.  easy_install.py uses the first_line_re
>test from distutils but seems to unconditionally add a wrapper to the
>script during installation regardless of the script's type.

There's another wrinkle, as it turns out.  It's not sufficient to inspect 
the first line for a #!python presence - it's perfectly valid and possible 
to have .py or .pyw scripts that don't have a #! line, especially on 
Windows.  So, the actual condition to test if the program is a Python 
script should be to compile it.  If it's valid Python syntax, easy_install 
will assume it's a Python script, rather than a shell, batch, or other kind 
of script file.

But, it's *also* necessary to check whether there is a *non*-Python #! 
line, if the first line begins with #!.  It's quite possible for a shell 
script to be syntactically valid Python, especially if you have a shell 
prolog hidden in what looks like a Python docstring.  This is a trick that 
I believe is listed in the Python FAQ, although I don't know if it's used 
in any currently-distributed scripts.  But it should be supported.

So the full rules for analyzing scripts should be:

* If there's a #! line, use first_line_re to tell if it's Python
* If there's a '.py' or '.pyw' extension, assume it's Python
* Otherwise, try to compile the script.  If it compiles, assume it's Python
* Otherwise, assume it's not.

There is one possiblity I'm leaving out here, which is that you have some 
other script interpreter (e.g. .bat or .cmd files on Windows) that just 
magically happen to be syntactically-valid Python.  In the .bat/.cmd case, 
anything more complex than a series of one-word commands with no arguments 
would fail to be valid Python, so it seems incredibly unlikely that this 
would ever happen.

Actually, there's one other possibility I'm also leaving out, which is that 
you have a .bat or .cmd or some other script whose first line uses -x to 
skip that line.  In such a case, lines 2-N of the file are valid Python, 
but line 1 doesn't start with #!.  In theory, what this should do is keep 
the first line and wrap the rest.  However, such scripts aren't supported 
by the distutils, so I'm not going to add them to setuptools for 0.6.


From pje at telecommunity.com  Thu Jun 15 21:05:48 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Jun 2006 15:05:48 -0400
Subject: [Distutils] setuptools: mishandles install of non-python scripts
In-Reply-To: <5.1.1.6.0.20060612124800.01ead038@sparrow.telecommunity.co
 m>
References: <nad-332EEB.17221509062006@sea.gmane.org>
Message-ID: <5.1.1.6.0.20060615141308.01f10bd0@sparrow.telecommunity.com>

At 12:52 PM 6/12/2006 -0400, Phillip J. Eby wrote:
>At 05:22 PM 6/9/2006 -0700, Ned Deily wrote:
> >In build_scripts.py in distutils.command, distuitils uses the results of
> >first_line_re to decide whether to adjust the python value in the shbang
> >line of the script: if no "python" is found, distutils installs the
> >script without modification.  easy_install.py uses the first_line_re
> >test from distutils but seems to unconditionally add a wrapper to the
> >script during installation regardless of the script's type.
>
>Who needs shell scripts when you have Python?  ;)
>
>I can (sort of) fix this, but there are some potentially-significant
>limitations.

There's another wrinkle, as it turns out.  It's not sufficient to inspect 
the first line for a #!python presence - it's perfectly valid and possible 
to have .py or .pyw scripts that don't have a #! line, especially on 
Windows.  So, the actual condition to test if the program is a Python 
script should be to compile it.  If it's valid Python syntax, easy_install 
will assume it's a Python script, rather than a shell, batch, or other kind 
of script file.

But, it's *also* necessary to check whether there is a *non*-Python #! 
line, if the first line begins with #!.  It's quite possible for a shell 
script to be syntactically valid Python, especially if you have a shell 
prolog hidden in what looks like a Python docstring.  This is a trick that 
I believe is listed in the Python FAQ, although I don't know if it's used 
in any currently-distributed scripts.  But it should be supported.

So the full rules for analyzing scripts should be:

* If there's a #! line, use first_line_re to tell if it's Python
* If there's a '.py' or '.pyw' extension, assume it's Python
* Otherwise, try to compile the script.  If it compiles, assume it's Python
* Otherwise, assume it's not.

There is one possiblity I'm leaving out here, which is that you have some 
other script interpreter (e.g. .bat or .cmd files on Windows) that just 
magically happen to be syntactically-valid Python.  In the .bat/.cmd case, 
anything more complex than a series of one-word commands with no arguments 
would fail to be valid Python, so it seems incredibly unlikely that this 
would ever happen.

Actually, there's one other possibility I'm also leaving out, which is that 
you have a .bat or .cmd or some other script whose first line uses -x to 
skip that line.  In such a case, lines 2-N of the file are valid Python, 
but line 1 doesn't start with #!.  In theory, what this should do is keep 
the first line and wrap the rest.  However, such scripts aren't supported 
by the distutils, so I'm not going to add them to setuptools for 0.6.


From public-dstml at giss.ath.cx  Mon Jun 19 00:58:29 2006
From: public-dstml at giss.ath.cx (AGiss)
Date: Mon, 19 Jun 2006 00:58:29 +0200
Subject: [Distutils] setuptools : easy_install error on berlios
Message-ID: <20060618225829.GA31234@gissehel.mine.nu>

berlios, sourceforge like, now use prdownloads system, like sourceforge.

setuptools fail to find eggs, because it tries to download on
"prdownload" which serves only html page with mirrors.

package_index.py need a fix_berlios_url (or perhaps a more
generic fix_forges_url with a list of translations)

I'll have a look at the code, but I don't have much time thoses days.

You can test with :

easy_install pydirstat


From pje at telecommunity.com  Mon Jun 19 02:48:02 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 18 Jun 2006 20:48:02 -0400
Subject: [Distutils] setuptools : easy_install error on berlios
In-Reply-To: <20060618225829.GA31234@gissehel.mine.nu>
Message-ID: <5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>

At 12:58 AM 6/19/2006 +0200, AGiss wrote:
>berlios, sourceforge like, now use prdownloads system, like sourceforge.
>
>setuptools fail to find eggs, because it tries to download on
>"prdownload" which serves only html page with mirrors.
>
>package_index.py need a fix_berlios_url (or perhaps a more
>generic fix_forges_url with a list of translations)

Sigh.  I was really hoping sourceforge were the only ones that did this.

However, Sourceforge at least has a round-robin DNS for their mirrors 
now.  I suggest finding out if there is a similar setup for berlios.  I say 
this because I'm not going to add any more screen-scraping code to 
easy_install to figure out what mirrors are available.  URL translation, 
*maybe*.  But trying to discover Sourceforge mirrors by screenscraping 
tricks was a bad idea and the code to do it is long gone.

I would suggest to the package author that PyPI hosting is quite 
convenient; they are clearly already running "python setup.py bdist_egg", 
and uploading to PyPI simply makes the command be this instead:

     python setup.py register bdist_egg upload

If you want to also upload the source, add "sdist" before the "upload".

It is possible that some day the cheeseshop will be bogged down for 
uploads, but presumably people will step up to provide mirrors for it in 
that event.


From public-dstml at giss.ath.cx  Mon Jun 19 10:51:22 2006
From: public-dstml at giss.ath.cx (AGiss)
Date: Mon, 19 Jun 2006 10:51:22 +0200
Subject: [Distutils] setuptools : easy_install error on berlios
In-Reply-To: <5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
References: <20060618225829.GA31234@gissehel.mine.nu>
	<5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
Message-ID: <20060619085122.GA3920@gissehel.mine.nu>

On Sun, Jun 18, 2006 at 08:48:02PM -0400, Phillip J. Eby wrote:
> At 12:58 AM 6/19/2006 +0200, AGiss wrote:
> >berlios, sourceforge like, now use prdownloads system, like sourceforge.
> >
> >setuptools fail to find eggs, because it tries to download on
> >"prdownload" which serves only html page with mirrors.
> >
> >package_index.py need a fix_berlios_url (or perhaps a more
> >generic fix_forges_url with a list of translations)


> Sigh.  I was really hoping sourceforge were the only ones that did this.
> 
> However, Sourceforge at least has a round-robin DNS for their mirrors 
> now.  I suggest finding out if there is a similar setup for berlios.  I say 
> this because I'm not going to add any more screen-scraping code to 
> easy_install to figure out what mirrors are available.  URL translation, 
> *maybe*.  But trying to discover Sourceforge mirrors by screenscraping 
> tricks was a bad idea and the code to do it is long gone.

  Well, there is no roundrobin, only a "prdownload" to translate to
"download" or "download2".
 
> I would suggest to the package author that PyPI hosting is quite 
> convenient; they are clearly already running "python setup.py bdist_egg", 
> and uploading to PyPI simply makes the command be this instead:
> 
>     python setup.py register bdist_egg upload

Well, eggs are for long on pypi, but... easy_install doesn't download 
them because it fails on berlios, and doesn't try pypi.
(The workaround is to not provide the download URL on berlios in 
setup.py. That's what I'll do in next release.)


From pje at telecommunity.com  Mon Jun 19 14:44:48 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 19 Jun 2006 08:44:48 -0400
Subject: [Distutils] setuptools : easy_install error on berlios
In-Reply-To: <20060619085122.GA3920@gissehel.mine.nu>
References: <5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
	<20060618225829.GA31234@gissehel.mine.nu>
	<5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060619084407.030f8d38@sparrow.telecommunity.com>

At 10:51 AM 6/19/2006 +0200, AGiss wrote:
>(The workaround is to not provide the download URL on berlios in
>setup.py. That's what I'll do in next release.)

FYI, you don't need a new release; just edit the setup.py and run "setup.py 
register" -- that will update your PyPI listing immediately, with no need 
for a new release.


From strawman at astraw.com  Mon Jun 19 18:13:33 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 19 Jun 2006 09:13:33 -0700
Subject: [Distutils] [Announce] stdeb - Python to Debian source package
	conversion utility
Message-ID: <4496CD2D.9040804@astraw.com>

I would like to announce the initial public release of stdeb, which may
be found at http://stdeb.python-hosting.com/ . The rest of this email is
copied directly from that web-page.

stdeb - Python to Debian source package conversion utility
==========================================================

stdeb_ ("setuptools debian") produces Debian source packages from
Python packages via a new distutils command, ``sdist_dsc``, which
produces a Debian source package of a Python package.  Automatic
defaults are provided for the Debian package, but many aspects of the
resulting package can be customized via a configuration file.

.. _stdeb: http://stdeb.python-hosting.com/

News
----

2006-06-19: Version 0.1 Released. See the `download page`_.

Invocation
----------

All methods eventually result in a call to the ``sdist_dsc`` distutils
command. You may prefer to do so directly::

  python -c "import stdeb; execfile('setup.py')" sdist_dsc

Alternatively, two scripts are provided::

  stdeb_run_setup [options] # calls "python setup.py sdist_dsc [options]"

  py2dsc [options] mypackage-0.1.tar.gz # uses pre-built Python source
package

In all cases, a Debian source package is produced from unmodified
Python packages. The following files are produced:

 * ``packagename_versionname.orig.tar.gz``
 * ``packagename_versionname-debianversion.dsc``
 * ``packagename_versionname-debianversion.diff.gz``

These can then be compiled into binary packages using the standard
Debian machinery (e.g. dpkg-buildpackage).

Download
--------

Files are available at the `download page`_.

.. _download page: http://stdeb.python-hosting.com/wiki/Download

The subversion repository is available at
https://svn.stdeb.python-hosting.com/trunk

Background
----------

For the average Python package, its source distribution
(python_package.tar.gz created with ``python setup.py sdist``)
contains nearly everything necessary to make a Debian source
package. This near-equivalence encouraged me to write this little
distutils extension, which executes the setup.py file to extract
relevant information. This process is made significantly easier
through the use of setuptools_.

.. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools

setuptools is used because of the opportunities it provides, although
many of these features are currently un(der)-utilized. For example,
setuptools could make the job of "Debianizing" python console and gui
scripts much easier.

I wrote this initially to Debianize several Python packages of my own,
but I have the feeling it could be generally useful. It appears
similar, at least in theory, to easydeb_ and `Logilab's Devtools`_.

.. _easydeb: http://easy-deb.sourceforge.net/
.. _Logilab's DevTools: http://www.logilab.org/projects/devtools

Prerequisites
-------------

 * Python_ 2.3 or greater
 * setuptools_
 * subprocess.py_ (included with Python 2.4, backwards compatible with
Python 2.3)

.. _Python: http://www.python.org/
.. _subprocess.py:
http://svn.python.org/view/python/trunk/Lib/subprocess.py?rev=46651&view=log

Customizing the produced Debian source package
----------------------------------------------

stdeb will attempt to provide reasonable defaults, but these are only
guesses.

To customize the Debian source package produced, you may write config
files of the format understood by ConfigParser_. When building each
package, stdeb looks for the existance of a ``stdeb.cfg`` file in the
``.egg-info`` directory. You may specify an additional config file
with the command-line option --extra-cfg-file.

.. _ConfigParser: http://docs.python.org/lib/module-ConfigParser.html

Here's an example .cfg file which builds several packages::

 [DEFAULT]
 Debian-Version: 0ads1

 [setuptools]
 Source: python-setuptools

 [numpy]
 Source: python-numpy
 Upstream-Version-Prefix: 0.9.8+
 Build-Depends: python-dev, refblas3-dev, lapack3-dev
 Build-Conflicts: atlas3-base, atlas3-base-dev

 [matplotlib]
 # matplotlib doesn't incorporate its SVN version number into
sdist-built tarballs.
 # Therefore, if building the SVN version, substitute the version into the
 # "Upstream-Version-Suffix" variable and use py2dsc.
 # (For some reason, "debuild -sa" won't build matplotlib because tk.h
isn't found.)
 Source: python-matplotlib
 Upstream-Version-Suffix: .dev2500
 Build-Depends: python-dev, python-numpy, python-numarray,
python-numeric, python-gtk2-dev, tk8.4-dev, libwxgtk2.4-dev
 Depends: python-gtk2, python-numpy, python-numeric, python-numarray
 Suggests: gs-gpl

 [scipy]
 Source: python-scipy
 Upstream-Version-Prefix: 0.4.9+
 Build-Depends: python-numpy
 Depends: python-numpy

.. _numpy: http://scipy.org/NumPy

Using stdeb on stdeb
--------------------

There is a chicken-and-egg problem when trying to make a Debian
package of stdeb with stdeb. Here's a recipe to avoid it::

 # in the stdeb distribution directory (with setup.py)
 python setup.py sdist
 python setup.py build
 PYTHONPATH="build/lib" python stdeb/py2dsc.py dist/stdeb-VERSION.tar.gz

TODO
----

* Make output meet `Debian Python Policy`_ specifications or the `new
  python policy`_. This will include several things, including:

  - the ability to make custom changelogs
  - the ability to patch upstream source
  - the ability to include project-supplied documentation (including
    license information) as a -doc package
  - the ability to include project-supplied examples, tests, and data
    as a separate package
  - much more not listed

* Support python-central_ and/or python-support.

* Create (better) documentation

* Log output using standard distutils mechanisms

* Allow distribution-specific configuration parameters (e.g. numpy-dapper)

.. _debian python policy:
http://www.debian.org/doc/packaging-manuals/python-policy/
.. _new python policy: http://wiki.debian.org/DebianPython/NewPolicy
.. _python-central:
http://python-modules.alioth.debian.org/python-central_howto.txt

Call for volunteers
-------------------

I don't have a lot of time for this. This project stands a very real
chance of being only a shadow of its potential self unless people step
up and contribute. There are numerous ways in which people could
help. In particular, I'd be interested in finding a co-maintainer or
maintainer if the project generates any interest. Secondarily, I would
appreciate advice from Debian developers or Ubuntu MOTUs about the
arcane details of Python packaging.

License
-------

MIT-style license. Copyright (c) 2006 stdeb authors.

See the LICENSE.txt file provided with the source distribution for
full details.

Authors
-------

Andrew Straw, California Institute of Technology <strawman at astraw.com>

From constant.beta at gmail.com  Tue Jun 20 22:58:40 2006
From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=)
Date: Tue, 20 Jun 2006 22:58:40 +0200
Subject: [Distutils] easy_install fails for Amara XML Toolkit
Message-ID: <5e8b0f6b0606201358p417c898bi398586044023322f@mail.gmail.com>

Hi,

easy_install from setuptools 0.6b3 fails when trying to download Amara
XML Toolkit from PyPI. Full backtrace in attachement.

mk
-- 
 . o .       >>  http://joker.linuxstuff.pl  <<
 . . o   It's easier to get forgiveness for being wrong
 o o o   than forgiveness for being right.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-amara.log
Type: text/x-log
Size: 4042 bytes
Desc: not available
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20060620/91cec73e/attachment.bin 

From pje at telecommunity.com  Tue Jun 20 23:14:25 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 20 Jun 2006 17:14:25 -0400
Subject: [Distutils] easy_install fails for Amara XML Toolkit
In-Reply-To: <5e8b0f6b0606201358p417c898bi398586044023322f@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20060620171248.03ab59a8@sparrow.telecommunity.com>

At 10:58 PM 6/20/2006 +0200, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote:
>easy_install from setuptools 0.6b3 fails when trying to download Amara
>XML Toolkit from PyPI. Full backtrace in attachement.

It appears that ftp:// URLs opened by urllib2 don't get any content-type 
information.  I'll fix this in the next release.  In the meantime, if you 
change this line:

   File 
"/usr/lib/python2.3/site-packages/setuptools-0.6b3-py2.3.egg/setuptools/package_index.py", 
line 165, in process_url
     if 'html' not in f.headers['content-type'].lower():

to read instead:

     if 'html' not in f.headers.get('content-type', '').lower():

that should get you out of trouble till then.  Thanks.


From jim at zope.com  Wed Jun 21 22:52:19 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 21 Jun 2006 16:52:19 -0400
Subject: [Distutils] Requirent specifiers specified? :)
Message-ID: <4499B183.5030602@zope.com>


When specifying a dependencies and when asking for a package with
easy_install, you can specify one or more specifiers.  It's unclear what
the rules are for combining specifiers.

Imagine that I have a collection of eggs like:

   /home/jim/tmp/dist:
   used 92 available 41345796
   -rw-rw-r--  1 jim jim   671 Jun 19 17:43 demoneeded-1.0-py2.4.egg
   -rw-rw-r--  1 jim jim   672 Jun 19 17:46 demoneeded-1.1-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.2-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.3-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.4-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.5-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.6-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.7-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.8-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.9-py2.4.egg

Now, here are some examples of using the pkg_resources api to fetch a
required distribution:

 >>> import pkg_resources
 >>> e = pkg_resources.Environment(['tmp/dist'])
 >>> ws = pkg_resources.WorkingSet()

 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded ==1.1, ==1.4')], e)
[demoneeded 1.4 (/home/jim/tmp/dist/demoneeded-1.4-py2.4.egg)]

Here, the specifiers were or-ed. OK.

 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded >1.1, <1.6')], e)
[demoneeded 1.5 (/home/jim/tmp/dist/demoneeded-1.5-py2.4.egg)]

Here they were and-ed.  This makes sense, from a dwimy point of view. :)
If they were or-ed, I'd expect to get 1.9.

 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded >1.1, <1.6, ==1.8')], e)
[demoneeded 1.8 (/home/jim/tmp/dist/demoneeded-1.8-py2.4.egg)]

Hm, here the ==1.8 seems to have been or-ed with the result of anding >1.1
and <1.6.

 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded <1.1, >1.6')], e)
[demoneeded 1.9 (/home/jim/tmp/dist/demoneeded-1.9-py2.4.egg)]
 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded !=1.8, ==1.8')], e)
[demoneeded 1.9 (/home/jim/tmp/dist/demoneeded-1.9-py2.4.egg)]

I really don't know what's going on with these. :)

Are the rules for combining specifiers specified anywhere?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From constant.beta at gmail.com  Wed Jun 21 23:02:56 2006
From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=)
Date: Wed, 21 Jun 2006 23:02:56 +0200
Subject: [Distutils] easy_install download logic?
Message-ID: <5e8b0f6b0606211402y471dc2cg3ed807c8a1bdf438@mail.gmail.com>

Hi,

I'm wondering how exactly easy_install looks up packages on PyPI.
Right now I'm looking at two specific packages: moofx and SQLObject.
Both have "Home page" link and both have eggs distributions listen on
PyPI. The only difference is that moofx has also a "Download URL",
which seems to be bad.

When I do "easy_install SQLObject" it looks at PyPI, then link from
"Home page" field and fall backs to egg:

=====
$ sudo easy_install SQLObject==0.7.0
Searching for SQLObject==0.7.0
Reading http://www.python.org/pypi/SQLObject/
Reading http://sqlobject.org
Best match: SQLObject 0.7.0
Downloading http://cheeseshop.python.org/packages/2.3/S/SQLObject/SQLObject-0.7.0-py2.3.egg#md5=3105dc13b1df383007d08c7d0e1f8c2a
=====

But this doesn't happen for moofx. It looks at PyPI, then at "Home
page", then at "Download URL" and fails, ignoring eggs that are there:

=====
$ sudo easy_install moofx
Searching for moofx
Reading http://www.python.org/pypi/moofx/
Reading http://moofx.mad4milk.net/
Reading http://www.turbogears.org/cogbin/
No local packages or download links found for moofx
error: Could not find suitable distribution for Requirement.parse('moofx')
=====

Having broken download URL is a bad thing, but shouldn't easy_install
use eggs when they're available? And why search "Homepage" and
"Download URL" links when eggs are already on PyPI?

Thanks for any help,
mk
-- 
 . o .       >>  http://joker.linuxstuff.pl  <<
 . . o   It's easier to get forgiveness for being wrong
 o o o   than forgiveness for being right.

From pje at telecommunity.com  Wed Jun 21 23:11:16 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Jun 2006 17:11:16 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <4499B183.5030602@zope.com>
Message-ID: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>

At 04:52 PM 6/21/2006 -0400, Jim Fulton wrote:
>  >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded >1.1, <1.6, 
> ==1.8')], e)
>[demoneeded 1.8 (/home/jim/tmp/dist/demoneeded-1.8-py2.4.egg)]
>
>Hm, here the ==1.8 seems to have been or-ed with the result of anding >1.1
>and <1.6.

Yes.


>  >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded 
> <1.1, >1.6')], e)
>[demoneeded 1.9 (/home/jim/tmp/dist/demoneeded-1.9-py2.4.egg)]
>  >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded !=1.8, 
> ==1.8')], e)
>[demoneeded 1.9 (/home/jim/tmp/dist/demoneeded-1.9-py2.4.egg)]
>
>I really don't know what's going on with these. :)
>
>Are the rules for combining specifiers specified anywhere?

I was going to say, of course, and give you a link, but I'm surprised to 
discover there's nothing I can find.  Which is weird because I *know* I 
wrote something.  However I suspect it was originally on the PythonEggs 
wiki page, and it didn't get transferred to the PkgResources or setuptools 
pages before being deleted.

Ah, yes, that's in fact the case.  Here's the missing paragraph, grabbed 
from the wiki history:

"""Requirement strings basically consist of a distribution name, an 
optional list of "options" (more on this in a moment), and a 
comma-separated list of zero or more version conditions. Version conditions 
basically specify ranges of valid versions, using comparison operators. The 
version conditions you supply are sorted into ascending version order, and 
then scanned left to right until the package's version falls between a pair 
of > or >= and < or <= conditions, or exactly matches a == or != condition."""

I'll have to insert this back in the docs somewhere.  Thanks for alerting 
me to it.


From jim at zope.com  Wed Jun 21 23:36:21 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 21 Jun 2006 17:36:21 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <4499BBD5.40103@zope.com>

Phillip J. Eby wrote:
> At 04:52 PM 6/21/2006 -0400, Jim Fulton wrote:
> 
...

> I was going to say, of course, and give you a link, but I'm surprised to 
> discover there's nothing I can find.  Which is weird because I *know* I 
> wrote something. 

I thought you did too.

 > However I suspect it was originally on the PythonEggs
> wiki page, and it didn't get transferred to the PkgResources or 
> setuptools pages before being deleted.
> 
> Ah, yes, that's in fact the case.  Here's the missing paragraph, grabbed 
> from the wiki history:
> 
> """Requirement strings basically consist of a distribution name, an 
> optional list of "options" (more on this in a moment), and a 
> comma-separated list of zero or more version conditions. Version 
> conditions basically specify ranges of valid versions, using comparison 
> operators. The version conditions you supply are sorted into ascending 
> version order, and then scanned left to right until the package's 
> version falls between a pair of > or >= and < or <= conditions, or 
> exactly matches a == or != condition."""

OK, this is something.  I'm sure what "ascending version order" is.
In ascending version order, what is the ordering of:

   <1.2, >1.2, ==1.2, <=1.2, >=1.2, and !=1.2

?

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Wed Jun 21 23:37:10 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 21 Jun 2006 17:37:10 -0400
Subject: [Distutils] Optimizing easy-install upgrade
Message-ID: <4499BC06.5070203@zope.com>


Suppose I have the directory:

   /home/jim/tmp/dist:
   used 92 available 41345796
   -rw-rw-r--  1 jim jim   671 Jun 19 17:43 demoneeded-1.0-py2.4.egg
   -rw-rw-r--  1 jim jim   672 Jun 19 17:46 demoneeded-1.1-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.2-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.3-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.4-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.5-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.6-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.7-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.8-py2.4.egg
   -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.9-py2.4.egg

and then run easy install telling it to install something I already have:

   [jim at ds9 ~]$ /usr/local/python/2.4/bin/easy_install -d tmp/dist -mxU demoneeded==1.1
   Searching for demoneeded==1.1
   Reading http://www.python.org/pypi/demoneeded/
   Couldn't find index page for 'demoneeded' (maybe misspelled?)
   Scanning index of all packages (this may take a while)
   Reading http://www.python.org/pypi/
   Best match: demoneeded 1.1
   Processing demoneeded-1.1-py2.4.egg

   Using /home/jim/tmp/dist/demoneeded-1.1-py2.4.egg

   Because this distribution was installed --multi-version, before you can
   import modules from this package in an application, you will need to
   'import pkg_resources' and then use a 'require()' call similar to one of
   these examples, in order to select the desired version:

       pkg_resources.require("demoneeded")  # latest installed version
       pkg_resources.require("demoneeded==1.1")  # this exact version
       pkg_resources.require("demoneeded>=1.1")  # this version or higher


   Note also that the installation directory must be on sys.path at runtime for
   this to work.  (e.g. by being the application's script directory, by being on
   PYTHONPATH, or by being added to sys.path by your code.)

   Processing dependencies for demoneeded==1.1

Note that it looked at pypi even though it already had the only acceptable
version.  I'm not sure why it did this.  Is this just a case of a missing
optimization?  Or is there some other reason for checking the index in
a situation like this?

I have some scripts that invoke easy_setup and I'd like to try to
do some of this logic myself.  Given a requirement, I'd like to
get the specifiers and decide myself whether to invoke easy_install.
I have 2 problems:

- I don't want to parse the requirement myself, but, rather,
   use Requirement.parse.  If I use Requirement.parse, I can use
   the specs attribute to get the specifiers, however, this
   attribute isn't documented.  Should I assume that it is
   private? Or is it safe to use.

- I'm still a bit foggy on how to interpret the specifiers.
   But we're discussing that in another thread.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From pje at telecommunity.com  Thu Jun 22 00:21:54 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Jun 2006 18:21:54 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <4499BBD5.40103@zope.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060621181919.039f58a0@sparrow.telecommunity.com>

At 05:36 PM 6/21/2006 -0400, Jim Fulton wrote:
>In ascending version order, what is the ordering of:
>
>   <1.2, >1.2, ==1.2, <=1.2, >=1.2, and !=1.2
>
>?

It's actually unspecified, because it makes no sense to have more than one 
condition for the same version.  What could the assortment of values you 
just listed possibly *mean*?


From pje at telecommunity.com  Thu Jun 22 00:27:27 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Jun 2006 18:27:27 -0400
Subject: [Distutils] Optimizing easy-install upgrade
In-Reply-To: <4499BC06.5070203@zope.com>
Message-ID: <5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>

At 05:37 PM 6/21/2006 -0400, Jim Fulton wrote:

>Suppose I have the directory:
>
>    /home/jim/tmp/dist:
>    used 92 available 41345796
>    -rw-rw-r--  1 jim jim   671 Jun 19 17:43 demoneeded-1.0-py2.4.egg
>    -rw-rw-r--  1 jim jim   672 Jun 19 17:46 demoneeded-1.1-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.2-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.3-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.4-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.5-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.6-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.7-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.8-py2.4.egg
>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.9-py2.4.egg
>
>and then run easy install telling it to install something I already have:
>
>    [jim at ds9 ~]$ /usr/local/python/2.4/bin/easy_install -d tmp/dist -mxU 
> demoneeded==1.1

The -U option means "always search PyPI".


>I have some scripts that invoke easy_setup and I'd like to try to
>do some of this logic myself.  Given a requirement, I'd like to
>get the specifiers and decide myself whether to invoke easy_install.
>I have 2 problems:
>
>- I don't want to parse the requirement myself, but, rather,
>    use Requirement.parse.  If I use Requirement.parse, I can use
>    the specs attribute to get the specifiers, however, this
>    attribute isn't documented.  Should I assume that it is
>    private? Or is it safe to use.

Why do that when you can just ask the Requirement whether it matches a 
particular version?  The __contains__ method of Requirement objects accepts 
a version string, distribution object, or parsed version number you can use.

(For that matter, you can query an Environment or WorkingSet for the 
distributions whose versions you want to check.)


From constant.beta at gmail.com  Thu Jun 22 00:31:32 2006
From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=)
Date: Thu, 22 Jun 2006 00:31:32 +0200
Subject: [Distutils] Infinite easy_install loop for mathdom==0.7.1
Message-ID: <5e8b0f6b0606211531t5b885ee9le3fb015e4f49d60a@mail.gmail.com>

Sorry for flooding the list with my questions, but I think I found
another problem with easy_install. Calling it on "mathdom==0.7.1"
invokes an infinite sourceforge download loop. I've given up after
watching "Download failed" messages for about 20 minutes. So I can't
prove this loop is infinite. ;-)

BTW, is there any kind of tracker for setuptools, so I could use that
instead of writing directly to this list? I couldn't found one one the
homepage.

mk
-- 
 . o .       >>  http://joker.linuxstuff.pl  <<
 . . o   It's easier to get forgiveness for being wrong
 o o o   than forgiveness for being right.

From pje at telecommunity.com  Thu Jun 22 00:36:48 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Jun 2006 18:36:48 -0400
Subject: [Distutils] easy_install download logic?
In-Reply-To: <5e8b0f6b0606211402y471dc2cg3ed807c8a1bdf438@mail.gmail.com
 >
Message-ID: <5.1.1.6.0.20060621183019.03b58fe8@sparrow.telecommunity.com>

At 11:02 PM 6/21/2006 +0200, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote:
>Hi,
>
>I'm wondering how exactly easy_install looks up packages on PyPI.
>Right now I'm looking at two specific packages: moofx and SQLObject.
>Both have "Home page" link and both have eggs distributions listen on
>PyPI. The only difference is that moofx has also a "Download URL",
>which seems to be bad.
>
>When I do "easy_install SQLObject" it looks at PyPI, then link from
>"Home page" field and fall backs to egg:
>
>=====
>$ sudo easy_install SQLObject==0.7.0
>Searching for SQLObject==0.7.0
>Reading http://www.python.org/pypi/SQLObject/
>Reading http://sqlobject.org
>Best match: SQLObject 0.7.0
>Downloading 
>http://cheeseshop.python.org/packages/2.3/S/SQLObject/SQLObject-0.7.0-py2.3.egg#md5=3105dc13b1df383007d08c7d0e1f8c2a
>=====
>
>But this doesn't happen for moofx.

Actually, it does.  Notice that you're using Python 2.3 -- moofx only has a 
Python 2.4 egg uploaded, and no source distro or 2.3 egg.


>It looks at PyPI, then at "Home
>page", then at "Download URL" and fails, ignoring eggs that are there:

Because the egg is for 2.4, and you're using 2.3.


>Having broken download URL is a bad thing, but shouldn't easy_install
>use eggs when they're available?

easy_install builds a complete index of the eggs or other downloadable 
distributions it finds, and then chooses the "best" one.


>And why search "Homepage" and
>"Download URL" links when eggs are already on PyPI?

To ensure that you can request the latest version; sometimes packagers 
don't update their PyPI pages as often.


From pje at telecommunity.com  Thu Jun 22 01:00:23 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Jun 2006 19:00:23 -0400
Subject: [Distutils] Infinite easy_install loop for mathdom==0.7.1
In-Reply-To: <5e8b0f6b0606211531t5b885ee9le3fb015e4f49d60a@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20060621185624.03c5c690@sparrow.telecommunity.com>

At 12:31 AM 6/22/2006 +0200, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote:
>Sorry for flooding the list with my questions, but I think I found
>another problem with easy_install. Calling it on "mathdom==0.7.1"
>invokes an infinite sourceforge download loop. I've given up after
>watching "Download failed" messages for about 20 minutes. So I can't
>prove this loop is infinite. ;-)

The problem is the broken URL for mathdom 0.7.1 provided on the mathdom 
PyPI page.  Sourceforge has no such file.  Please let the author know about 
the problem.  In the meantime:

easy_install 
http://cheeseshop.python.org/packages/source/m/mathdom/mathdom-0.7.1.tar.gz

should get you the package that is available.  (It's not using the 2.4 egg 
because again, you're on Python 2.3.)


>BTW, is there any kind of tracker for setuptools, so I could use that
>instead of writing directly to this list?

No; the intent is to allow everyone to learn from the answers, and thus 
better support others.  This wouldn't happen as well with a walled-off tracker.


From jim at zope.com  Thu Jun 22 13:13:35 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 07:13:35 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060621181919.039f58a0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621181919.039f58a0@sparrow.telecommunity.com>
Message-ID: <3ADEAC92-9514-41EE-948C-9485B8CB0BDC@zope.com>


On Jun 21, 2006, at 6:21 PM, Phillip J. Eby wrote:

> At 05:36 PM 6/21/2006 -0400, Jim Fulton wrote:
>> In ascending version order, what is the ordering of:
>>
>>   <1.2, >1.2, ==1.2, <=1.2, >=1.2, and !=1.2
>>
>> ?
>
> It's actually unspecified, because it makes no sense to have more  
> than one condition for the same version.  What could the assortment  
> of values you just listed possibly *mean*?

I don't know.  If it's illegal, then say so.  The software` certainly  
accepts it
now.  I think it would be helpful to call this out in your  
specification.  I also think
it might be good to expand the explanation a bit, perhaps with some
examples.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 22 13:17:05 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 07:17:05 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <827FF7D8-2052-403D-8A4E-D45442C64420@zope.com>


On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
...
> """Requirement strings basically consist of a distribution name, an  
> optional list of "options" (more on this in a moment), and a comma- 
> separated list of zero or more version conditions. Version  
> conditions basically specify ranges of valid versions, using  
> comparison operators. The version conditions you supply are sorted  
> into ascending version order, and then scanned left to right until  
> the package's version falls between a pair of > or >= and < or <=  
> conditions, or exactly matches a == or != condition."""

I suppose that given:

   <1.2, >1.3

there is an implicit >-infinity and <infinity so that, for example,  
1.9 falls between the pair >1,3 and <infinity.

Jim


--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 22 13:25:48 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 07:25:48 -0400
Subject: [Distutils] Optimizing easy-install upgrade
In-Reply-To: <5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
Message-ID: <515D9513-8C0E-4745-976D-5BD63F41F3F7@zope.com>


On Jun 21, 2006, at 6:27 PM, Phillip J. Eby wrote:

> At 05:37 PM 6/21/2006 -0400, Jim Fulton wrote:
>
>> Suppose I have the directory:
>>
>>    /home/jim/tmp/dist:
>>    used 92 available 41345796
>>    -rw-rw-r--  1 jim jim   671 Jun 19 17:43 demoneeded-1.0-py2.4.egg
>>    -rw-rw-r--  1 jim jim   672 Jun 19 17:46 demoneeded-1.1-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.2-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.3-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.4-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.5-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.6-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.7-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.8-py2.4.egg
>>    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.9-py2.4.egg
>>
>> and then run easy install telling it to install something I  
>> already have:
>>
>>    [jim at ds9 ~]$ /usr/local/python/2.4/bin/easy_install -d tmp/dist  
>> -mxU demoneeded==1.1
>
> The -U option means "always search PyPI".
>
>> I have some scripts that invoke easy_setup and I'd like to try to
>> do some of this logic myself.  Given a requirement, I'd like to
>> get the specifiers and decide myself whether to invoke easy_install.
>> I have 2 problems:
>>
>> - I don't want to parse the requirement myself, but, rather,
>>    use Requirement.parse.  If I use Requirement.parse, I can use
>>    the specs attribute to get the specifiers, however, this
>>    attribute isn't documented.  Should I assume that it is
>>    private? Or is it safe to use.
>
> Why do that when you can just ask the Requirement whether it  
> matches a particular version?  The __contains__ method of  
> Requirement objects accepts a version string, distribution object,  
> or parsed version number you can use.
>
> (For that matter, you can query an Environment or WorkingSet for  
> the distributions whose versions you want to check.)

Here's my use case: I want to get the most recent distribution that  
satisfies my
requirement.  If my requirement sets an upper bound, and I already  
have the distribution
at the upper bound, then I don't want to have to search the index.

For us, it will be very common to specify specific distribution  
versions.  I don't
want to have to search an index if I already have the specific version.

I understand that the -U option isn't designed to meet this use  
case.  That's
fine.  I want to be able to introspect a requirement  to determine  
wether
it set an upper bound.  __contains__ doesn't let me do that.  With  
the rules
that you've explained for evaluating a set of specifiers, I can do  
that if
I can get at the specifiers.  What I want is either a public API for  
getting the
specifiers, or an API that lets me retrieve the upper bound, if there  
is one.
(Obviously, it could return None if no upper bound exists.)

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 22 17:15:22 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 11:15:22 -0400
Subject: [Distutils] Puzzling easy_install behavior
Message-ID: <EB05E50A-5C50-4714-81B1-64005C598614@zope.com>

I'd like to share an experiment that I find puzzling and see if anyone
could share some insight. Perhaps I'm doing something incorrectly.

So, I have an links directory:

   [jim at ds9 tmp]$ ls links
   demo-1.0-py2.4.egg  demoneeded-1.0-py2.4.egg  index.html
   [jim at ds9 tmp]$ cat links/index.html
   <html><body>
     <a href="demo-1.0-py2.4.egg">demoneeded-1.0-py2.4.egg</a>
     <a href="demoneeded-1.0-py2.4.egg">demoneeded-1.0-py2.4.egg</a>
   </body></html>

and an empty eggs directory:

   [jim at ds9 tmp]$ ls eggs

Note that demo depends on demoneeded, without any qualifiers.

I run easy_setup (0.6b3):

   [jim at ds9 tmp]$ /usr/local/python/2.4/bin/easy_install -q \
                  -f file:///home/jim/tmp/links/index.html -m -deggs  
demo

and get the eggs I expect.

   [jim at ds9 tmp]$ ls eggs
   demo-1.0-py2.4.egg  demoneeded-1.0-py2.4.egg

I update the links with new versions of demo and demoneeded:

   [jim at ds9 tmp]$ ls links
   demo-1.0-py2.4.egg  demoneeded-1.0-py2.4.egg  index.html
   demo-1.1-py2.4.egg  demoneeded-1.1-py2.4.egg
   [jim at ds9 tmp]$ cat links/index.html
   <html><body>
     <a href="demo-1.0-py2.4.egg">demoneeded-1.0-py2.4.egg</a>
     <a href="demoneeded-1.0-py2.4.egg">demoneeded-1.0-py2.4.egg</a>
     <a href="demo-1.1-py2.4.egg">demoneeded-1.1-py2.4.egg</a>
     <a href="demoneeded-1.1-py2.4.egg">demoneeded-1.1-py2.4.egg</a>
   </body></html>

I haven't changed the unqualified dependence on demoneeded.

Now I rerun easy_install:

   [jim at ds9 tmp]$ /usr/local/python/2.4/bin/easy_install -q \
                  -f file:///home/jim/tmp/links/index.html -m -deggs  
demo

and I get a new demo egg even though I didn't elect to upgrade:

   [jim at ds9 tmp]$ ls eggs
   demo-1.0-py2.4.egg  demo-1.1-py2.4.egg  demoneeded-1.0-py2.4.egg

I find this a bit mysterious.  I thought easy_install wouldn't search
the find links and index if existing distributions meet a requirement
unless --upgrade was used.  Is special consideration given to file
URLs? (If so, this makes test writing a bit more challenging.)

If I supply the upgrade option, I get:

   [jim at ds9 tmp]$ /usr/local/python/2.4/bin/easy_install -q \
                  -f file:///home/jim/tmp/links/index.html -m -U - 
deggs demo
   Couldn't find index page for 'demo' (maybe misspelled?)
   Scanning index of all packages (this may take a while)

   [jim at ds9 tmp]$ ls eggs
   demo-1.0-py2.4.egg  demo-1.1-py2.4.egg  demoneeded-1.0-py2.4.egg

So when I said upgrade, the index was checked, but when I installed
the distributions initially, it wasn't.  Why?  Why is upgrade handled
differently from an initial install.

Also, it's a bit surprising that I didn't get the available upgrade
for demoneeded.  Is that intended?  Are there plans for a recursive
upgrade option?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 17:38:32 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 11:38:32 -0400
Subject: [Distutils] Puzzling easy_install behavior
In-Reply-To: <EB05E50A-5C50-4714-81B1-64005C598614@zope.com>
Message-ID: <5.1.1.6.0.20060622113339.01ffae70@sparrow.telecommunity.com>

At 11:15 AM 6/22/2006 -0400, Jim Fulton wrote:
>I find this a bit mysterious.  I thought easy_install wouldn't search
>the find links and index if existing distributions meet a requirement
>unless --upgrade was used.

--upgrade means "look for things remotely, even if you can satisfy the 
requirements locally"

>Is special consideration given to file
>URLs?

Yes, in that they are considered "local".


>If I supply the upgrade option, I get:
>
>    [jim at ds9 tmp]$ /usr/local/python/2.4/bin/easy_install -q \
>                   -f file:///home/jim/tmp/links/index.html -m -U -
>deggs demo
>    Couldn't find index page for 'demo' (maybe misspelled?)
>    Scanning index of all packages (this may take a while)
>
>    [jim at ds9 tmp]$ ls eggs
>    demo-1.0-py2.4.egg  demo-1.1-py2.4.egg  demoneeded-1.0-py2.4.egg
>
>So when I said upgrade, the index was checked, but when I installed
>the distributions initially, it wasn't.  Why?  Why is upgrade handled
>differently from an initial install.

Because upgrade really just overrides easy_install's preference to only 
look at the local machine to resolve dependencies.


>Also, it's a bit surprising that I didn't get the available upgrade
>for demoneeded.  Is that intended?  Are there plans for a recursive
>upgrade option?

The answer to both questions is "more or less."  :)  It's intended only in 
the sense that that's just what it does, and I don't plan to change 
it.  There are plans for recursive upgrade of some kind, but not to be 
added to easy_install.  Instead, that would be part of "nest", whenever I 
actually get some work done on it.

"nest" is supposed to be part of setuptools 0.7, but 0.6 final isn't out yet.


From jim at zope.com  Thu Jun 22 17:41:31 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 11:41:31 -0400
Subject: [Distutils] Puzzling easy_install behavior
In-Reply-To: <5.1.1.6.0.20060622113339.01ffae70@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060622113339.01ffae70@sparrow.telecommunity.com>
Message-ID: <4684AD7D-BEDA-425F-8AEA-2157AA0B01FD@zope.com>


On Jun 22, 2006, at 11:38 AM, Phillip J. Eby wrote:
>
> "nest" is supposed to be part of setuptools 0.7, but 0.6 final  
> isn't out yet.

I don't know what "nest" is.  Is it described anywhere?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 17:48:45 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 11:48:45 -0400
Subject: [Distutils] Optimizing easy-install upgrade
In-Reply-To: <515D9513-8C0E-4745-976D-5BD63F41F3F7@zope.com>
References: <5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622114327.01ff3db0@sparrow.telecommunity.com>

At 07:25 AM 6/22/2006 -0400, Jim Fulton wrote:
>I can get at the specifiers.  What I want is either a public API for
>getting the
>specifiers, or an API that lets me retrieve the upper bound, if there
>is one.
>(Obviously, it could return None if no upper bound exists.)

Use the specs attribute, which is a list of (op,ver) pairs in ascending 
version order (and an undefined order among pairs that have equivalent 
versions).  The 'ver' is an *unparsed* version number, so you should parse 
it in order to do comparisons.

The specs attribute is currently undocumented, but I'll document it.  It 
has to be the way it is, because it's needed in order to produce a 
consistent string representation of a particular requirement.


From pje at telecommunity.com  Thu Jun 22 17:51:23 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 11:51:23 -0400
Subject: [Distutils] Puzzling easy_install behavior
In-Reply-To: <4684AD7D-BEDA-425F-8AEA-2157AA0B01FD@zope.com>
References: <5.1.1.6.0.20060622113339.01ffae70@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622113339.01ffae70@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622114935.034d5678@sparrow.telecommunity.com>

At 11:41 AM 6/22/2006 -0400, Jim Fulton wrote:

>On Jun 22, 2006, at 11:38 AM, Phillip J. Eby wrote:
>>
>>"nest" is supposed to be part of setuptools 0.7, but 0.6 final
>>isn't out yet.
>
>I don't know what "nest" is.  Is it described anywhere?

Yes.  For example, in some of our previous IRC chats regarding setuptools.  :)

You can also google for distutils nest or setuptools nest and find quite a 
lot of previous discussion.  The design (such as it is) is only found in 
outline form in my PIM.


From jim at zope.com  Thu Jun 22 17:52:29 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 11:52:29 -0400
Subject: [Distutils] Optimizing easy-install upgrade
In-Reply-To: <5.1.1.6.0.20060622114327.01ff3db0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621182252.032772f0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622114327.01ff3db0@sparrow.telecommunity.com>
Message-ID: <210491BD-BB20-49F9-82DC-C1B75AD799DC@zope.com>


On Jun 22, 2006, at 11:48 AM, Phillip J. Eby wrote:

> At 07:25 AM 6/22/2006 -0400, Jim Fulton wrote:
>> I can get at the specifiers.  What I want is either a public API for
>> getting the
>> specifiers, or an API that lets me retrieve the upper bound, if there
>> is one.
>> (Obviously, it could return None if no upper bound exists.)
>
> Use the specs attribute, which is a list of (op,ver) pairs in  
> ascending version order (and an undefined order among pairs that  
> have equivalent versions).  The 'ver' is an *unparsed* version  
> number, so you should parse it in order to do comparisons.
>
> The specs attribute is currently undocumented, but I'll document  
> it.  It has to be the way it is, because it's needed in order to  
> produce a consistent string representation of a particular  
> requirement.

Cool. Thanks.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 17:52:15 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 11:52:15 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <827FF7D8-2052-403D-8A4E-D45442C64420@zope.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622115137.04f35488@sparrow.telecommunity.com>

At 07:17 AM 6/22/2006 -0400, Jim Fulton wrote:

>On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
>...
>>"""Requirement strings basically consist of a distribution name, an
>>optional list of "options" (more on this in a moment), and a comma- 
>>separated list of zero or more version conditions. Version
>>conditions basically specify ranges of valid versions, using
>>comparison operators. The version conditions you supply are sorted
>>into ascending version order, and then scanned left to right until
>>the package's version falls between a pair of > or >= and < or <=
>>conditions, or exactly matches a == or != condition."""
>
>I suppose that given:
>
>   <1.2, >1.3
>
>there is an implicit >-infinity and <infinity so that, for example,
>1.9 falls between the pair >1,3 and <infinity.

Yes.


From jim at zope.com  Thu Jun 22 18:26:53 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 12:26:53 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <BFA0D4B5-AA24-4928-93A2-57B93496D846@zope.com>


On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:

> """Requirement strings basically consist of a distribution name, an  
> optional list of "options" (more on this in a moment), and a comma- 
> separated list of zero or more version conditions. Version  
> conditions basically specify ranges of valid versions, using  
> comparison operators. The version conditions you supply are sorted  
> into ascending version order, and then scanned left to right until  
> the package's version falls between a pair of > or >= and < or <=  
> conditions, or exactly matches a == or != condition."""

I don't think this is right. :)

If I have a collection of eggs like:

    /home/jim/tmp/dist:
    used 92 available 41345796
    -rw-rw-r--  1 jim jim   671 Jun 19 17:43 demoneeded-1.0-py2.4.egg
    -rw-rw-r--  1 jim jim   672 Jun 19 17:46 demoneeded-1.1-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.2-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.3-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.4-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.5-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.6-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.7-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.8-py2.4.egg
    -rw-rw-r--  1 jim jim   673 Jun 19 17:46 demoneeded-1.9-py2.4.egg

Then:

  >>> import pkg_resources
  >>> e = pkg_resources.Environment(['tmp/dist'])
  >>> ws = pkg_resources.WorkingSet()
 >>> ws.resolve([pkg_resources.Requirement.parse('demoneeded !=1.1,  
<1.4')], e)
[demoneeded 1.3 (/home/jim/tmp/dist/demoneeded-1.3-py2.4.egg)]

When scanning left to right, 1.9 matches !=1.1, so it should match and,
since it is the highest version, it should be returned.  Either your  
description
of the algorithm is incorrect or I'm misunderstanding it.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 18:49:18 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 12:49:18 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <BFA0D4B5-AA24-4928-93A2-57B93496D846@zope.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>

At 12:26 PM 6/22/2006 -0400, Jim Fulton wrote:
>On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
>>"""Requirement strings basically consist of a distribution name, an
>>optional list of "options" (more on this in a moment), and a comma- 
>>separated list of zero or more version conditions. Version
>>conditions basically specify ranges of valid versions, using
>>comparison operators. The version conditions you supply are sorted
>>into ascending version order, and then scanned left to right until
>>the package's version falls between a pair of > or >= and < or <=
>>conditions, or exactly matches a == or != condition."""
>
>I don't think this is right. :)
>...
>When scanning left to right, 1.9 matches !=1.1, so it should match and,
>since it is the highest version, it should be returned.  Either your
>description
>of the algorithm is incorrect or I'm misunderstanding it.

You're missing the "exactly matches" part.  The relevant context is: "Until 
the package's version ... exactly matches a == or != condition".  Perhaps 
making that "a == or != condition's version" would have been clearer, as 
that's what I meant by that phrase.

Anyway, "1.9" does not exactly match the "1.1" in "!=1.1".


From jim at zope.com  Thu Jun 22 19:05:10 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 13:05:10 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
Message-ID: <93EEBA2C-9982-4ECC-920D-4868636110E7@zope.com>


On Jun 22, 2006, at 12:49 PM, Phillip J. Eby wrote:

> At 12:26 PM 6/22/2006 -0400, Jim Fulton wrote:
>> On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
>>> """Requirement strings basically consist of a distribution name, an
>>> optional list of "options" (more on this in a moment), and a  
>>> comma- separated list of zero or more version conditions. Version
>>> conditions basically specify ranges of valid versions, using
>>> comparison operators. The version conditions you supply are sorted
>>> into ascending version order, and then scanned left to right until
>>> the package's version falls between a pair of > or >= and < or <=
>>> conditions, or exactly matches a == or != condition."""
>>
>> I don't think this is right. :)
>> ...
>> When scanning left to right, 1.9 matches !=1.1, so it should match  
>> and,
>> since it is the highest version, it should be returned.  Either your
>> description
>> of the algorithm is incorrect or I'm misunderstanding it.
>
> You're missing the "exactly matches" part.  The relevant context  
> is: "Until the package's version ... exactly matches a == or !=  
> condition".  Perhaps making that "a == or != condition's version"  
> would have been clearer, as that's what I meant by that phrase.
>
> Anyway, "1.9" does not exactly match the "1.1" in "!=1.1".

Um, OK.  So I guess the idea is that we scan these things trying to  
make a decision.
The decision is either match or not match.  Is that how I was supposed
to read the above quote?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 19:42:24 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 13:42:24 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <93EEBA2C-9982-4ECC-920D-4868636110E7@zope.com>
References: <5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>

At 01:05 PM 6/22/2006 -0400, Jim Fulton wrote:

>On Jun 22, 2006, at 12:49 PM, Phillip J. Eby wrote:
>
>>At 12:26 PM 6/22/2006 -0400, Jim Fulton wrote:
>>>On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
>>>>"""Requirement strings basically consist of a distribution name, an
>>>>optional list of "options" (more on this in a moment), and a
>>>>comma- separated list of zero or more version conditions. Version
>>>>conditions basically specify ranges of valid versions, using
>>>>comparison operators. The version conditions you supply are sorted
>>>>into ascending version order, and then scanned left to right until
>>>>the package's version falls between a pair of > or >= and < or <=
>>>>conditions, or exactly matches a == or != condition."""
>>>
>>>I don't think this is right. :)
>>>...
>>>When scanning left to right, 1.9 matches !=1.1, so it should match
>>>and,
>>>since it is the highest version, it should be returned.  Either your
>>>description
>>>of the algorithm is incorrect or I'm misunderstanding it.
>>
>>You're missing the "exactly matches" part.  The relevant context
>>is: "Until the package's version ... exactly matches a == or !=
>>condition".  Perhaps making that "a == or != condition's version"
>>would have been clearer, as that's what I meant by that phrase.
>>
>>Anyway, "1.9" does not exactly match the "1.1" in "!=1.1".
>
>Um, OK.  So I guess the idea is that we scan these things trying to
>make a decision.
>The decision is either match or not match.  Is that how I was supposed
>to read the above quote?

Um, no.  :)  The specification is:

1. "The version conditions you supply are sorted into ascending version order"

2. "and then scanned left to right"  (from low to high version number)

3. "until the package's version"

3a. "falls between a pair of > or >= and < or <= conditions"

3b. "or exactly matches a == or != condition"

In both #3a and #3b, we are saying that the package's version is compared 
to the condition's version.  If the version is *between* the version of a > 
or >= condition on the left, and the version of a < or <= condition on the 
right, then it is accepted.  If the version exactly matches the version of 
a == or != condition, then it is either accepted (==) or rejected (!=).  If 
neither #3a nor #3b happens, then we continue "scanning left to right" (per 
#2).

I suspect that you will next come back and ask about this:

 >1.0, !=1.2, <2.0

at which point I will remind you that #3a says nothing about the pairs 
being adjacent.  :)


From jim at zope.com  Thu Jun 22 19:56:22 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 13:56:22 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
Message-ID: <E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>


On Jun 22, 2006, at 1:42 PM, Phillip J. Eby wrote:

> At 01:05 PM 6/22/2006 -0400, Jim Fulton wrote:
>
>> On Jun 22, 2006, at 12:49 PM, Phillip J. Eby wrote:
>>
>>> At 12:26 PM 6/22/2006 -0400, Jim Fulton wrote:
>>>> On Jun 21, 2006, at 5:11 PM, Phillip J. Eby wrote:
>>>>> """Requirement strings basically consist of a distribution  
>>>>> name, an
>>>>> optional list of "options" (more on this in a moment), and a
>>>>> comma- separated list of zero or more version conditions. Version
>>>>> conditions basically specify ranges of valid versions, using
>>>>> comparison operators. The version conditions you supply are sorted
>>>>> into ascending version order, and then scanned left to right until
>>>>> the package's version falls between a pair of > or >= and < or <=
>>>>> conditions, or exactly matches a == or != condition."""
>>>>
>>>> I don't think this is right. :)
>>>> ...
>>>> When scanning left to right, 1.9 matches !=1.1, so it should match
>>>> and,
>>>> since it is the highest version, it should be returned.  Either  
>>>> your
>>>> description
>>>> of the algorithm is incorrect or I'm misunderstanding it.
>>>
>>> You're missing the "exactly matches" part.  The relevant context
>>> is: "Until the package's version ... exactly matches a == or !=
>>> condition".  Perhaps making that "a == or != condition's version"
>>> would have been clearer, as that's what I meant by that phrase.
>>>
>>> Anyway, "1.9" does not exactly match the "1.1" in "!=1.1".
>>
>> Um, OK.  So I guess the idea is that we scan these things trying to
>> make a decision.
>> The decision is either match or not match.  Is that how I was  
>> supposed
>> to read the above quote?
>
> Um, no.  :)  The specification is:
>
> 1. "The version conditions you supply are sorted into ascending  
> version order"
>
> 2. "and then scanned left to right"  (from low to high version number)
>
> 3. "until the package's version"
>
> 3a. "falls between a pair of > or >= and < or <= conditions"
>
> 3b. "or exactly matches a == or != condition"
>
> In both #3a and #3b, we are saying that the package's version is  
> compared to the condition's version.  If the version is *between*  
> the version of a > or >= condition on the left, and the version of  
> a < or <= condition on the right, then it is accepted.  If the  
> version exactly matches the version of a == or != condition, then  
> it is either accepted (==) or rejected (!=).  If neither #3a nor  
> #3b happens, then we continue "scanning left to right" (per #2).

This is the clearest thing I have seen so far and matches what I was  
trying to say.

> I suspect that you will next come back and ask about this:
>
> >1.0, !=1.2, <2.0
>
> at which point I will remind you that #3a says nothing about the  
> pairs being adjacent.  :)

I'm not sure what you are trying to say.  I'm really trying to make  
sense of this.

I assume you are alluding to what I assume is the case that that  
there is an implicit
 >-infinity and <infinity, so the above example becomes:

   >-infinity, >1.0, !=1.2, <2.0, <infinity

Which isn't all that different from:

   >1.0, >2.0, <5.0, <6.0

so how is a case like this handled?  How should "fals between a pair  
of" be interpreted?
I would tend to expect innermost, otherwise:

   >1.0, <3.0, >5.0, <7.0

is ambiguous too.

I suggest that a more careful specification is needed.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 22 20:20:56 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 14:20:56 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
References: <5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
Message-ID: <FCF2414F-9692-4E79-B64E-E167C09E151F@zope.com>


On Jun 22, 2006, at 1:56 PM, Jim Fulton wrote:
>
> I assume you are alluding to what I assume is the case that that
> there is an implicit
> >-infinity and <infinity, so the above example becomes:

No, that can't be right.  If that were so, then

   ==1.2 would be meaningless

So I have no idea what you were trying to say.

The more I find out about the specification specification, the
less I understand it.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 21:00:40 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 15:00:40 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <FCF2414F-9692-4E79-B64E-E167C09E151F@zope.com>
References: <E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
Message-ID: <5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>

At 02:20 PM 6/22/2006 -0400, Jim Fulton wrote:

>On Jun 22, 2006, at 1:56 PM, Jim Fulton wrote:
>>
>>I assume you are alluding to what I assume is the case that that
>>there is an implicit
>> >-infinity and <infinity, so the above example becomes:
>
>No, that can't be right.  If that were so, then
>
>   ==1.2 would be meaningless
>
>So I have no idea what you were trying to say.

The infinities are implied only if they are needed to complete a pairing 
with an actual condition.


>The more I find out about the specification specification, the
>less I understand it.

Exactly, so stop trying to understand it, and you'll see how obvious it is.  :)

The implementation scans from left to right until it's *sure* that the 
version is either accepted or rejected.  Each condition is a point, a 
bound, or a point and a bound.  If the version matches the "point" part of 
condition, it's an exact match and you are *sure* of an accept, unless it's 
a "!=", in which case you're *sure* it's a reject.

If the version falls below an upper bound (< or <=), you are also *sure* 
that it's accepted.  If it is above an upper bound, it is *tentatively* 
rejected.

If the version falls below a lower bound (> or >=), you are *sure* that 
it's rejected.  If it is above a lower bound, it is *tentatively* accepted.

If you reach the end of the conditions without being "sure" of anything, 
then your most recent tentative acceptance or rejection is used.

A simple state machine is used to implement this:

state_machine = {
     #       =><
     '<' :  '--T',
     '<=':  'T-T',
     '>' :  'F+F',
     '>=':  'T+F',
     '==':  'T..',
     '!=':  'F++',
}

cmp() is used to determine whether the version is =, >, or < than the 
condition's version, and the appropriate row and column is pulled from the 
above table.  "T" means "sure accept", "F" means "sure reject", "+" means 
"tentative accept", and "-" means "tentative reject".  ("." means "don't 
care".)  The state machine simply compares versions until its sure or there 
are no more to compare.


From jim at zope.com  Thu Jun 22 21:20:26 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 15:20:26 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
References: <E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
Message-ID: <0CCBFD05-9A90-44F1-B2CE-43CA4E1013E1@zope.com>


On Jun 22, 2006, at 3:00 PM, Phillip J. Eby wrote:

> At 02:20 PM 6/22/2006 -0400, Jim Fulton wrote:
>
>> On Jun 22, 2006, at 1:56 PM, Jim Fulton wrote:
>>>
>>> I assume you are alluding to what I assume is the case that that
>>> there is an implicit
>>> >-infinity and <infinity, so the above example becomes:
>>
>> No, that can't be right.  If that were so, then
>>
>>   ==1.2 would be meaningless
>>
>> So I have no idea what you were trying to say.
>
> The infinities are implied only if they are needed to complete a  
> pairing with an actual condition.
>
>
>> The more I find out about the specification specification, the
>> less I understand it.
>
> Exactly, so stop trying to understand it, and you'll see how  
> obvious it is.  :)

Ah, if only it was.

> The implementation scans from left to right until it's *sure* that  
> the version is either accepted or rejected.  Each condition is a  
> point, a bound, or a point and a bound.  If the version matches the  
> "point" part of condition, it's an exact match and you are *sure*  
> of an accept, unless it's a "!=", in which case you're *sure* it's  
> a reject.
>
> If the version falls below an upper bound (< or <=), you are also  
> *sure* that it's accepted.  If it is above an upper bound, it is  
> *tentatively* rejected.

So given:    >1, <3, >5, <7
You are sure that 4 is accepted?

Given:  <2, <5
Is 3 accepted or rejected?

> If the version falls below a lower bound (> or >=), you are *sure*  
> that it's rejected.  If it is above a lower bound, it is  
> *tentatively* accepted.

So given:    >1, <3, >5, <7
You are sure that 2 is rejected?


> If you reach the end of the conditions without being "sure" of  
> anything, then your most recent tentative acceptance or rejection  
> is used.
>
> A simple state machine is used to implement this:
>
> state_machine = {
>     #       =><
>     '<' :  '--T',
>     '<=':  'T-T',
>     '>' :  'F+F',
>     '>=':  'T+F',
>     '==':  'T..',
>     '!=':  'F++',
> }
>
> cmp() is used to determine whether the version is =, >, or < than  
> the condition's version, and the appropriate row and column is  
> pulled from the above table.  "T" means "sure accept", "F" means  
> "sure reject", "+" means "tentative accept", and "-" means  
> "tentative reject".  ("." means "don't care".)  The state machine  
> simply compares versions until its sure or there are no more to  
> compare.

I don't understand the meaning of the values in the dictionary above.
Do the character positions reflect states somehow?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 21:44:43 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 15:44:43 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <0CCBFD05-9A90-44F1-B2CE-43CA4E1013E1@zope.com>
References: <5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>

At 03:20 PM 6/22/2006 -0400, Jim Fulton wrote:

>>The implementation scans from left to right until it's *sure* that
>>the version is either accepted or rejected.  Each condition is a
>>point, a bound, or a point and a bound.  If the version matches the
>>"point" part of condition, it's an exact match and you are *sure*
>>of an accept, unless it's a "!=", in which case you're *sure* it's
>>a reject.
>>
>>If the version falls below an upper bound (< or <=), you are also
>>*sure* that it's accepted.  If it is above an upper bound, it is
>>*tentatively* rejected.
>
>So given:    >1, <3, >5, <7
>You are sure that 4 is accepted?

Scanning left to right, >1 is a lower bound and 4 is above it, so that's a 
tentative accept.  Next we see <3, and it is an upper bound, and we fall 
below it, so it is a sure accept at that point, and we stop scanning.


>Given:  <2, <5
>Is 3 accepted or rejected?

<2, upper bound, 3 is above, tentative reject.  <5, upper bound, 3 is below 
it, sure accept.  (Intuitively -- at least for me -- <5 is paired 
with >-infinity here.)


>>If the version falls below a lower bound (> or >=), you are *sure*
>>that it's rejected.  If it is above a lower bound, it is
>>*tentatively* accepted.
>
>So given:    >1, <3, >5, <7
>You are sure that 2 is rejected?

 >1, lower bound, 3 is above, tentative accept.  <3, upper bound, 2 is 
below, sure accept, scanning stops.


>>A simple state machine is used to implement this:
>>
>>state_machine = {
>>     #       =><
>>     '<' :  '--T',
>>     '<=':  'T-T',
>>     '>' :  'F+F',
>>     '>=':  'T+F',
>>     '==':  'T..',
>>     '!=':  'F++',
>>}
>>
>>cmp() is used to determine whether the version is =, >, or < than
>>the condition's version, and the appropriate row and column is
>>pulled from the above table.  "T" means "sure accept", "F" means
>>"sure reject", "+" means "tentative accept", and "-" means
>>"tentative reject".  ("." means "don't care".)  The state machine
>>simply compares versions until its sure or there are no more to
>>compare.
>
>I don't understand the meaning of the values in the dictionary above.
>Do the character positions reflect states somehow?

It's a truth table: the rows are condition operators, and the columns are 
cmp() results.  It is simply a transcription of the rules about points and 
bounds that I spelled out verbally, reduced to a table lookup on the 
condition and the comparison results.


From jim at zope.com  Thu Jun 22 22:08:28 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 16:08:28 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
Message-ID: <C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>


On Jun 22, 2006, at 3:44 PM, Phillip J. Eby wrote:

> At 03:20 PM 6/22/2006 -0400, Jim Fulton wrote:
>
>>> The implementation scans from left to right until it's *sure* that
>>> the version is either accepted or rejected.  Each condition is a
>>> point, a bound, or a point and a bound.  If the version matches the
>>> "point" part of condition, it's an exact match and you are *sure*
>>> of an accept, unless it's a "!=", in which case you're *sure* it's
>>> a reject.
>>>
>>> If the version falls below an upper bound (< or <=), you are also
>>> *sure* that it's accepted.  If it is above an upper bound, it is
>>> *tentatively* rejected.
>>
>> So given:    >1, <3, >5, <7
>> You are sure that 4 is accepted?
>
> Scanning left to right, >1 is a lower bound and 4 is above it, so  
> that's a tentative accept.  Next we see <3, and it is an upper  
> bound, and we fall below it, so it is a sure accept at that point,  
> and we stop scanning.

Huh? 4 is below 3?


>
>> Given:  <2, <5
>> Is 3 accepted or rejected?
>
> <2, upper bound, 3 is above, tentative reject.  <5, upper bound, 3  
> is below it, sure accept.  (Intuitively -- at least for me -- <5 is  
> paired with >-infinity here.)

OK.  I think that many people would find this non-obvious.

>
>>> If the version falls below a lower bound (> or >=), you are *sure*
>>> that it's rejected.  If it is above a lower bound, it is
>>> *tentatively* accepted.
>>
>> So given:    >1, <3, >5, <7
>> You are sure that 2 is rejected?
>
> >1, lower bound, 2 is above, tentative accept.  <3, upper bound, 2  
> is below, sure accept, scanning stops.

Ah, so in your explanation above, "f the version falls below a lower  
bound"
only applies to a scanning position.  OK, that clarifies this case.

>
>>> A simple state machine is used to implement this:
>>>
>>> state_machine = {
>>>     #       =><
>>>     '<' :  '--T',
>>>     '<=':  'T-T',
>>>     '>' :  'F+F',
>>>     '>=':  'T+F',
>>>     '==':  'T..',
>>>     '!=':  'F++',
>>> }
>>>
>>> cmp() is used to determine whether the version is =, >, or < than
>>> the condition's version, and the appropriate row and column is
>>> pulled from the above table.  "T" means "sure accept", "F" means
>>> "sure reject", "+" means "tentative accept", and "-" means
>>> "tentative reject".  ("." means "don't care".)  The state machine
>>> simply compares versions until its sure or there are no more to
>>> compare.
>>
>> I don't understand the meaning of the values in the dictionary above.
>> Do the character positions reflect states somehow?
>
> It's a truth table: the rows are condition operators, and the  
> columns are cmp() results.  It is simply a transcription of the  
> rules about points and bounds that I spelled out verbally, reduced  
> to a table lookup on the condition and the comparison results.

OK, than makes sense.  So, with the requirement: >1, <3, >5, <7
So let's see, with 4, we get +-F and we reject it.  OK, that makes  
sense.

The state machine helps a lot.   My question is now answered.

I think that the fact that you need to understand a non-trivial  
algorithm
with a state machine to understand how non-trivial specifications are
interpreted is a problem.  Maybe it's enough to tell people "don't
use complex specifications", but maybe it would be better to use a
simpler system.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 22 22:12:36 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 16:12:36 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>
References: <5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>
Message-ID: <FCEA9F16-D28E-46B9-A865-9ABBC72D6512@zope.com>


On Jun 22, 2006, at 4:08 PM, Jim Fulton wrote:
...
>
>>
>>>> A simple state machine is used to implement this:
>>>>
>>>> state_machine = {
>>>>     #       =><
>>>>     '<' :  '--T',
>>>>     '<=':  'T-T',
>>>>     '>' :  'F+F',
>>>>     '>=':  'T+F',
>>>>     '==':  'T..',
>>>>     '!=':  'F++',
>>>> }
>>>>
>>>> cmp() is used to determine whether the version is =, >, or < than
>>>> the condition's version, and the appropriate row and column is
>>>> pulled from the above table.  "T" means "sure accept", "F" means
>>>> "sure reject", "+" means "tentative accept", and "-" means
>>>> "tentative reject".  ("." means "don't care".)  The state machine
>>>> simply compares versions until its sure or there are no more to
>>>> compare.
>>>
>>> I don't understand the meaning of the values in the dictionary  
>>> above.
>>> Do the character positions reflect states somehow?
>>
>> It's a truth table: the rows are condition operators, and the  
>> columns are cmp() results.  It is simply a transcription of the  
>> rules about points and bounds that I spelled out verbally, reduced  
>> to a table lookup on the condition and the comparison results.
>
> OK, than makes sense.  So, with the requirement: >1, <3, >5, <7
> So let's see, with 4, we get +-F and we reject it.  OK, that makes  
> sense.
>
> The state machine helps a lot.   My question is now answered.
>
> I think that the fact that you need to understand a non-trivial  
> algorithm
> with a state machine to understand how non-trivial specifications are
> interpreted is a problem.  Maybe it's enough to tell people "don't
> use complex specifications", but maybe it would be better to use a
> simpler system.

I also think that the complete algorithm, including the state machine
needs to be documented clearly.  Your original paragraph really
isn't adequate.

Maybe, you should document simple cases and refer to the full
complex model for non-trivial cases.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 22 22:36:10 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 16:36:10 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>
References: <5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>

At 04:08 PM 6/22/2006 -0400, Jim Fulton wrote:
>>>So given:    >1, <3, >5, <7
>>>You are sure that 4 is accepted?
>>
>>Scanning left to right, >1 is a lower bound and 4 is above it, so
>>that's a tentative accept.  Next we see <3, and it is an upper
>>bound, and we fall below it, so it is a sure accept at that point,
>>and we stop scanning.
>
>Huh? 4 is below 3?

Hm, somehow I got confused with 2.  It should say, <3 is an upper bound, 4 
is above it, tentative reject.  >5 is a lower bound, 4 is below it, it's a 
sure reject.  I think I copied 2 in from another example while editing.


>>>Given:  <2, <5
>>>Is 3 accepted or rejected?
>>
>><2, upper bound, 3 is above, tentative reject.  <5, upper bound, 3
>>is below it, sure accept.  (Intuitively -- at least for me -- <5 is
>>paired with >-infinity here.)
>
>OK.  I think that many people would find this non-obvious.

You keep missing the big picture.  The algorithm is designed to interpret a 
*meaningful* list of versions *specified by a human*.  Humans are usually 
not excessively redundant, and will thus tend to express a version 
requirement in the form of ranges with exceptions.

Humans tend to think that it is obvious that if they ask for a version <5, 
they do not need to also specify that the version be more than negative 
infinity, or even that it be greater than zero.  It is, after all, a 
*version* number.  :)

Similarly, if I ask for a version >10, I assume that you will understand I 
want it to be less than negative infinity.  :)

If it weren't for the fact that '.' and '-' are often used in version 
numbers, I might have used range expressions using '-' or '...', but these 
are also harder to read for the most common case where you want 
'>=someversion'.


>Ah, so in your explanation above, "f the version falls below a lower
>bound"
>only applies to a scanning position.  OK, that clarifies this case.

If you look back at my previous descriptions, the concept of scanning is 
repeated many times, including the simple 1-paragraph description.  It's 
all scanning, just like a human would do when reading a series of version 
requirements.


>I think that the fact that you need to understand a non-trivial
>algorithm
>with a state machine to understand how non-trivial specifications are
>interpreted is a problem.

It's only a problem if you try to reduce it to an algorithm.  If you simply 
try to *understand* a version requirement, it's really quite 
straightforward.  Think of what the human who wrote the requirement is saying:

"Um, let's see, it should be less than version 3, because that version's 
got a new API, and it should be more than version 1.2, because there were 
some bugfixes, and oh yeah, don't use version 1.6 because it was just plain 
hosed.  Oh, and 1.1.3 is also good."

The algorithm is designed to correctly interpret a person who is simply 
recording their wishes in this manner -- describing a series of acceptable 
or unacceptable ranges, and mentioning specific versions that are 
exceptions to the ranges.


>   Maybe it's enough to tell people "don't
>use complex specifications", but maybe it would be better to use a
>simpler system.

People don't normally express themselves redundantly -- especially 
programmers.  The target users of specifications are these humans, not the 
programmers trying to interpret the specification for specifications.  I 
did not anticipate that particular user and use case.  ;)


From pje at telecommunity.com  Thu Jun 22 22:52:27 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 16:52:27 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <FCEA9F16-D28E-46B9-A865-9ABBC72D6512@zope.com>
References: <C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<C4F1CE1F-F10B-4C86-B775-D37471449C6B@zope.com>
Message-ID: <5.1.1.6.0.20060622163639.03997278@sparrow.telecommunity.com>

At 04:12 PM 6/22/2006 -0400, Jim Fulton wrote:
>I also think that the complete algorithm, including the state machine
>needs to be documented clearly.  Your original paragraph really
>isn't adequate.

Only if you try to use it to interpret requirements that contain redundant 
information.


>Maybe, you should document simple cases and refer to the full
>complex model for non-trivial cases.

If by non-trivial you mean cases that contain redundant information, sure.

For example "<1,<2" is redundant, since "<1" implies "<2".  Only the "<1" 
is needed, and my assumption was that a person will not write "<1, <2" 
unless there is also a ">" or ">=" somewhere between the "<1" and the "<2" 
(in ascending version order).  Similarly, the only time multiple conditions 
for one version make sense is to have "<1,>1", but this is the same as 
"<>1" or "!=1".

If you simply write the shortest possible requirement that expresses the 
version range(s) and exceptions that you do or don't want, you will find 
that the 1-paragraph explanation will provide you with an easy and 
unambiguous interpretation: scanning left to right, see if the actual 
version is in one of the allowed ranges, or matches a point condition.

Translating this specification into an algorithm is tedious, but 
essentially a mechanical matter.  All of your questions, on the other hand, 
have been assuming that there is a simple algorithm that translates into 
something not necessarily simple to explain.  But it's the explanation 
that's simple, not the algorithm.  Version specifications are just ranges 
and exceptions.

Admittedly, it would have been better if Requirement had been designed to 
reject redundant input -- if only because it would've kept you from then 
asking what various redundant things *mean*.  ;-)  For the most part 
though, the implementation just ignores anything redundant.


From jim at zope.com  Thu Jun 22 22:54:03 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jun 2006 16:54:03 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
Message-ID: <A889B05B-634B-4772-B2CD-3807233E5F7F@zope.com>


I think you are making a mistake to assume that everyone has the same  
notion of obvious
that you do.  For example, someone could reasonably expect that ==1.0  
matches 1.1.1.
(I'm not advocating this interpretation, but simply pointing out that  
"obvious" is not
universal.)

In addition, people sometimes make typos.  A system that figures out  
what they "really meant"
rather than complain when they enter something that doesn't make  
sense isn't doing them any
favors.

In any case, I expect that having people build tools on top of  
setuptools is a use case
you anticipated.  For people to do that, they sometimes need precise  
specifications
of behavior.

Jim

On Jun 22, 2006, at 4:36 PM, Phillip J. Eby wrote:
> At 04:08 PM 6/22/2006 -0400, Jim Fulton wrote:
>>>> So given:    >1, <3, >5, <7
>>>> You are sure that 4 is accepted?
>>>
>>> Scanning left to right, >1 is a lower bound and 4 is above it, so
>>> that's a tentative accept.  Next we see <3, and it is an upper
>>> bound, and we fall below it, so it is a sure accept at that point,
>>> and we stop scanning.
>>
>> Huh? 4 is below 3?
>
> Hm, somehow I got confused with 2.  It should say, <3 is an upper  
> bound, 4 is above it, tentative reject.  >5 is a lower bound, 4 is  
> below it, it's a sure reject.  I think I copied 2 in from another  
> example while editing.
>
>
>>>> Given:  <2, <5
>>>> Is 3 accepted or rejected?
>>>
>>> <2, upper bound, 3 is above, tentative reject.  <5, upper bound, 3
>>> is below it, sure accept.  (Intuitively -- at least for me -- <5 is
>>> paired with >-infinity here.)
>>
>> OK.  I think that many people would find this non-obvious.
>
> You keep missing the big picture.  The algorithm is designed to  
> interpret a *meaningful* list of versions *specified by a human*.   
> Humans are usually not excessively redundant, and will thus tend to  
> express a version requirement in the form of ranges with exceptions.
>
> Humans tend to think that it is obvious that if they ask for a  
> version <5, they do not need to also specify that the version be  
> more than negative infinity, or even that it be greater than zero.   
> It is, after all, a *version* number.  :)
>
> Similarly, if I ask for a version >10, I assume that you will  
> understand I want it to be less than negative infinity.  :)
>
> If it weren't for the fact that '.' and '-' are often used in  
> version numbers, I might have used range expressions using '-' or  
> '...', but these are also harder to read for the most common case  
> where you want '>=someversion'.
>
>
>> Ah, so in your explanation above, "f the version falls below a lower
>> bound"
>> only applies to a scanning position.  OK, that clarifies this case.
>
> If you look back at my previous descriptions, the concept of  
> scanning is repeated many times, including the simple 1-paragraph  
> description.  It's all scanning, just like a human would do when  
> reading a series of version requirements.
>
>
>> I think that the fact that you need to understand a non-trivial
>> algorithm
>> with a state machine to understand how non-trivial specifications are
>> interpreted is a problem.
>
> It's only a problem if you try to reduce it to an algorithm.  If  
> you simply try to *understand* a version requirement, it's really  
> quite straightforward.  Think of what the human who wrote the  
> requirement is saying:
>
> "Um, let's see, it should be less than version 3, because that  
> version's got a new API, and it should be more than version 1.2,  
> because there were some bugfixes, and oh yeah, don't use version  
> 1.6 because it was just plain hosed.  Oh, and 1.1.3 is also good."
>
> The algorithm is designed to correctly interpret a person who is  
> simply recording their wishes in this manner -- describing a series  
> of acceptable or unacceptable ranges, and mentioning specific  
> versions that are exceptions to the ranges.
>
>
>>   Maybe it's enough to tell people "don't
>> use complex specifications", but maybe it would be better to use a
>> simpler system.
>
> People don't normally express themselves redundantly -- especially  
> programmers.  The target users of specifications are these humans,  
> not the programmers trying to interpret the specification for  
> specifications.  I did not anticipate that particular user and use  
> case.  ;)
>

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Fri Jun 23 00:51:04 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Jun 2006 18:51:04 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <A889B05B-634B-4772-B2CD-3807233E5F7F@zope.com>
References: <5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060622184137.01f811f0@sparrow.telecommunity.com>

At 04:54 PM 6/22/2006 -0400, Jim Fulton wrote:
>In any case, I expect that having people build tools on top of
>setuptools is a use case
>you anticipated.  For people to do that, they sometimes need precise
>specifications
>of behavior.

Of course -- but I didn't anticipate that they would need anything more 
than the ability to test whether a version matches a requirement.  Given 
all of the tradeoffs involved, including at least:

* similarity to version specifiers used in other packaging tools (e.g. RPM)
* ease of comprehension and simplicity of expressing common use cases

I chose to lean in favor of an approach that handles reasonable cases 
reasonably, and provides repeatable answers always.

Also, at some point, one does what one can with the available 
manpower.  Bob Ippolito was the only person besides me who was actively 
involved in the design process at the time -- and he was kind of like, 
"whatever, man" on the subject.  :)  It would've been nice to have had your 
participation then.  Now, we'll pretty much have to live with what we've got.


From public-dstml at giss.ath.cx  Fri Jun 23 08:01:27 2006
From: public-dstml at giss.ath.cx (AGiss)
Date: Fri, 23 Jun 2006 08:01:27 +0200
Subject: [Distutils] setuptools : easy_install error on berlios
In-Reply-To: <5.1.1.6.0.20060619084407.030f8d38@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
	<20060618225829.GA31234@gissehel.mine.nu>
	<5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060619084407.030f8d38@sparrow.telecommunity.com>
Message-ID: <20060623060127.GA382@gissehel.mine.nu>

On Mon, Jun 19, 2006 at 08:44:48AM -0400, Phillip J. Eby wrote:
> At 10:51 AM 6/19/2006 +0200, AGiss wrote:
> >(The workaround is to not provide the download URL on berlios in
> >setup.py. That's what I'll do in next release.)
> 
> FYI, you don't need a new release; just edit the setup.py and run "setup.py 
> register" -- that will update your PyPI listing immediately, with no need 
> for a new release.

Yes, but setup.py comes from the file tagged for a release, so if I 
change it, I need to make a new release, not because of a technical 
tool, but because I don't like to have a version that is not coherent
between repository/website/pypi/tarball/win32exe/etc.


From jim at zope.com  Fri Jun 23 13:47:28 2006
From: jim at zope.com (Jim Fulton)
Date: Fri, 23 Jun 2006 07:47:28 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <5.1.1.6.0.20060622184137.01f811f0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622184137.01f811f0@sparrow.telecommunity.com>
Message-ID: <81D69CA7-0BA9-4FED-A508-9985F38FBC70@zope.com>


On Jun 22, 2006, at 6:51 PM, Phillip J. Eby wrote:

> At 04:54 PM 6/22/2006 -0400, Jim Fulton wrote:
>> In any case, I expect that having people build tools on top of
>> setuptools is a use case
>> you anticipated.  For people to do that, they sometimes need precise
>> specifications
>> of behavior.
>
> Of course -- but I didn't anticipate that they would need anything  
> more than the ability to test whether a version matches a requirement.

I think I've explained, and I thought you understood why that wasn't  
adequate for me.

...

> Also, at some point, one does what one can with the available  
> manpower.  Bob Ippolito was the only person besides me who was  
> actively involved in the design process at the time -- and he was  
> kind of like, "whatever, man" on the subject.  :)  It would've been  
> nice to have had your participation then.  Now, we'll pretty much  
> have to live with what we've got.

1. You should know that no system is perfect.  You can't anticipate  
all requirements up
      front.  Surely, you want people to use this system beyond those  
who invented it.
     It's also not practical for all of us to get involved in every  
design at it's inception.

2. I have tried not to ask you to implement my use cases.  There are  
a number of reasons for
     this, including:

     - I don't know fully what my use cases are.  After all, use  
cases change with experience.

     - I don't want to disrupt setuptools development. I don't think  
setuptools should
       try to meet everyone's needs.  I do think that people should  
be able to meet their
       own needs on top of setuptools.  I think setuptools has stood  
up very well in this regard.

I really haven't asked for much. I believe the only changes you've  
made for me so far have been
bug fixes.  I don't intend to ask for much.  The only non-bug-fix  
change I've asked for
so far has been to declare public a pre-exiting API so that I can  
implement what I want
myself.

Of course, I've also asked that some semantics be documented.  I  
needed this documentation
for what I was doing.  I also believe that documenting things well  
sheds important light on
them.  You learn things about decisions to make by explaining these  
decisions to other.

I think setuptools is pretty cool and I have sung it's praises in  
many forums, admittedly,
even before I know what I was talking about. ;)  I don't regret  
that.  I'm building a "buildout"
system on top of setup tools that will help it meet use cases that it  
doesn't meet now. I
appreciate your advice and support during this process process and  
look forward to
more of the same in the future.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Fri Jun 23 17:37:49 2006
From: jim at zope.com (Jim Fulton)
Date: Fri, 23 Jun 2006 11:37:49 -0400
Subject: [Distutils] Is there any harm in clearing sys.path_importer_cache
	from time to time?
Message-ID: <CFD6F97A-D7CF-46A3-A1B1-8036437A1F35@zope.com>


I have a script based on setuptools that invokes easy install to  
check for
and download newer distributions.  I allow users to specify whether or
not they want unzipped downloads.  As a result, easy_install will  
sometimes
replace a zip file with a directory or the other way around.  This  
leads to
problems in the invoking program because the sys.path_importer_cache
is stale.  Does anyone know if it is safe to clear the cache from  
time to
time?  If so, I'd be inclined to clear it any time I invoke  
easy_install.

If it isn't safe, I'll probably just forego the feature of letting users
force downloads to be unzipped, which I may do anyway to be safe. :)

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Fri Jun 23 17:39:45 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jun 2006 11:39:45 -0400
Subject: [Distutils] Requirent specifiers specified? :)
In-Reply-To: <81D69CA7-0BA9-4FED-A508-9985F38FBC70@zope.com>
References: <5.1.1.6.0.20060622184137.01f811f0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060621170630.0338a480@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622124458.034d1a50@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622133405.04c76d48@sparrow.telecommunity.com>
	<E2BB11B5-2836-48B6-A675-45C425E60A13@zope.com>
	<5.1.1.6.0.20060622144048.01eedf08@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622153858.03bfc458@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622161624.03b17f80@sparrow.telecommunity.com>
	<5.1.1.6.0.20060622184137.01f811f0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060623113216.01eb2c08@sparrow.telecommunity.com>

At 07:47 AM 6/23/2006 -0400, Jim Fulton wrote:
>I think I've explained, and I thought you understood why that wasn't
>adequate for me.

I do - which is why I've promised to make 'specs' a documented attribute, 
so you can do what you need to do.


>I really haven't asked for much. I believe the only changes you've
>made for me so far have been
>bug fixes.  I don't intend to ask for much.  The only non-bug-fix
>change I've asked for
>so far has been to declare public a pre-exiting API so that I can
>implement what I want
>myself.

Yep, already agreed to do that.


>Of course, I've also asked that some semantics be documented.

They are documented, or rather, were documented, in what I believe is a 
sufficient manner, if all you want to do is understand how to read and 
write version requirements.  I'm going to restore that documentation, and 
make more explicit some of the parts you misunderstood, as well as 
expanding it in general.

I don't think, however, that the state-machine specification or 
step-by-step algorithm is appropriate for the API documentation or the 
setuptools manual.  Such information, if it goes anywhere, would be in the 
"formats.txt" file (in the doc/ directory of setuptools 0.7's source tree).

My comments about wishing you'd been involved in the design were simply 
that: I wish you had.  I wish more people had been involved, in 
general.  That's all I meant.


From pje at telecommunity.com  Fri Jun 23 17:40:46 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jun 2006 11:40:46 -0400
Subject: [Distutils] setuptools : easy_install error on berlios
In-Reply-To: <20060623060127.GA382@gissehel.mine.nu>
References: <5.1.1.6.0.20060619084407.030f8d38@sparrow.telecommunity.com>
	<5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
	<20060618225829.GA31234@gissehel.mine.nu>
	<5.1.1.6.0.20060618204141.03210dd8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060619084407.030f8d38@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060623114010.03b24b58@sparrow.telecommunity.com>

At 08:01 AM 6/23/2006 +0200, AGiss wrote:
>On Mon, Jun 19, 2006 at 08:44:48AM -0400, Phillip J. Eby wrote:
> > At 10:51 AM 6/19/2006 +0200, AGiss wrote:
> > >(The workaround is to not provide the download URL on berlios in
> > >setup.py. That's what I'll do in next release.)
> >
> > FYI, you don't need a new release; just edit the setup.py and run 
> "setup.py
> > register" -- that will update your PyPI listing immediately, with no need
> > for a new release.
>
>Yes, but setup.py comes from the file tagged for a release, so if I
>change it, I need to make a new release, not because of a technical
>tool, but because I don't like to have a version that is not coherent
>between repository/website/pypi/tarball/win32exe/etc.

Well, you could also just edit the PyPI listing via the web interface to 
remove the URL in the meantime.  :)


From pje at telecommunity.com  Fri Jun 23 17:53:54 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jun 2006 11:53:54 -0400
Subject: [Distutils] Is there any harm in clearing
 sys.path_importer_cache from time to time?
In-Reply-To: <CFD6F97A-D7CF-46A3-A1B1-8036437A1F35@zope.com>
Message-ID: <5.1.1.6.0.20060623114836.03b34ca0@sparrow.telecommunity.com>

At 11:37 AM 6/23/2006 -0400, Jim Fulton wrote:
>I have a script based on setuptools that invokes easy install to
>check for
>and download newer distributions.  I allow users to specify whether or
>not they want unzipped downloads.  As a result, easy_install will
>sometimes
>replace a zip file with a directory or the other way around.  This
>leads to
>problems in the invoking program because the sys.path_importer_cache
>is stale.  Does anyone know if it is safe to clear the cache from
>time to
>time?  If so, I'd be inclined to clear it any time I invoke
>easy_install.

Hm.  easy_install should be doing this itself...  oh wait, it's clearing 
the *zip directory cache*, which fixes the problem of overwriting a zip 
with a zip.  You're having a problem when overwriting a zip with a 
directory.  Now I understand why sometimes I still get reports of behavior 
that resembles this problem, but then they aren't reproducible.

I'll fix easy_install to also clear the path importer cache for a path 
entry when it clears the zip directory cache for that entry.

(And to answer your general question, it's fine to clear the 
path_importer_cache, per PEP 302's advice to do so whenever you add new 
import hooks.  It just slows down the next import search a bit.)


From jim at zope.com  Fri Jun 23 18:47:20 2006
From: jim at zope.com (Jim Fulton)
Date: Fri, 23 Jun 2006 12:47:20 -0400
Subject: [Distutils] Is there any harm in clearing
	sys.path_importer_cache from time to time?
In-Reply-To: <5.1.1.6.0.20060623114836.03b34ca0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060623114836.03b34ca0@sparrow.telecommunity.com>
Message-ID: <085D014F-C680-48E7-A88E-730A8873A97B@zope.com>


On Jun 23, 2006, at 11:53 AM, Phillip J. Eby wrote:

> At 11:37 AM 6/23/2006 -0400, Jim Fulton wrote:
>> I have a script based on setuptools that invokes easy install to
>> check for
>> and download newer distributions.  I allow users to specify  
>> whether or
>> not they want unzipped downloads.  As a result, easy_install will
>> sometimes
>> replace a zip file with a directory or the other way around.  This
>> leads to
>> problems in the invoking program because the sys.path_importer_cache
>> is stale.  Does anyone know if it is safe to clear the cache from
>> time to
>> time?  If so, I'd be inclined to clear it any time I invoke
>> easy_install.
>
> Hm.  easy_install should be doing this itself...  oh wait, it's  
> clearing the *zip directory cache*, which fixes the problem of  
> overwriting a zip with a zip.  You're having a problem when  
> overwriting a zip with a directory.  Now I understand why sometimes  
> I still get reports of behavior that resembles this problem, but  
> then they aren't reproducible.

Actually, I happened to be having a problem overriding a directory  
with a zip.  This was in a test in which I demonstrate being able to  
unzip and went back to not unzipping in a later example.  After the  
egg is reinstalled unzipped, the cache entry has None, left over from  
finding the directory before.

This was in a doctest which invoked my buildout script as a  
subprocess, which then invoked  easy_install as a subprocess.  When  
the buildout script started, the egg in question was a directory.  It  
then ran easy_install, which replaced the directory with a zip file.  
Then the buildout script tried to load the distribution for the  
installed file, which failed.

I haven't been able to manually find the right combination to make  
easy_install override a distribution manually, so I'm a bit puzzled.   
If you wish me to pursue that I will.


> I'll fix easy_install to also clear the path importer cache for a  
> path entry when it clears the zip directory cache for that entry.

OK. Note that that won't help me, because the cache I'm affected by  
is in the parent
process.

> (And to answer your general question, it's fine to clear the  
> path_importer_cache, per PEP 302's advice to do so whenever you add  
> new import hooks.  It just slows down the next import search a bit.)

Cool, cause that solves my immediate problem. :)

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Fri Jun 23 22:10:37 2006
From: jim at zope.com (Jim Fulton)
Date: Fri, 23 Jun 2006 16:10:37 -0400
Subject: [Distutils] Specification for package indexes?
Message-ID: <D1721FA8-BF0D-476D-B27A-A0C7318EDBC1@zope.com>


What form must an index take to be usable with setuptools?

Is there anything documented how such a beast should be organized?
What should it's pages should look like?  Is there any special
pattern setuptools (easy_install?) looks for to find pages with  
download links?
Or does it search any link given?

Also, I've noticed that if you gibe a location with find-links that has
links to distributions for the thing you're looking for, it you can  
specify
pretty much anything as an index.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Fri Jun 23 22:30:52 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jun 2006 16:30:52 -0400
Subject: [Distutils] Specification for package indexes?
In-Reply-To: <D1721FA8-BF0D-476D-B27A-A0C7318EDBC1@zope.com>
Message-ID: <5.1.1.6.0.20060623162538.01eabc20@sparrow.telecommunity.com>

At 04:10 PM 6/23/2006 -0400, Jim Fulton wrote:

>What form must an index take to be usable with setuptools?
>
>Is there anything documented how such a beast should be organized?
>What should it's pages should look like?  Is there any special
>pattern setuptools (easy_install?) looks for to find pages with
>download links?
>Or does it search any link given?
>
>Also, I've noticed that if you gibe a location with find-links that has
>links to distributions for the thing you're looking for, it you can
>specify
>pretty much anything as an index.

That's correct.  If all your packages can just be linked from a single 
page, that's more than enough to satisfy.

However, if you want to give easy_install an alternate --index-url to use 
in place of PyPI, it must meet the four simple requirements described here:

     http://mail.python.org/pipermail/catalog-sig/2005-June/000654.html

These are the only conditions that easy_install has for the organization of 
an "index" in the PyPI sense.  Note that they can all be accomodated via 
static HTML pages.

I'm considering adding XML-RPC support to easy_install in 0.7, 
though.  PyPI now has a nice XML-RPC API that is more responsive than the 
web UI, and it supports case-insensitive partial match searches, making it 
suitable for easy_install to query when a typed-in name doesn't exactly 
match the spelling of a PyPI entry.


From jim at zope.com  Fri Jun 23 22:51:20 2006
From: jim at zope.com (Jim Fulton)
Date: Fri, 23 Jun 2006 16:51:20 -0400
Subject: [Distutils] Specification for package indexes?
In-Reply-To: <5.1.1.6.0.20060623162538.01eabc20@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060623162538.01eabc20@sparrow.telecommunity.com>
Message-ID: <D5993AEE-1BB6-46A9-95D0-1031BC5EF58A@zope.com>


On Jun 23, 2006, at 4:30 PM, Phillip J. Eby wrote:

> At 04:10 PM 6/23/2006 -0400, Jim Fulton wrote:
>
>> What form must an index take to be usable with setuptools?
>>
>> Is there anything documented how such a beast should be organized?
>> What should it's pages should look like?  Is there any special
>> pattern setuptools (easy_install?) looks for to find pages with
>> download links?
>> Or does it search any link given?
>>
>> Also, I've noticed that if you gibe a location with find-links  
>> that has
>> links to distributions for the thing you're looking for, it you can
>> specify
>> pretty much anything as an index.
>
> That's correct.  If all your packages can just be linked from a  
> single page, that's more than enough to satisfy.
>
> However, if you want to give easy_install an alternate --index-url  
> to use in place of PyPI, it must meet the four simple requirements  
> described here:
>
>     http://mail.python.org/pipermail/catalog-sig/2005-June/000654.html
>
> These are the only conditions that easy_install has for the  
> organization of an "index" in the PyPI sense.  Note that they can  
> all be accomodated via static HTML pages.

That's a lot of screen scraping. :)

It would be good to capture this as part of the documentation IMO

> I'm considering adding XML-RPC support to easy_install in 0.7,  
> though.  PyPI now has a nice XML-RPC API that is more responsive  
> than the web UI, and it supports case-insensitive partial match  
> searches, making it suitable for easy_install to query when a typed- 
> in name doesn't exactly match the spelling of a PyPI entry.

I think that would be much better.

I think it would also be helpful to have an option (-I) to
disable index search.  There will be cases where people
don't want to put distributions in PyPI and won't want to have
to implement an index server.  The find-links mechanism seems to
work fine most of the time.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jorge.vargas at gmail.com  Sat Jun 24 09:04:30 2006
From: jorge.vargas at gmail.com (Jorge Vargas)
Date: Sat, 24 Jun 2006 03:04:30 -0400
Subject: [Distutils] conditional external packages dependancies with
	setuptools
Message-ID: <32822fe60606240004nd8f40f6t6f7898141ea87173@mail.gmail.com>

hi all

I'm new to creating my own setuptools projects, although I have work with
other people projects before, so I'm not that lost here.

but What I want to accomplish is having dependancies on some packages. for
example my project will have diferent database backends so I want to depend
on pysqlite or mysql-python depending on which the user wants as his/her
backend. I'm going to have lots of diferent backends, even some that are not
databases, so this will be a crutial thing for me or I'll end up with very
big dist files per release, which will mostly be waste if the user never
uses them.

is something like this possible with setuptools now? is it plan to be
possible in the future? if so any estimates on when will this be?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20060624/cb522fd3/attachment.htm 

From pje at telecommunity.com  Sat Jun 24 18:51:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 24 Jun 2006 12:51:00 -0400
Subject: [Distutils] conditional external packages dependancies with
 setuptools
In-Reply-To: <32822fe60606240004nd8f40f6t6f7898141ea87173@mail.gmail.com
 >
Message-ID: <5.1.1.6.0.20060624124845.03a8a1e0@sparrow.telecommunity.com>

At 03:04 AM 6/24/2006 -0400, Jorge Vargas wrote:
>but What I want to accomplish is having dependancies on some packages. for 
>example my project will have diferent database backends so I want to 
>depend on pysqlite or mysql-python depending on which the user wants as 
>his/her backend. I'm going to have lots of diferent backends, even some 
>that are not databases, so this will be a crutial thing for me or I'll end 
>up with very big dist files per release, which will mostly be waste if the 
>user never uses them.
>
>is something like this possible with setuptools now?

Yes:

http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras


From pje at telecommunity.com  Sat Jun 24 21:59:50 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 24 Jun 2006 15:59:50 -0400
Subject: [Distutils] conditional external packages dependancies with
 setuptools
In-Reply-To: <32822fe60606241029p39814dfame3b071feef01c954@mail.gmail.co
 m>
References: <5.1.1.6.0.20060624124845.03a8a1e0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060624124845.03a8a1e0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060624155657.01ecaad8@sparrow.telecommunity.com>

At 01:29 PM 6/24/2006 -0400, Jorge Vargas wrote:
>On 6/24/06, Phillip J. Eby 
><<mailto:pje at telecommunity.com>pje at telecommunity.com> wrote:
>>At 03:04 AM 6/24/2006 -0400, Jorge Vargas wrote:
>> >but What I want to accomplish is having dependancies on some packages. for
>> >example my project will have diferent database backends so I want to
>> >depend on pysqlite or mysql-python depending on which the user wants as
>> >his/her backend. I'm going to have lots of diferent backends, even some
>> >that are not databases, so this will be a crutial thing for me or I'll end
>> >up with very big dist files per release, which will mostly be waste if the
>> >user never uses them.
>> >
>> >is something like this possible with setuptools now?
>>
>>Yes:
>>
>><http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras>http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras 
>>
>
>there it says that I can set  them and then project depending on mine will 
>be able to add them to their install_required, but what I want is my 
>project to have it optional
>
>there is a reference that they can be pass as commands to easy_install but 
>can't find the switch at 
><http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options>http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options

If you have an optional feature called "bar" for your package named "foo", 
you would use:

     easy_install foo[bar]

if there's also a "baz" feature and the user wants to install that too, 
they can use:

     easy_install foo[bar,baz]


>I'm looking at something like the gentoo's USE flags system. where I can 
>have a script or config file that each install will use with the optional 
>packages they want, and distribute that with a sane number of defaults.

I'm not familiar with this, although it sounds like something that might be 
useful to add in future.  Presumably this could be done by standardizing 
names of "extras" and then having an option for easy_install (which then 
would be configurable via the normal distutils config files).  I'll keep 
that in mind for 0.7.


From jorge.vargas at gmail.com  Sat Jun 24 22:14:07 2006
From: jorge.vargas at gmail.com (Jorge Vargas)
Date: Sat, 24 Jun 2006 16:14:07 -0400
Subject: [Distutils] conditional external packages dependancies with
	setuptools
In-Reply-To: <32822fe60606241313o3e8b2287s63d309afed9e3483@mail.gmail.com>
References: <5.1.1.6.0.20060624124845.03a8a1e0@sparrow.telecommunity.com>
	<5.1.1.6.0.20060624155657.01ecaad8@sparrow.telecommunity.com>
	<32822fe60606241313o3e8b2287s63d309afed9e3483@mail.gmail.com>
Message-ID: <32822fe60606241314w2b079e61xe9b4f81d5a7f7983@mail.gmail.com>

sorry forgot to send this to the list.

---------- Forwarded message ----------
From: Jorge Vargas <jorge.vargas at gmail.com>
Date: Jun 24, 2006 4:13 PM
Subject: Re: [Distutils] conditional external packages dependancies with
setuptools
To: "Phillip J. Eby" <pje at telecommunity.com>

On 6/24/06, Phillip J. Eby <pje at telecommunity.com> wrote:

> At 01:29 PM 6/24/2006 -0400, Jorge Vargas wrote:
> >On 6/24/06, Phillip J. Eby
> ><<mailto:pje at telecommunity.com>pje at telecommunity.com > wrote:
> >>At 03:04 AM 6/24/2006 -0400, Jorge Vargas wrote:
> >> >but What I want to accomplish is having dependancies on some packages.
> for
> >> >example my project will have diferent database backends so I want to
> >> >depend on pysqlite or mysql-python depending on which the user wants
> as
> >> >his/her backend. I'm going to have lots of diferent backends, even
> some
> >> >that are not databases, so this will be a crutial thing for me or I'll
> end
> >> >up with very big dist files per release, which will mostly be waste if
> the
> >> >user never uses them.
> >> >
> >> >is something like this possible with setuptools now?
> >>
> >>Yes:
> >>
> >><http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras>http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras
> >>
> >
> >there it says that I can set  them and then project depending on mine
> will
> >be able to add them to their install_required, but what I want is my
> >project to have it optional
> >
> >there is a reference that they can be pass as commands to easy_install
> but
> >can't find the switch at
> ><http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options
> >http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options
>
> If you have an optional feature called "bar" for your package named "foo",
> you would use:
>
>      easy_install foo[bar]
>
> if there's also a "baz" feature and the user wants to install that too,
> they can use:
>
>      easy_install foo[bar,baz]


ahh yes now I remenber TurboGears uses those.

will there be a way to create defaults for this or maybe provide diferent
"instalation scripts" for example
<mypackage>-minimal.py equivalente of easy_install <mypackage>
<mypackage>-std.py equivalente of easy_install <mypackage> [for,bar]
<mypackage>-full.py equivalente of easy_install <mypackage>
[for,bar,baz,bazz]


>I'm looking at something like the gentoo's USE flags system. where I can
> >have a script or config file that each install will use with the optional
> >packages they want, and distribute that with a sane number of defaults.
>
> I'm not familiar with this, although it sounds like something that might
> be
> useful to add in future.  Presumably this could be done by standardizing
> names of "extras" and then having an option for easy_install (which then
> would be configurable via the normal distutils config files).  I'll keep
> that in mind for 0.7.


yes indeed is one of the most powerfull feature of emerge. the way it works
on gentoo is as follows.

there is a global config file (/etc/make.conf) that has all the global
useflags the user has set
then each package can set it's own set of useflags

so when someone installs something the engine sets its global flags and then
looks for package specific (package.use) and then for commandline flags,
although commandline in gentoo is deprecated due to errors in stuff like
package A I needs foo compile with bar support... anyway.

I'll love to see something like this for setuptools this will make it easier
to integrate into gentoo since one of the biggest problems right now with
easy_install in gentoo is the lack of flexibility in the instalation.

ohh I almost forgot a big part of emerge is build on python, so maybe even
some code could be borrow.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20060624/607032ac/attachment.htm 

From jim at zope.com  Sun Jun 25 19:24:11 2006
From: jim at zope.com (Jim Fulton)
Date: Sun, 25 Jun 2006 13:24:11 -0400
Subject: [Distutils] setup.py develop ignores --no-deps option
Message-ID: <2ECF0C2B-8EF8-41FC-95EF-0494772A3566@zope.com>


AFAICT, with 0.6b3, setup.py develop ignores the --no-deps option.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Wed Jun 28 22:37:19 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 16:37:19 -0400
Subject: [Distutils] README(.txt)?
Message-ID: <44B8A376-8BB0-4E7A-A484-0EF13352F12E@zope.com>


If I don't have a README or README.txt file next to my setup.py file
when I create an edd, I'll get the warning:

   warning: manifest_maker: standard file not found: should have one  
of README, README.txt

I don't like warnings.  So I create a REAME,txt file and then  
generate an
egg and, guess what?  It's not included.

Bug or feature?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Wed Jun 28 22:39:25 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 16:39:25 -0400
Subject: [Distutils] Best practices for creating eggs?
Message-ID: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>


Has anyone written up any best practices for creating eggs?

How do people handle documentation?  Our packages tend to have  
documentation files included as doctests, but It's not clear how folks
are expected to get to them, especially if the eggs are zip files.

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Wed Jun 28 22:46:15 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 16:46:15 -0400
Subject: [Distutils] Weird PyPI Behavior
Message-ID: <CE18F3F6-F4AF-4E74-8FC7-5EF064105079@zope.com>


If I do something like:

   python2.4 setup.py register bdist_egg upload

as recommended in the setuptools documentation, I end up  with 2  
"releases" in PyPi.  One has all my descriptive information and the  
other has the egg.  One or more will or won't be hidden according  
some rule that I haven't been able to divine.  If the release with  
the meta data (e.g. description, author, etc.) is not hidden, then  
easy_install can't seem to find the eggs.  At least this is true  
today.  I thought it wasn't true yesterday, but I don't trust my  
recollection.  If I hide the release with the descriptive  
information, then easy_install can find the eggs, but most of my meta  
data isn't shown.  Has anyone else seen this behavior, or am I just  
insane.

For now, I'm not uploading eggs to PyPI, which is a shame, because  
setuptools sure made it convenient. :(

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Wed Jun 28 23:04:57 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 17:04:57 -0400
Subject: [Distutils] README(.txt)?
In-Reply-To: <5.1.1.6.0.20060628165437.02e8a258@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060628165437.02e8a258@sparrow.telecommunity.com>
Message-ID: <19A501B5-8528-4BBE-9CF2-83173F0E91AA@zope.com>


On Jun 28, 2006, at 4:56 PM, Phillip J. Eby wrote:

> At 04:37 PM 6/28/2006 -0400, Jim Fulton wrote:
>
>> If I don't have a README or README.txt file next to my setup.py file
>> when I create an edd, I'll get the warning:
>>
>>    warning: manifest_maker: standard file not found: should have one
>> of README, README.txt
>>
>> I don't like warnings.  So I create a REAME,txt file and then
>> generate an
>> egg and, guess what?  It's not included.
>>
>> Bug or feature?
>
> The warning comes from the distutils, in a method that I call to  
> create a source manifest.  I don't know of any way to suppress it,  
> although I'll look again.  In normal distutils usage, you would  
> only see it when doing an "sdist", as it wants a README for the  
> source distribution.

Is the right thing to do to suppress the warning? Or to include the  
README
in the egg?

I'm not clear on what the README is for.  If it contains installation  
instructions for a source distribution, then obviously, it shouldn't  
be included.

FWIW, this is most annoying to me when doing setup.py develop.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Wed Jun 28 23:16:00 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 17:16:00 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
Message-ID: <38037B8A-837B-48A7-BBB3-D4A9B752CAB4@zope.com>


On Jun 28, 2006, at 4:58 PM, Phillip J. Eby wrote:

> At 04:46 PM 6/28/2006 -0400, Jim Fulton wrote:
>
>> If I do something like:
>>
>>    python2.4 setup.py register bdist_egg upload
>>
>> as recommended in the setuptools documentation, I end up  with 2
>> "releases" in PyPi.  One has all my descriptive information and the
>> other has the egg.
>
> I've never seen this behavior myself, but I would guess that  
> "register" and "upload" are disagreeing about the package version  
> number, perhaps due to you having --tag-svn-revision set on your  
> egg_info command for the bdist_egg.

Interesting.  I thought that --tag-svn-revision was a common/recommended
practice.  I followed the advice to have a setup.cfg like:

[egg_info]
tag_build = .dev
tag_svn_revision = 1

But I noticed that the release with the meta data didn't have the  
subversion information or the .dev tag, which is consistent with  
register not being aware of the extra information.

Is this a bug? :)

If it isn't, then I recommend that you either stop recommending  that  
people use setup.cfg this way or stop recommending that they upload  
their files to PyPI.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Wed Jun 28 22:56:42 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 16:56:42 -0400
Subject: [Distutils] README(.txt)?
In-Reply-To: <44B8A376-8BB0-4E7A-A484-0EF13352F12E@zope.com>
Message-ID: <5.1.1.6.0.20060628165437.02e8a258@sparrow.telecommunity.com>

At 04:37 PM 6/28/2006 -0400, Jim Fulton wrote:

>If I don't have a README or README.txt file next to my setup.py file
>when I create an edd, I'll get the warning:
>
>    warning: manifest_maker: standard file not found: should have one
>of README, README.txt
>
>I don't like warnings.  So I create a REAME,txt file and then
>generate an
>egg and, guess what?  It's not included.
>
>Bug or feature?

The warning comes from the distutils, in a method that I call to create a 
source manifest.  I don't know of any way to suppress it, although I'll 
look again.  In normal distutils usage, you would only see it when doing an 
"sdist", as it wants a README for the source distribution.


From pje at telecommunity.com  Wed Jun 28 22:58:47 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 16:58:47 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <CE18F3F6-F4AF-4E74-8FC7-5EF064105079@zope.com>
Message-ID: <5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>

At 04:46 PM 6/28/2006 -0400, Jim Fulton wrote:

>If I do something like:
>
>    python2.4 setup.py register bdist_egg upload
>
>as recommended in the setuptools documentation, I end up  with 2
>"releases" in PyPi.  One has all my descriptive information and the
>other has the egg.

I've never seen this behavior myself, but I would guess that "register" and 
"upload" are disagreeing about the package version number, perhaps due to 
you having --tag-svn-revision set on your egg_info command for the bdist_egg.


From pje at telecommunity.com  Wed Jun 28 23:30:49 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 17:30:49 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <38037B8A-837B-48A7-BBB3-D4A9B752CAB4@zope.com>
References: <5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>

At 05:16 PM 6/28/2006 -0400, Jim Fulton wrote:
>Interesting.  I thought that --tag-svn-revision was a common/recommended
>practice.

It *is* -- but not for releases being sent to PyPI.  a dev-rNNNN release is 
a development snapshot rather than an official release.


>   I followed the advice to have a setup.cfg like:
>
>[egg_info]
>tag_build = .dev
>tag_svn_revision = 1
>
>But I noticed that the release with the meta data didn't have the
>subversion information or the .dev tag, which is consistent with
>register not being aware of the extra information.
>
>Is this a bug? :)

Probably.  :)  A workaround would be to either put "egg_info" before 
"register", or to move the "register" after the "bdist_egg".  I'll add this 
to the bug list nonetheless.


>If it isn't, then I recommend that you either stop recommending  that
>people use setup.cfg this way or stop recommending that they upload
>their files to PyPI.

Do you really need to upload every development snapshot to PyPI?  Common 
practice for the dev-rNNNN stuff is that you either remove it from 
setup.cfg when you create a release branch, or temporarily remove it when 
issuing a release snapshot.

Admittedly, this procedure is awkward, and in 0.7 there will be some kind 
of options you can give to egg_info like --no-revision and --release-build 
to indicate that the svn revision and build tag should be omitted.  The use 
in this case would be something like "setup.py egg_info --release-build 
--no-revision register bdist_egg upload" -- which of course you could 
create a shortcut for (using "setup.py alias"), maybe something like "release".


From pje at telecommunity.com  Wed Jun 28 23:41:26 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 17:41:26 -0400
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
Message-ID: <5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>

At 04:39 PM 6/28/2006 -0400, Jim Fulton wrote:

>Has anyone written up any best practices for creating eggs?
>
>How do people handle documentation?

I basically make web pages for them, or if it's small enough of a project, 
stick it in the long_description of setup() (usually by reading it from a 
README.txt file).  Otherwise, I make the documentation home page be easily 
reachable via information in PyPI (e.g., make it the 'url' in 'setup()').

I assume that most people will read the docs on the web, and that if they 
want the documentation source, they will download the "sdist" distribution 
that I always upload alongside of the eggs.


>  Our packages tend to have
>documentation files included as doctests, but It's not clear how folks
>are expected to get to them, especially if the eggs are zip files.

I just post them to the web.  In some cases, I have scripts that "svn up" 
and rebuild the docs nightly.  For others with relatively short docs, I 
just upload them by hand.

There's lots of opportunity to add more distutils or setuptools extensions 
for doc processing.  I imagine that Zope projects will probably want to 
have some declarative or discoverable way to get docs built, so you can 
automate the process of updating browsable versions of the doc, or for that 
matter assist people in building their own docs from source.

(By the way, "easy_install -eb somedir foo" will download and extract (or 
check out of Subversion) a copy of "foo" to "somedir/foo", so you are not 
limited to what is placed in an egg for execution/import purposes; I assume 
the typical user will read docs online, and thus most installations and 
downloads do not need to carry substantial documentation.)


From jim at zope.com  Wed Jun 28 23:42:27 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 17:42:27 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
Message-ID: <08EE5D04-5D41-4F55-9008-DA080CA69956@zope.com>


On Jun 28, 2006, at 5:30 PM, Phillip J. Eby wrote:

> At 05:16 PM 6/28/2006 -0400, Jim Fulton wrote:
>> Interesting.  I thought that --tag-svn-revision was a common/ 
>> recommended
>> practice.
>
> It *is* -- but not for releases being sent to PyPI.  a dev-rNNNN  
> release is a development snapshot rather than an official release.
>
>
>>   I followed the advice to have a setup.cfg like:
>>
>> [egg_info]
>> tag_build = .dev
>> tag_svn_revision = 1
>>
>> But I noticed that the release with the meta data didn't have the
>> subversion information or the .dev tag, which is consistent with
>> register not being aware of the extra information.
>>
>> Is this a bug? :)
>
> Probably.  :)  A workaround would be to either put "egg_info"  
> before "register", or to move the "register" after the  
> "bdist_egg".  I'll add this to the bug list nonetheless.

Interesting. That's good to know.  If it works, I'm happy. I suggest  
noting this in the docs.

>> If it isn't, then I recommend that you either stop recommending  that
>> people use setup.cfg this way or stop recommending that they upload
>> their files to PyPI.
>
> Do you really need to upload every development snapshot to PyPI?

If I want people to be able to download it, I have to upload it  
somewhere.

I have two choices, upload it to pypi, or create my own download  
area.  I would do one or the other, not both.  It seems confusing, to  
me and to consumers to have two download areas, my own and pypi's.   
I'd rather use PyPi's because it's integration with setup.py is so  
convenient.

OTOH, if it is considered bad form to upload dev releases to PyPi, I  
can just write
helper scripts to upload somewhere else.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 29 00:09:32 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 18:09:32 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <08EE5D04-5D41-4F55-9008-DA080CA69956@zope.com>
References: <5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>

At 05:42 PM 6/28/2006 -0400, Jim Fulton wrote:
>If I want people to be able to download it, I have to upload it
>somewhere.

Not necessarily.  If you have a pure-Python package, and your target 
audience has Subversion, you can provide a URL on PyPI that always obtains 
the most recent possible development snapshot.  For example, "easy_install 
setuptools==dev" will check out the setuptools trunk and install an egg 
from it, properly tagged as a development release with the subversion 
revision.  "easy_install setuptools==dev06" will check out the head of the 
0.6 maintenance branch.

Of course, if you want to provide *built* daily snapshots, then this 
doesn't help you.


>I have two choices, upload it to pypi, or create my own download
>area.  I would do one or the other, not both.  It seems confusing, to
>me and to consumers to have two download areas, my own and pypi's.
>I'd rather use PyPi's because it's integration with setup.py is so
>convenient.
>
>OTOH, if it is considered bad form to upload dev releases to PyPi, I
>can just write
>helper scripts to upload somewhere else.

I don't know if there's a consensus that it's bad form.  I do think that it 
would quickly become annoying to readers of Planet Python or even just the 
PyPI RSS feed if every Zope project was churning out snapshots of every SVN 
checkin, every day.  :)

As it is, there are lots of projects that churn their PyPI releases more 
often than I'd prefer to see going by, but ah well.  Moderation in all 
things is best, I suppose.  :)

I won't say you *should* do one thing or the other, but I will note that 
the "rotate" command of setuptools is meant to help clean up directories 
containing automated nightly builds or svn snapshots.  So, if you have 
servers whose job it is to do continuous builds, they can do stuff like 
"bdist_egg -b /output-dir rotate -k5 -m.egg -d /output-dir" to put the 
built egg in /output-dir and keep the 5 newest eggs.  Several projects can 
safely share the same output directory, and the output directory doesn't 
need to be served by anything fancy; see e.g. 
http://peak.telecommunity.com/snapshots/ for an example of such a directory.



From jim at zope.com  Thu Jun 29 00:27:11 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 18:27:11 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
Message-ID: <BAC66891-8BC0-4107-AC2B-FD3654F85D4D@zope.com>


On Jun 28, 2006, at 6:09 PM, Phillip J. Eby wrote:

> At 05:42 PM 6/28/2006 -0400, Jim Fulton wrote:
>> If I want people to be able to download it, I have to upload it
>> somewhere.
>
> Not necessarily.  If you have a pure-Python package, and your  
> target audience has Subversion, you can provide a URL on PyPI that  
> always obtains the most recent possible development snapshot.  For  
> example, "easy_install setuptools==dev" will check out the  
> setuptools trunk and install an egg from it, properly tagged as a  
> development release with the subversion revision.  "easy_install  
> setuptools==dev06" will check out the head of the 0.6 maintenance  
> branch.

The docs are rather unclear how to to set this up.  In any case, it  
isn't my
intent to have people track subversion

> Of course, if you want to provide *built* daily snapshots, then  
> this doesn't help you.

Or to do daily builds.

>
>> I have two choices, upload it to pypi, or create my own download
>> area.  I would do one or the other, not both.  It seems confusing, to
>> me and to consumers to have two download areas, my own and pypi's.
>> I'd rather use PyPi's because it's integration with setup.py is so
>> convenient.
>>
>> OTOH, if it is considered bad form to upload dev releases to PyPi, I
>> can just write
>> helper scripts to upload somewhere else.
>
> I don't know if there's a consensus that it's bad form.  I do think  
> that it would quickly become annoying to readers of Planet Python  
> or even just the PyPI RSS feed if every Zope project was churning  
> out snapshots of every SVN checkin, every day.  :)

I didn't realize that this information was being broadcast.  I  
imagine I am
unintentionally annoying people with all of my experiments. :(

It isn't my intention to provide daily builds.  I just happen wanted  
to make
quick and dirty releases while stuff is new.  These are like alpha  
release, but I'm
leveraging the revision tagging to automate generation of release  
numbers.

> As it is, there are lots of projects that churn their PyPI releases  
> more often than I'd prefer to see going by, but ah well.   
> Moderation in all things is best, I suppose.  :)
>
> I won't say you *should* do one thing or the other, but I will note  
> that the "rotate" command of setuptools is meant to help clean up  
> directories containing automated nightly builds or svn snapshots.   
> So, if you have servers whose job it is to do continuous builds,  
> they can do stuff like "bdist_egg -b /output-dir rotate -k5 -m.egg - 
> d /output-dir" to put the built egg in /output-dir and keep the 5  
> newest eggs.  Several projects can safely share the same output  
> directory, and the output directory doesn't need to be served by  
> anything fancy; see e.g. http://peak.telecommunity.com/snapshots/  
> for an example of such a directory.

That's not what I'm up to. :)

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From jim at zope.com  Thu Jun 29 01:30:46 2006
From: jim at zope.com (Jim Fulton)
Date: Wed, 28 Jun 2006 19:30:46 -0400
Subject: [Distutils] How does one register a framework classifier?
Message-ID: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>


How does one register a new Framework classifier?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From pje at telecommunity.com  Thu Jun 29 01:47:10 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 19:47:10 -0400
Subject: [Distutils] How does one register a framework classifier?
In-Reply-To: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>
Message-ID: <5.1.1.6.0.20060628194525.028bb070@sparrow.telecommunity.com>

At 07:30 PM 6/28/2006 -0400, Jim Fulton wrote:

>How does one register a new Framework classifier?

I believe you submit an entry to the PyPI request tracker; but you might 
want to check on the catalog-sig, which is where PyPI itself is discussed.

There is some outstanding sentiment that it should be easy to add 
classifiers, but also some pushback that they're supposed to be "Trove" 
classifiers, whatever those are (only partly joking).  Whatever they are, 
methinks they're a little too deep, hierarchically speaking, for 
Python.  Flat is better than nested and all that.
:)


From pje at telecommunity.com  Thu Jun 29 05:28:39 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 28 Jun 2006 23:28:39 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <BAC66891-8BC0-4107-AC2B-FD3654F85D4D@zope.com>
References: <5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060628232354.02dfb478@sparrow.telecommunity.com>

At 06:27 PM 6/28/2006 -0400, Jim Fulton wrote:
>I didn't realize that this information was being broadcast.  I
>imagine I am
>unintentionally annoying people with all of my experiments. :(

Don't worry about it; there's always a certain amount of release flutter 
taking place for other packages.  It just takes a while to get the 
procedures down.  I was more worried about the prospect of one release per 
commmit per project, or even daily snapshots of every project in Zope.  :)


>It isn't my intention to provide daily builds.  I just happen wanted
>to make
>quick and dirty releases while stuff is new.  These are like alpha
>release, but I'm
>leveraging the revision tagging to automate generation of release
>numbers.

Hm.  Well, different strokes for different folks.  I prefer to bump the 
alpha or beta number, since a short number is easier for humans to refer 
to; it's annoying to have to ask if someone has 0.6a9dev-r19684, versus 
something like 0.6a9snap3.  The latter can be done with tag_build='snap3' 
in your setup.cfg, modulo the "register" issue we've already discussed.

But of course your practices are up to you; I just mention this as an idea 
to make communications about your projects a little easier, at the cost of 
editing setup.cfg (or overriding --tag-build on the command line) when you 
do a release.


From p.f.moore at gmail.com  Thu Jun 29 11:37:58 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 29 Jun 2006 10:37:58 +0100
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
References: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
Message-ID: <79990c6b0606290237w6d307e0mce1da6180e43dcf4@mail.gmail.com>

On 6/28/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> I assume that most people will read the docs on the web, and that if they
> want the documentation source, they will download the "sdist" distribution
> that I always upload alongside of the eggs.

I work offline sufficiently often that not having local documentation
is frustrating. There's no standard for local docs, which is a
nuisance, and makes for an inconsistent story between different
packages, but I'd be concerned if setuptools made it more difficult to
bundle local docs.

An example - cx_Oracle provides a full set of HTML documentation which
the bdist_wininst installer drops in C:\Python24\cx_Oracle-doc. The
one time I tried using easy_install to convert this to an egg, the
documentation got missed out. I've never retried the experiment,
however, so things could well have changed, as this was quite a while
ago.

And of course, eggs which get installed as zip files don't really
offer anywhere to *put* documentation which is accessible (web
browsers can't load HTML out of zip files, for example).

Paul.

From jim at zope.com  Thu Jun 29 12:19:24 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 29 Jun 2006 06:19:24 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <5.1.1.6.0.20060628232354.02dfb478@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628232354.02dfb478@sparrow.telecommunity.com>
Message-ID: <9D8D5EC9-FC6B-4083-BD9A-E2032E5D644D@zope.com>


On Jun 28, 2006, at 11:28 PM, Phillip J. Eby wrote
...
>> It isn't my intention to provide daily builds.  I just happen wanted
>> to make
>> quick and dirty releases while stuff is new.  These are like alpha
>> release, but I'm
>> leveraging the revision tagging to automate generation of release
>> numbers.
>
> Hm.  Well, different strokes for different folks.  I prefer to bump  
> the alpha or beta number, since a short number is easier for humans  
> to refer to; it's annoying to have to ask if someone has 0.6a9dev- 
> r19684, versus something like 0.6a9snap3.  The latter can be done  
> with tag_build='snap3' in your setup.cfg, modulo the "register"  
> issue we've already discussed.
>
> But of course your practices are up to you; I just mention this as  
> an idea to make communications about your projects a little easier,  
> at the cost of editing setup.cfg (or overriding --tag-build on the  
> command line) when you do a release.

I'm happy to get input.

1. If I'm going to edit setup.cfg, I might as well edit setup.py

2. A reason I want to automate this is a feat that I'll forget to  
edit setup.py
     or setup.cfg and overrite existing releases (or fail to do an  
update without
     realizing it because the existing release doesn't get overwritten.

Another issue is that I see us moving toward lots of fairly fine-grained
packages and I want to keep the ceremony pretty low.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From dangoor at gmail.com  Thu Jun 29 12:52:30 2006
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Jun 2006 06:52:30 -0400
Subject: [Distutils] How does one register a framework classifier?
In-Reply-To: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>
References: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>
Message-ID: <6470F2F9-5925-48C4-A0D4-310824720444@gmail.com>

On Jun 28, 2006, at 7:30 PM, Jim Fulton wrote:

>
> How does one register a new Framework classifier?

FWIW, I've been using keywords a lot, because I can add arbitrary  
ones whenever I wish and still do searches. TurboGears has a package  
listing (the CogBin) which is built from Cheeseshop queries.

Kevin

From jim at zope.com  Thu Jun 29 12:58:49 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 29 Jun 2006 06:58:49 -0400
Subject: [Distutils] How does one register a framework classifier?
In-Reply-To: <6470F2F9-5925-48C4-A0D4-310824720444@gmail.com>
References: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>
	<6470F2F9-5925-48C4-A0D4-310824720444@gmail.com>
Message-ID: <6CE519FB-23EB-4CB9-8C4F-8AB80602C7CE@zope.com>


On Jun 29, 2006, at 6:52 AM, Kevin Dangoor wrote:

> On Jun 28, 2006, at 7:30 PM, Jim Fulton wrote:
>
>>
>> How does one register a new Framework classifier?
>
> FWIW, I've been using keywords a lot, because I can add arbitrary  
> ones whenever I wish and still do searches. TurboGears has a  
> package listing (the CogBin) which is built from Cheeseshop queries.

Sure, but I assume that TurboGears also uses classifiers since 4 out  
of the 5 Framework
classifiers are for TurboGears.  Well, actually, 3 out of the 5.  One  
is for TruboGears,
which, I suppose, if for a related framework. ;)

If we think that classifiers shouldn't be used for Frameworks, then  
lets get rid of the
existing Framework classifiers.  Otherwise, there needs to be a way  
to add new
ones as new frameworks emerge or old frameworks make more use of PyPI.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org




From dangoor at gmail.com  Thu Jun 29 13:18:56 2006
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Jun 2006 07:18:56 -0400
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <79990c6b0606290237w6d307e0mce1da6180e43dcf4@mail.gmail.com>
References: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<79990c6b0606290237w6d307e0mce1da6180e43dcf4@mail.gmail.com>
Message-ID: <8BD1B994-3E06-45BD-BFAA-51785944076A@gmail.com>

On Jun 29, 2006, at 5:37 AM, Paul Moore wrote:

> On 6/28/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>> I assume that most people will read the docs on the web, and that  
>> if they
>> want the documentation source, they will download the "sdist"  
>> distribution
>> that I always upload alongside of the eggs.
>
> I work offline sufficiently often that not having local documentation
> is frustrating. There's no standard for local docs, which is a
> nuisance, and makes for an inconsistent story between different
> packages, but I'd be concerned if setuptools made it more difficult to
> bundle local docs.

I'll second this (or is it third, since Jim was asking something  
similar)? Being able to get at a package's documentation after it's  
installed would be nice.

If you use docstrings, at least you can use help() and pydoc for  
apidocs.

Kevin

From dangoor at gmail.com  Thu Jun 29 13:31:09 2006
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Jun 2006 07:31:09 -0400
Subject: [Distutils] How does one register a framework classifier?
In-Reply-To: <6CE519FB-23EB-4CB9-8C4F-8AB80602C7CE@zope.com>
References: <C761C2B1-E3C1-44C6-B2E6-D3D716346C29@zope.com>
	<6470F2F9-5925-48C4-A0D4-310824720444@gmail.com>
	<6CE519FB-23EB-4CB9-8C4F-8AB80602C7CE@zope.com>
Message-ID: <7D8EFD13-B751-41EC-A7B6-C6BF663F471C@gmail.com>

On Jun 29, 2006, at 6:58 AM, Jim Fulton wrote:

>
> On Jun 29, 2006, at 6:52 AM, Kevin Dangoor wrote:
>
>> On Jun 28, 2006, at 7:30 PM, Jim Fulton wrote:
>>
>>>
>>> How does one register a new Framework classifier?
>>
>> FWIW, I've been using keywords a lot, because I can add arbitrary  
>> ones whenever I wish and still do searches. TurboGears has a  
>> package listing (the CogBin) which is built from Cheeseshop queries.
>
> Sure, but I assume that TurboGears also uses classifiers since 4  
> out of the 5 Framework
> classifiers are for TurboGears.  Well, actually, 3 out of the 5.   
> One is for TruboGears,
> which, I suppose, if for a related framework. ;)
>
> If we think that classifiers shouldn't be used for Frameworks, then  
> lets get rid of the
> existing Framework classifiers.  Otherwise, there needs to be a way  
> to add new
> ones as new frameworks emerge or old frameworks make more use of PyPI.

Yes, we do use the classifiers as well, and I agree that it should be  
possible to add new ones.  The classifiers do have value at this  
point because they provide the "browse" interface at the Cheeseshop.

Kevin

From p.f.moore at gmail.com  Thu Jun 29 14:53:49 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 29 Jun 2006 13:53:49 +0100
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <8BD1B994-3E06-45BD-BFAA-51785944076A@gmail.com>
References: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<79990c6b0606290237w6d307e0mce1da6180e43dcf4@mail.gmail.com>
	<8BD1B994-3E06-45BD-BFAA-51785944076A@gmail.com>
Message-ID: <79990c6b0606290553v5719de58p459272437fb77f5d@mail.gmail.com>

On 6/29/06, Kevin Dangoor <dangoor at gmail.com> wrote:
> If you use docstrings, at least you can use help() and pydoc for
> apidocs.

Agreed, but existing HTML (or HTML-help, or other format)
documentation often covers more than docstrings alone do (tutorial
stuff, user guides, etc). I'd rather not discourage this type of
documentation either.

Paul.

From pobrien at orbtech.com  Thu Jun 29 15:08:30 2006
From: pobrien at orbtech.com (Patrick K. O'Brien)
Date: Thu, 29 Jun 2006 08:08:30 -0500
Subject: [Distutils] Optimize option for windows exe scripts?
Message-ID: <44A3D0CE.8090602@orbtech.com>

Is there a way to have the Windows .exe created by setuptools use
optimized Python, like "python -O myscript.py"?

http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation

In particular, I'd like this option to work with a project where I am
working out of Subversion using "python setup.py develop".

-- 
Patrick K. O'Brien
Orbtech       http://www.orbtech.com
Schevo        http://www.schevo.org
Louie         http://www.pylouie.org

From pje at telecommunity.com  Thu Jun 29 16:34:24 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 10:34:24 -0400
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <79990c6b0606290237w6d307e0mce1da6180e43dcf4@mail.gmail.com
 >
References: <5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>

At 10:37 AM 6/29/2006 +0100, Paul Moore wrote:
>On 6/28/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>>I assume that most people will read the docs on the web, and that if they
>>want the documentation source, they will download the "sdist" distribution
>>that I always upload alongside of the eggs.
>
>I work offline sufficiently often that not having local documentation
>is frustrating. There's no standard for local docs, which is a
>nuisance, and makes for an inconsistent story between different
>packages, but I'd be concerned if setuptools made it more difficult to
>bundle local docs.
>
>An example - cx_Oracle provides a full set of HTML documentation which
>the bdist_wininst installer drops in C:\Python24\cx_Oracle-doc. The
>one time I tried using easy_install to convert this to an egg, the
>documentation got missed out. I've never retried the experiment,
>however, so things could well have changed, as this was quite a while
>ago.
>
>And of course, eggs which get installed as zip files don't really
>offer anywhere to *put* documentation which is accessible (web
>browsers can't load HTML out of zip files, for example).

Note that I said above that I always put the documentation in an sdist 
form; to obtain a package's source distribution, use:


     easy_install -e -b somedir arg...

Where 'arg...' is some specification of a package, and 'somedir' is the 
parent directory of where you want the package's source to be unpacked or 
checked out.  If you requested e.g. 'cx_Oracle', you will end up with a 
'somedir/cx_oracle' directory containing the extracted source 
distribution.  You can then decide what to do with any docs in it.

A standard for how to install documentation would be great, because then 
you could run the docinstall command or whatever it's called on the source 
directory.  For that matter, easy_install could be made to do it also.

(Of course, this could presumably be included in eggs also, but I'm 
thinking of it being a separate operation from installing the eggs, just 
because it's increasingly common to be installing a package to satisfy some 
other package's dependency -- not because you actually intend to use that 
package directly.  OTOH, relatively small packages compared to their 
documentation size might just want to throw it in.)


From p.f.moore at gmail.com  Thu Jun 29 17:13:07 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 29 Jun 2006 16:13:07 +0100
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>
References: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>
Message-ID: <79990c6b0606290813r4f0e2af1j7e5a4fcc6b7b20e8@mail.gmail.com>

On 6/29/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> Note that I said above that I always put the documentation in an sdist
> form;

Noted. The distinction I'm trying to make is that with bdist_wininst,
it's part of the binary installer - a single install gets the module
and its documentation, with no decisions needed by me.

I've no experience with other formats, so I'm afraid I can't compare
this with bdist_rpm, bdist_deb (spot-checking bdist_msi, it looks like
that packages the docs, as well). However, the package author clearly
got existing distutils mechanisms to include the docs,  so it's
possible.

I'm not trying to say this is a huge failing, just pointing out that
it's something that existing formats do, that eggs might be interested
in supporting.

> You can then decide what to do with any docs in it.

Having to make a decision at all rather than them "just being there"
is the (small) barrier I'm pointing out :-)

> A standard for how to install documentation would be great, because then
> you could run the docinstall command or whatever it's called on the source
> directory.  For that matter, easy_install could be made to do it also.

Agreed. But in the absence of a standard, supporting package authors'
existing approaches, which work with other distutils mechanisms, seems
like a reasonable requirement.

Paul.

From p.f.moore at gmail.com  Thu Jun 29 17:21:38 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 29 Jun 2006 16:21:38 +0100
Subject: [Distutils] Problem between bdist_msi and egg-info (Python 2.5b1)
Message-ID: <79990c6b0606290821n2d7770c1t7156d123551edfeb@mail.gmail.com>

If I try to build a bdist_msi installer for a trivial module, using
Python 2.5b1, I get an error.

It looks like a problem with the interaction of the new (in 2.5)
egg-info stuff and the bdist_msi code - but I have no further ideas.
Here is the setup.py

from distutils.core import setup
setup(
    name='test',
    version='1.0',
    py_modules=['test'],
)

And here is the build output:

>python setup.py bdist_msi
running bdist_msi
running build
running build_py
creating build
creating build\lib
copying test.py -> build\lib
installing to build\bdist.win32\msi
running install_lib
creating build\bdist.win32
creating build\bdist.win32\msi
creating build\bdist.win32\msi\Lib
creating build\bdist.win32\msi\Lib\site-packages
copying build\lib\test.py -> build\bdist.win32\msi\Lib\site-packages
running install_egg_info
Writing build\bdist.win32\msi\Lib\site-packages\test-1.0-py2.5.egg-info
creating dist
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    py_modules=['test'],
  File "D:\Apps\Python25\Lib\distutils\core.py", line 151, in setup
    dist.run_commands()
  File "D:\Apps\Python25\Lib\distutils\dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "D:\Apps\Python25\Lib\distutils\dist.py", line 994, in run_command
    cmd_obj.run()
  File "D:\Apps\Python25\Lib\distutils\command\bdist_msi.py", line 235, in run
    self.add_files()
  File "D:\Apps\Python25\Lib\distutils\command\bdist_msi.py", line
263, in add_files
    key = dir.add_file(file)
  File "D:\Apps\Python25\Lib\msilib\__init__.py", line 343, in add_file
    language, attributes, sequence)])
  File "D:\Apps\Python25\Lib\msilib\__init__.py", line 115, in add_data
    raise MSIError("Could not insert "+repr(values)+" into "+table)
_msi.MSIError: Could not insert [(None, 'site_packages',
'TEST-1~1.EGG|test-1.0-py2.5.egg-info', 186L, None, None, 512, 1)]
into File

I can't log a SF bug at the moment, as SF isn't accepting logins, but
maybe someone could take a look and judge where the problem lies. Then
when I can log a bug, I can put it in the right category :-)

Thanks,
Paul.

From pje at telecommunity.com  Thu Jun 29 21:45:49 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 15:45:49 -0400
Subject: [Distutils] unrecognised swig options ?
In-Reply-To: <186780900606100049k7921eb15g4618bd9507854f70@mail.gmail.co
 m>
References: <186780900606070759m8314ae2vdea0c9373f739fc9@mail.gmail.com>
	<186780900606070759m8314ae2vdea0c9373f739fc9@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060629154455.03b31288@sparrow.telecommunity.com>

At 03:49 PM 6/10/2006 +0800, Risky Martin wrote:
>2006/6/7, Risky Martin <<mailto:risky.martin at gmail.com> 
>risky.martin at gmail.com>:
>>I am trying to use disutils's features with 'SWIG'... but hit
>>a rather strange problem.
>>
>>To reproduce it:
>>
>>setup.py
>>------------------------------------------------------
>>from distutils.core import setup, Extension
>>
>>e = Extension("_foo",
>>               sources = ["foo.i", ],
>>               swig_opts = ["-outdir .", ])
>>
>>
>>setup(name="foo",
>>       ext_modules=[e, ])
>>-------------------------------------------------------
>>
>>building fails like:
>>
>> > python setup.py build
>>running build
>>running build_ext
>>building '_foo' extension
>>swigging foo.i to foo_wrap.c
>>swig -python -outdir . -o foo_wrap.c foo.i
>>swig error : Unrecognized option -outdir .
>>Use 'swig -help' for available options.
>>error: command 'swig' failed with exit status 1
>>
>>but a copy/paste of the swig command shows that '-outdir' *is* recognised...
>> > swig -python -outdir . -o foo_wrap.c foo.i
>>Unable to find file 'foo.i'.

Are you sure that you are running the same swig on the command line that 
Python is running?


From pje at telecommunity.com  Thu Jun 29 21:47:43 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 15:47:43 -0400
Subject: [Distutils] Why using distutils.core.setup requires Python
 sources ?
In-Reply-To: <1150293569.8913.5.camel@localhost>
Message-ID: <5.1.1.6.0.20060629154655.03b50e00@sparrow.telecommunity.com>

At 03:59 PM 6/14/2006 +0200, J?r?me Bouat wrote:
>I don't understand why distutils.core.setup needs Python sources (a
>missing Makefile on my distro).
>
>I did not found in the mail archives.
>
>Could you explain me ?

Because the Makefile contains configuration options for how Python was 
built, and these options are read by the distutils.


From pje at telecommunity.com  Thu Jun 29 22:28:03 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 16:28:03 -0400
Subject: [Distutils] Optimize option for windows exe scripts?
In-Reply-To: <44A3D0CE.8090602@orbtech.com>
Message-ID: <5.1.1.6.0.20060629162425.03b40278@sparrow.telecommunity.com>

At 08:08 AM 6/29/2006 -0500, Patrick K. O'Brien wrote:
>Is there a way to have the Windows .exe created by setuptools use
>optimized Python, like "python -O myscript.py"?

Not at the moment, no.

I'm thinking that future versions of setuptools will need to provide more 
control over script details like interpreter options, but I haven't done 
much design on it yet.  For now, I would suggest setting the PYTHONOPTIMIZE 
environment variable as a workaround.


From pje at telecommunity.com  Thu Jun 29 22:42:06 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 16:42:06 -0400
Subject: [Distutils] Weird PyPI Behavior
In-Reply-To: <9D8D5EC9-FC6B-4083-BD9A-E2032E5D644D@zope.com>
References: <5.1.1.6.0.20060628232354.02dfb478@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628165710.038d7938@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628172318.03ee6fb8@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628175602.02b73508@sparrow.telecommunity.com>
	<5.1.1.6.0.20060628232354.02dfb478@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060629162832.03b183f0@sparrow.telecommunity.com>

At 06:19 AM 6/29/2006 -0400, Jim Fulton wrote:
>1. If I'm going to edit setup.cfg, I might as well edit setup.py

Note that you needn't edit anything if you use the --tag-build option, e.g.:

     python setup.py egg_info --tag-build=snap3 register sdist bdist_egg upload

Assuming that setup.cfg normally contains:

    [egg_info]
    tag_build=dev

and you're not using tag-svn-revision by default.

In a future version of setuptools (maybe 0.6c1), there'll also be a command 
line option to turn *off* --tag-svn-revision.


>2. A reason I want to automate this is a feat that I'll forget to
>edit setup.py
>     or setup.cfg and overrite existing releases (or fail to do an
>update without
>     realizing it because the existing release doesn't get overwritten.
>
>Another issue is that I see us moving toward lots of fairly fine-grained
>packages and I want to keep the ceremony pretty low.

Yes, I once forgot to do this for setuptools itself, which is why I want 
options to remove the tags, then I can just add those notes to my release 
script.

However, I also have an automated tool to do version edits across many 
files in a project.  A simple ZConfig-based file describes what version 
numbers appear in what formats in what files, and is made executable via a 
#! line that runs a Python program that accepts various version-changing 
commands.  I can run "./version incr build" to bump from 0.6a1 to 0.6a2, 
for example, or './version incr minor' to bump from 0.6 to 0.7.  The tool 
then edits the relevant .py and .txt files, including setup.py itself.

Here's the 'version' script for setuptools; it might give you some ideas 
for making a similar tool for Zope projects.  You'll notice that it also 
edits my release.sh script, so that it knows what files it will be 
uploading to various servers...


#!/usr/local/bin/invoke /usr/local/bin/c6peak version-config

# This is a PEAK 'version' tool configuration file, that's
# also executable.  PJE uses it to bump version numbers in
# the various parts of the project without having to edit them
# by hand.  The current version is stored in the version.dat
# file.

# These are not the droids you're looking for.  You can go on
# about your business...

<Scheme default>
     DefaultFormat full
     part major
     part minor
     part status choice alpha beta "release candidate" final
     part build
     part date   timestamp

     <Formats>
     trailer   remap status "a%(build)s" "b%(build)s" "c%(build)s" 
"%(dot-maint)s"
     dot-maint optional build ".%(build)s"
     full      "%(major)s.%(minor)s %(status)s %(build)s"
     short     "%(major)s.%(minor)s%(trailer)s"
     </Formats>
</Scheme>

<Module>
     Name setuptools

     <Edit>
     File setup.py
     File ez_setup.py
     Match 'VERSION = "%(short)s"'
     </Edit>

     <Edit>
     File release.sh
     Match 'VERSION="%(short)s"'
     </Edit>

     <Edit>
     File setuptools/__init__.py
     Match "__version__ = '%(short)s'"
     </Edit>

</Module>


From pje at telecommunity.com  Thu Jun 29 22:43:59 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 16:43:59 -0400
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <79990c6b0606290813r4f0e2af1j7e5a4fcc6b7b20e8@mail.gmail.co
 m>
References: <5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>
	<BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060629164229.03783d38@sparrow.telecommunity.com>

At 04:13 PM 6/29/2006 +0100, Paul Moore wrote:
>Agreed. But in the absence of a standard, supporting package authors'
>existing approaches, which work with other distutils mechanisms, seems
>like a reasonable requirement.

Anything that the package author installs as package data, or using the 
data_files option to setup(), is included in the egg.  And if you install 
unzipped, you should be able to browse the included docs.


From pje at telecommunity.com  Thu Jun 29 22:51:46 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 16:51:46 -0400
Subject: [Distutils] Problem between bdist_msi and egg-info (Python
 2.5b1)
In-Reply-To: <79990c6b0606290821n2d7770c1t7156d123551edfeb@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20060629164939.03b38078@sparrow.telecommunity.com>

At 04:21 PM 6/29/2006 +0100, Paul Moore wrote:
>   File "D:\Apps\Python25\Lib\msilib\__init__.py", line 115, in add_data
>     raise MSIError("Could not insert "+repr(values)+" into "+table)
>_msi.MSIError: Could not insert [(None, 'site_packages',
>'TEST-1~1.EGG|test-1.0-py2.5.egg-info', 186L, None, None, 512, 1)]
>into File

This line is masking whatever the actual error is.  If you look at the 
source, you'll see it's doing "except Exception,e:" and trapping whatever 
the real problem is.  I'd suggest commenting out the try/except and see 
what error comes up instead.  That should tell us more about the problem.

I believe Martin v. Loewis is the author of bdist_msi; I'm not sure if he 
reads this list or not.  Perhaps he will comment.


From pje at telecommunity.com  Thu Jun 29 22:55:20 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 16:55:20 -0400
Subject: [Distutils] Why using distutils.core.setup requires Python
 sources ?
In-Reply-To: <1151613557.11460.2.camel@localhost>
References: <5.1.1.6.0.20060629154655.03b50e00@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629154655.03b50e00@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060629165212.036e7528@sparrow.telecommunity.com>

At 10:39 PM 6/29/2006 +0200, J?r?me Bouat wrote:
>Thanks for your reply.
>
>Thus, distutils is not designed for distributing a module outside the
>Python source repository ?

I don't understand your question.  When Python is installed, it installs a 
copy of the Makefile that was used to create that Python installation.  So 
this has nothing to do with the source directory used to compile Python, 
only the contents of "/usr/lib/python2.x/config/" -- the directory where 
Python's build configuration information is copied upon installation.



From pje at telecommunity.com  Fri Jun 30 00:40:04 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Jun 2006 18:40:04 -0400
Subject: [Distutils] Why using distutils.core.setup requires Python
 sources ?
In-Reply-To: <1151618629.11460.22.camel@localhost>
References: <5.1.1.6.0.20060629165212.036e7528@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629154655.03b50e00@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629154655.03b50e00@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629165212.036e7528@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060629183809.01ef4868@sparrow.telecommunity.com>

At 12:03 AM 6/30/2006 +0200, J?r?me Bouat wrote:
> > only the contents of "/usr/lib/python2.x/config/" -- the directory where
> > Python's build configuration information is copied upon installation.
>
>Its seems that my Mandriva Linux distro did not package this well:
>
>[j at localhost ~]$ rpm -q --all | grep --ignore-case python
>python-2.4.1-3mdk
>python-numeric-24.0-1mdk
>python-base-2.4.1-3mdk
>libpython2.4-2.4.1-3mdk
>libxml2-python-2.6.21-3mdk

You might first want to check if there is a 'python-dev' or 'python-devel' 
package you can install; some distributions only install the 'config' 
directory with such a package.


>[j at localhost ~]$
>[j at localhost ~]$ rpm -q --list $(rpm -q --all | grep --ignore-case
>python) | grep --ignore-case makefile
>[j at localhost ~]$
>[j at localhost ~]$ locate -i makefile | grep --ignore-case python
>/usr/share/doc/libidn11-0.5.18/contrib/idn-python/Makefile
>[j at localhost ~]$
>[j at localhost ~]$ rpm -q
>--whatprovides /usr/share/doc/libidn11-0.5.18/contrib/idn-python/Makefile
>libidn11-0.5.18-2mdk
>[j at localhost ~]$
>
>Maybe I should report this as a Mandriva bug.

What you've shown above is an unrelated Makefile.  It's likely that the 
needed 'config' directory is installed by another package that you don't 
currently have installed.

If there is no such package, or it doesn't install the 'config' directory, 
then you should indeed report a bug.


From p.f.moore at gmail.com  Fri Jun 30 11:34:25 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 30 Jun 2006 10:34:25 +0100
Subject: [Distutils] Problem between bdist_msi and egg-info (Python
	2.5b1)
In-Reply-To: <5.1.1.6.0.20060629164939.03b38078@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060629164939.03b38078@sparrow.telecommunity.com>
Message-ID: <79990c6b0606300234kcebb86epf1717b32dac005a6@mail.gmail.com>

On 6/29/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:21 PM 6/29/2006 +0100, Paul Moore wrote:
> >   File "D:\Apps\Python25\Lib\msilib\__init__.py", line 115, in add_data
> >     raise MSIError("Could not insert "+repr(values)+" into "+table)
> >_msi.MSIError: Could not insert [(None, 'site_packages',
> >'TEST-1~1.EGG|test-1.0-py2.5.egg-info', 186L, None, None, 512, 1)]
> >into File
>
> This line is masking whatever the actual error is.  If you look at the
> source, you'll see it's doing "except Exception,e:" and trapping whatever
> the real problem is.  I'd suggest commenting out the try/except and see
> what error comes up instead.  That should tell us more about the problem.

Not much better, I'm afraid:-(

>python setup.py bdist_msi
running bdist_msi
running build
running build_py
installing to build\bdist.win32\msi
running install_lib
running install_egg_info
Removing build\bdist.win32\msi\Lib\site-packages\test-1.0-py2.5.egg-info
Writing build\bdist.win32\msi\Lib\site-packages\test-1.0-py2.5.egg-info
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    py_modules=['test'],
  File "D:\Apps\Python25\Lib\distutils\core.py", line 151, in setup
    dist.run_commands()
  File "D:\Apps\Python25\Lib\distutils\dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "D:\Apps\Python25\Lib\distutils\dist.py", line 994, in run_command
    cmd_obj.run()
  File "D:\Apps\Python25\Lib\distutils\command\bdist_msi.py", line 235, in run
    self.add_files()
  File "D:\Apps\Python25\Lib\distutils\command\bdist_msi.py", line
263, in add_files
    key = dir.add_file(file)
  File "D:\Apps\Python25\lib\msilib\__init__.py", line 343, in add_file
    language, attributes, sequence)])
  File "D:\Apps\Python25\lib\msilib\__init__.py", line 113, in add_data
    v.Modify(MSIMODIFY_INSERT, r)
_msi.MSIError: 1: 2210 2:
D:\Data\simple_module\dist\test-1.0.win32-py2.5.msi 3: 0

Paul

From p.f.moore at gmail.com  Fri Jun 30 11:38:58 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 30 Jun 2006 10:38:58 +0100
Subject: [Distutils] Best practices for creating eggs?
In-Reply-To: <5.1.1.6.0.20060629164229.03783d38@sparrow.telecommunity.com>
References: <BD714A20-589F-47E8-BD51-57D147C7CD72@zope.com>
	<5.1.1.6.0.20060628173425.02b89198@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629102724.01ef1868@sparrow.telecommunity.com>
	<5.1.1.6.0.20060629164229.03783d38@sparrow.telecommunity.com>
Message-ID: <79990c6b0606300238y64c2300ej2bdfa3698e8d7957@mail.gmail.com>

On 6/29/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:13 PM 6/29/2006 +0100, Paul Moore wrote:
> >Agreed. But in the absence of a standard, supporting package authors'
> >existing approaches, which work with other distutils mechanisms, seems
> >like a reasonable requirement.
>
> Anything that the package author installs as package data, or using the
> data_files option to setup(), is included in the egg.  And if you install
> unzipped, you should be able to browse the included docs.

Hmm. I'll try building an egg from scratch, when I get access to a
machine with internet access without a painful proxy. It's too much
hassle right now to remember the magic incantations I need to make to
get setuptools installed through our firewall (more a firewall problem
than a setuptools one...) :-(

It may be that the problem I recall was with easy_install's process
for building an egg from a bdist_wininst, rather than with a proper
egg build.

Paul.