[Distutils] setuptools 0.5a5: "develop" and "test" commands

[Phillip J. Eby]

I've just released setuptools 0.5a5, which updates the old "test" command 
to do in-place build-and-test instead of the previous install-and-test 
behavior.  It also automatically require()s the package's dependencies, if 
any, and updates the .egg-info metadata.

I also added a new "develop" command, which is like an install that doesn't 
actually install the package.  It runs "build_ext --inplace" (so that any C 
extensions get built in the source directory) and then installs a .egg-link 
pointing to the source tree, optionally updating easy-install.pth and 
installing any dependencies for the package.  (It only takes a subset of 
the easy_install command options, however, so it's currently better if you 
install your dependencies before using the "develop" command, especially if 
you want to "develop" any of those as well.)

The "develop" command basically adds your source checkout to the eggs that 
will be searched at runtime to resolve a 'require()' request, and if you're 
installing to site-packages, then it will be added to sys.path as well 
(unless you use --multi-version or -m).  This lets you edit your source 
checkout and do development without having to actually install the 
package.  Wrapper scripts are installed to the --script-dir; the wrappers 
do a require() and then execfile() your in-development script source files, 
so you don't have to re-run "develop" just because you've changed script 
contents.  The only time you need to re-run the "develop" command is when 
your metadata (e.g. version or requirements) changes, or if you add a new 
script, or need to rebuild changed C extensions.

When you're done with development, "setup.py develop --uninstall" will 
remove the .egg-link and any easy-install.pth entry for your development 
tree, but you will need to update or replace any installed scripts by hand.

All of these commands respect the standard distutils .cfg files, so you can 
e.g. use setup.cfg to set options for developing and testing.

There were a couple of other important changes in this release; 
easy_install the *module* is now found under setuptools.command; 
easy_install the *script* no longer contains the actual command 
code.  Also, a new "egg_info" command was split out of "bdist_egg", so that 
it could be used by "test" and "develop" as well as "bdist_egg".  As a 
result of this, "bdist_egg" no longer supports the various "tag" options 
that it did before; you must now do something like this::

     setup.py egg_info --tag-svn-revision bdist_egg

if you want to set tagging options.  You can also use this with the 
"develop" and "test" commands, e.g.::

     setup.py egg_info --tag-svn-revision develop

in order to change the version info that will be used for the "virtual egg".

At this point, I think setuptools now has sufficient features to support 
development of packages with "cheap" external dependencies.  That is, a 
package using setuptools to create its setup script can add external 
dependencies with almost no effort on the part of the package developer, 
and no effort at all for the package installer.  And, with further effort 
by persons familiar with the appropriate packaging systems, the dependency 
information included in setup.py could be used to generate native packages 
with dependency information (assuming there is some uniform mapping from 
PyPI project names to platform package names).

In other words, I think we've now achieved an interim state of packaging 
nirvana.  There are many additional stages of enlightenment possible, but I 
think we are now in a position to begin the journey.  Of course, having 
some docs would help quite a lot, as right now there's no formal 
documentation for stuff like version/requirement syntax, how to specify 
requirements in the setup() call, etc.  I'd like to start upgrading my 
existing packages to use the new features before I try and write any, but 
if any brave souls happen to get to it first, well, the more the merrier 
where docs are concerned.

The one area where docs might be a problem for a while is the actual 
pkg_resources infrastructure, since I intend to do a major refactoring of 
it in the 0.6 releases.  In addition to restructuring based on the cleaner 
concepts of distributions, environments, working sets, etc., it will also 
be test-driven, such that nearly all the pkg_resources code will have unit 
tests for it, and as much of the setuptools command code as possible will 
too.  I'll also be trying to clean up the older attempts at a dependency 
management system that lurk deeper in setuptools; these never quite worked 
out before.  Finally, there are a few older features that are now of 
questionable utility given the ease of adding external dependencies.  But, 
I'm not sure if I can remove any/all of these things yet, because I don't 
know if anybody (besides Fred Drake and myself) has based anything 
interesting on those features.  So, those changes might have to wait for 
the 0.7 release series, depending.