[Distutils] Further extending of setup.py

[Ian Bicking]

Phillip J. Eby wrote:
> At 12:47 PM 4/19/2006 -0500, Ian Bicking wrote:
> 
>> Phillip -- any thoughts on this?  If this went in, this would allow me
>> to start splitting up and simplifying some commands, as outlined in:
>> http://blog.ianbicking.org/paster-and-scripts.html
>>
>> Ian Bicking wrote:
>> > I'd like to get rid of the "paster" script, moving some of what it does
>> > into setup.py or elsewhere.
>> >
>> > To a degree that is possible, but I think requires some additions to
>> > setuptools:
>> >
>> > * An entry point group that is not globally looked up, like
>> > distutils.commands.  This would be a set of commands the package
>> > provides only for itself.  I expect it to look like:
>> >
>> >    [distutils.commands.local]
>> >    sql_record = sqlobject.manage.distextension:sql_record
>> >
>> > Exactly like what we have currently, just not looked up globally.
>> >
>> > * Additionally, or probably even better, something that enumerates
>> > locations for commands.  E.g.:
>> >
>> >    extra_commands=['SQLObject']
> 
> 
> You can pass a 'cmdclass' argument to setup() to accomplish these 
> things.  Normally, it's expected to be a dictionary, but as long as you 
> give it an object that implements 'in', 'get()', 'keys()' and 
> setitem/getitem operations, you should be good to go.  (The distutils 
> and setuptools both cache their own lookups by assigning items back into 
> the cmdclass object.)

It looks like, if I do this, it will also have to implement the current 
default behavior of finding distutils.commands providers.  Which isn't a 
big deal as long as that doesn't change, I guess.  Though cmdclass won't 
get a reference to the setuptools.dist.Distribution object, so some 
things might not be possible.

> If you come up with a nice "command map" object that lets you do 
> something like:
> 
>      setup(
>          cmdclass = CommandMap(
>              # configuration stuff here
>          ),
>          ...
>      )
> 
> then it would make a nice addition to setuptools for people that need it.

OK, I'll look into that.  I'm guessing it'll be something like 
CommandMap([eggs]), and maybe some other options, though I can't think 
of any at the moment.  Maybe also reading .egg-info/command_map.txt if 
not given any list of eggs -- I'd like to keep options for putting 
metadata into .egg-info/*.txt files instead of setup.py.

>> >A project could list itself to provide its own custom commands;
>> > I think that won't be too circular.
> 
> 
> That's only 'cause you haven't tried it yet.  Setuptools has to do this, 
> sort of, and trust me, it's a bitch.  :)

Yeah... it probably isn't important anyway, I can't think of any cases 
where I'd really want to use this.

>> > * Everything that can be done on a deployed egg will probably go in
>> > app/egg-specific command-line scripts, and maybe I'll make a little
>> > framework to make these easier to write, but that's entirely an
>> > implementation detail.
> 
> 
> I actually do want "nest" to provide a nice mini-framework for writing 
> command-line apps as functions -- sort of a stripped-down cross between 
> peak.running.commands and peak.running.options.  One key feature will be 
> that the commands have to accept an argument that's a logger for them to 
> use, and the verbosity of that logger will be managed by the 
> caller/framework, not the function.  (Meaning that verbosity options 
> will be handled by the framework when running via the command line.)

In my code I do a lot of specific testing of verbosity, to try to avoid 
redudancy.  For instance:

def ensure_dir(dir):
     if verbosity >= 2:
         print 'Trying to make directory %s' % dir
     if os.path.exists(dir):
         if verbosity >= 2:
             print '  (directory already exists)'
         elif verbosity >= 1:
             print 'Directory %s already exists' % dir

It's a little annoying to write this, but log-style output is fairly 
crappy; a better compromise might be:

def ensure_dir(dir):
     log.debug('Trying to make directory %s' % dir)
     if os.path.exists(dir):
         log.debug('  (directory already exists)')
         log.message((log.INFO, log.DEBUG),
                     'Directory %s already exists' % dir)


I think the features for a simple two-level command framework would be 
fairly simple:

* Register commands somehow (entry points, I suppose).  Commands have a 
name and aliases.
* Commands can be called to run them, with all arguments after the 
command name.
* Commands have a help() method or something like that, that returns a 
string help message.  Probably two forms -- a long-form help, and 
short-form.  Maybe only the short form?
* The framework provides "cmd -h", "cmd help", "cmd -h subcommand", and 
"cmd help subcommand".  The last two might just call "cmd subcommand -h" 
internally.
* I suppose the framework needs to be given some global help message.
* It should be possible to hide commands or aliases from help.  E.g., 
commands that represent an internal interface, or aliases that are only 
provided for backward compatibility.
* Maybe a standard way to activate specific eggs, e.g., "cmd 
--require='Foo==1.0' ...".  Maybe also a way of controlling the 
PYTHONPATH, since setting environmental variables is not equally easy in 
all environments.

Everything else should be possible to build from there with other 
sub-frameworks.

Self-referencing entry point loading would be nice.  Then, for instance, 
you could do:

   entry_points="""
   [whatever.this.group.is]
   sql-setup = sqlobject.commands:setup_command
   """

If setup_command is called with the package's Distribution object, then 
it can read config files or whatever from the egg, and you don't have to 
create little bits of glue all over the place.

Hmm... putting information into the entry point name is problematic 
here, because SQLObject would give a set of commands which would be a 
bit tedious to enumerate, and it would be better if you could introduce 
the whole set of commands with one entry point.  Also, it doesn't 
resolve the problem of aliases.  Though it does give you an opportunity 
to resolve naming conflicts by renaming a command.




> Although perhaps I should just write this hypothetical framework as a 
> separate package...  but I want nest to be a part of the core setuptools 
> distribution.  I guess if you're going to depend on the command line 
> framework, you might as well depend on setuptools, since you'll need to 
> have setuptools in order to be able to depend on the hypothetical 
> command line package.  :)  (Confusing, isn't it?)

Would setuptools stop using the distutils command framework?


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org