When argparse will be in the python standard installation

[Steven Bethard]

[Thanks for looking through all these Martin!]

Martin v. Löwis wrote:
> Steven Bethard schrieb:
>> * alias ArgumentParser to OptionParser
>> * alias add_argument to add_option
>> * alias Values to Namespace
>> * alias OptionError and OptionValueError to ArgumentError
>> * alias add_help= keyword argument of ArgumentParser to add_help_option=
>> * alias namespace= keyword argument of parse_args to options=
> 
> Not sure what you mean by "alias": I don't think the argparse names
> should be retained (unless you feel they describe things better).

They do describe things a bit better. Given a command line like::

     prog --opt1 o --opt2 pos1 pos2 pos3

optparse can only parse out the optional command line arguments, 
``--opt1 o --opt2``, hence it uses names like ``add_option``. On the 
other hand, argparse can parse the entire command line (including the 
positional arguments ``pos1 pos2 pos3``) so it uses ``add_argument`` 
instead of ``add_option``.

>> * ArgumentParser.parse_args returns a single namespace object, not an
>>   (options, args) tuple, since argparse handles positional arguments.
>>   This could probably be addressed by adding an __iter__ method to the
>>   Namespace object which would return (self, [])
> 
> I find that I'm not that familiar with command line arguments :-)
> 
> To understand that difference, I would have to see examples of both,
> and why it is better to return a single result.

A brief example to illustrate the differences. Here is a program that 
reads the command-line args and writes the sum to a log file specified 
by --log.  With optparse, this would look something like::

     parser = optparse.OptionParser()
     parser.add_option('--log')
     options, args = parser.parse_args()
     if options.log is not None:
         logfile = open(options.log)
     else:
         logfile = sys.stdout
     try:
         integers = [int(arg) for arg in args]
     except ValueError:
         parser.error('positional arguments must be integers')
     logfile.write('%s\n' % sum(integers))
     logfile.close()

Since argparse handles positional arguments as well as optional 
arguments (and since it accepts type='outfile') the argparse version is 
much more concise::

     parser = argparse.ArgumentParser()
     parser.add_argument('integers', type=int, nargs='+')
     parser.add_argument('--log', type='outfile', default=sys.stdout)
     args = parser.parse_args()
     args.log.write('%s\n' % sum(args.integers))
     args.log.close()

So basically, since argparse parses the entire command line (instead of 
just the optional parts) there is no concept of "remaining command line 
arguments" -- *all* command line arguments have become attributes of the 
namespace returned by parse_args(). Thus, while optparse has to return 
both a namespace and the unparsed portion of the command line, argparse 
only has to return the namespace.

>> * argparse makes the default value for a "store_true" action False, and
>>   the default value for a "store_false" action True. This is what users
>>   expect, but different from optparse where the default is always None.
> 
> Could we deprecate "store_false" with no default?

That's probably reasonable.  In Python 3.0, the default could then 
become the appropriate one.

> It may be that users expect to see True and False, but how can you find
> out whether the option was given or not afterwards?

For both optparse and argparse, with "store_false", you know the option 
was not given if the stored value is something other than ``False``, and 
with "store_true", you know the option was not given if the stored value 
is something other than ``True``.

With optparse, that "something other" is always ``None`` which is 
unfortunate because it makes the following code fail::

     parser = optparse.OptionParser()
     parser.add_option('--not-foo', action='store_false')
     options, args = parser.parse_args()
     if options.not_foo:
         # do something

The ``do something`` block will execute every time because 
``options.not_foo`` is either ``False`` or ``None``, both of which 
indicate that the if-statement should not be executed.

In argparse, the code above works fine because the default is ``True`` 
instead of ``None``.

> Alternatively, could the actions become keyword arguments (with the
> action argument only preserved for backwards compatibility)? E.g.
> 
> parser.add_option("-v", store_true_into="verbose")
> parser.add_option("-q", store_false_into="verbose")
> (same for store, append, count, callback)

It looks like you're trying to merge the ``dest=`` and ``action=`` 
functionality, but I'm not sure I understand where this is going...

>> * the choices argument is now checked *after* arguments have been
>>   type-converted. This is intentional, so that you can specify, say
>>   xrange(100) instead of ["0", "1", "2", "3", ... "99"]. There is also
>>   no "choices" type anymore since any action can also specify their
>>   choices.
> 
> As an incompatible change, this could be warned-about if the type is not
> string, yet the first choice is.
> 
> Alternatively, we could require users that pass converted values to
> specify converted_choices=True in 2.6, and warn all users that don't
> pass a converted_choices flag, and then flip the default in 2.7 and 3.0.
> Then current users had a quick fix to silence the warning, and new
> users could drop the extra flag in the future.

That seems reasonable.

>>   I could probably add callback actions by creating an appropriate
>>   Action subclass and registering it. However, any code relying on
>>   parser.{largs,rargs,values} would break because the parsing algorithm
>>   is completely different in argparse.
> 
> If you can find a way to make callbacks work in the "common case",
> this flag could be deprecated and removed.

Sounds good.

>> I guess I don't know how to proceed from here. I'm reluctant to start
>> adding the code necessary to support even the easily solved issues when
>> the issues that I don't know how to solve seem like they could be deal
>> breakers.
> 
> This asks for a PEP. The views above are mine only, others may feel
> differently.

Ok.  I'll try to write a PEP up this week.

Thanks again for all your useful feedback!

STeVe