[Python-3000] Substantial rewrite of PEP 3101

[Talin]

Eric V. Smith wrote:
> Talin wrote:
>> Other kinds of customization require replacing a much larger chunk of 
>> code. Changing the "underscores" and "check-unused" behavior requires 
>> overriding 'vformat', which means replacing the entire template string 
>> parser. I figured that there would be a lot of people who might want 
>> these features, but didn't want to rewrite all of vformat.
> 
> Actually you only have to replace get_positional or get_named, I think.

I don't think that people writing replacements for get_positional/named 
should have to reimplement the checking code. I'd like for them to worry 
only about accessing values, and leave the usage checking out of it.

> And I don't see how the "check-unused" behavior can be written in the 
> base class, in the presence of get_positional and get_named.  If the 
> list of identifiers isn't known to the base class (as in your example of 
> NamespaceFormatter), then how can the base class know if they're all used?

Because the checking only applies to arguments that are explicitly 
passed in to vformat(). It never applies to the default namespace.

Think of it this way: Would you consider it an error if the format 
string failed to refer to every global variable? Of course not. The 
default namespace is open-ended, whereas the positional and keyword 
arguments to vformat are a bounded set. So vformat can know exactly 
which arguments are and aren't used.

The checking code is, I think, relatively simple:

    checked_args = set()
    if checking_positional:
       checked_args.update(range(0,len(positional))
    if checking_named:
       checked_args.update(kwds.iterkeys())

    # now parse the template string, removing from the set
    # any arg names/indices that are referred to.

    if checked_args:  # If set non-empty
       # error

The code to populate the set of checked args could be in an overridable 
method, as suggested by Nick Coghlan. This method could simply return 
the set of args to check or None if checking is turned off.

The other way to do it would be to always build the set of 'used' names, 
and then call the method afterwards to do a set.difference operation. 
However, this means you always build a set even if you aren't checking, 
whereas with the first method you can skip creating the set if checking 
is turned off.

>>> I've started a sample implementation to test this API.  For starters,
>>> I'm writing it in pure Python, but my intention is to use the code in
>>> the pep3101 sandbox once I have some tests written and we're happy
>>> with the API.
>>
>> Cool.
> 
> I think we'll know more when I've made some more progress on this.
> 
> Eric.
>