[Python-3000] Support for PEP 3131

[Stephen J. Turnbull]

Jim Jewett writes:

 > Of course, I wouldn't type them if I knew they were wrong.  With an
 > ASCII-only install, I would get that error-check because the
 > (remaining original uses) were in Cyrillic.  With an "any unicode
 > character" install, ... well, I might figure out my problem the next
 > morning.

But this is something that only a small subset of developers-of-Python
seem to be concerned about.  If that's generally the case for all
developers-in-Python, shouldn't the burden be placed on those who do
care?

It seems to me that rather than *impose* restrictions on third
parties, the sensible thing to do is to provide those restrictions to
those who want them.  But as Guido points out, that's outside of the
scope of this PEP because it can easily be done by external tools.

You object that running an auditor program would "cramp your style".
I don't mean that in a pejorative way; like Josiah's desire to
continue using certain tools, a developer's style is a BCP for him and
should *not* be gratuitously undermined.

But I see no reason why that auditor program can't be run as a PEP 263
codec.  AFAICS, the following objections could be raised, and answered:

1.  PEP 263 codecs delegate the decision to the code's author; an
    auditor shouldn't do that.

    You personally could modify your Python installation to replace
    all the codecs with a wrapper codec that processes the input by
    calling the "real" codec, then audits the resulting stream as it
    passes it back to the compiler.  But it can be done with a vanilla
    Python executable today.

    This is *proof of concept*; possibly there should be a UI to
    install such a codec via command line flag or environment
    variable, although there may be other creative ways to install it
    without altering the current interface to PEP 263 codecs.  I'm not
    yet familiar with the implementation to guess.

2.  The auditor would have to duplicate the work of the parser, and
    might get it wrong.

    AIUI, the parser is available as a module.  Use it.

3.  Parsing is expensive in time and other resources.

    No, it's not.  It's the other stuff that the compiler does that is
    expensive.<wink>  This is going to be O(size of source) like any
    codec with a somewhat higher constant than typical codecs.  More
    important, AIUI PEP 263 codecs don't get run on compiled code, so
    in a production environment it isn't an issue.

That doesn't mollify those who think I should not be allowed to use
non-ASCII identifiers at all.  But I think that should work for you
(modulo the UI for invoking the auditor).

Does it?