[Python-Dev] Treating tokenize.Untokenizer as private

[Eric Snow]

On Tue, Feb 18, 2014 at 2:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> I am working through the multiple bugs afflicting tokenize.untokenize, which
> is described in the tokenize doc and has an even longer docstring. While the
> function could be implemented as one 70-line function, it happens to be
> implemented as a 4-line wrapper for a completely undocumented (Untokenizer
> class with 4 methods. (It is unmentioned in the doc and there are currently
> no docstrings.)
>
> I view the class as a private implementation detail and would like to treat
> it as such, and perhaps even rename it _Untokenizer to make that clear. The
> issue arises in #9974. It appears that a fix may require the addition of an
> instance attribute or .add_whitespace parameter. If there is objection to
> treating the whole class as private, I would at least like to treat
> add_whitespace as the private helper that it is. There is no reason to call
> it directly except for testing. Otherwise, it could just as well have been
> left inline at the one call site.

Is this still an open question, Terry?

-eric