[issue27112] tokenize.__all__ list is incomplete

[Martin Panter]

Martin Panter added the comment:

Changing the names to tokenize.<name> does solve the problem of the tokenize module versus the tokenize() fuction, so I can accept this way since you prefer it.

So I think that just leaves what to do with the actual test case. I don’t think it matters too much, but I would lean toward ensuring the test fails if someone adds a new implementation detail without an underscore prefix. It is also good to be explicit that the ISTERMINAL() etc functions are special cases.

On the other hand, neither the original patch nor Jacek’s proposal for “expected = token.__all__ + ...” would pick up the fact that the tok_name dictionary is another special case copied from the “token” module. (See also Issue 25324.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27112>
_______________________________________