[issue29812] test for token.py, and consistency tests for tokenize.py

[R. David Murray]

New submission from R. David Murray:

http://bugs.python.org/issue24622 made reminded me that a while back we added tests for the keyword module that includes a test that if you run it, you get the result that is checked in.  The same thing could be done for the token.py module.  And then we could add a cross-check test that tokenize.py has all the symbols defined as well.

----------
components: Tests
keywords: easy
messages: 289628
nosy: r.david.murray
priority: normal
severity: normal
stage: needs patch
status: open
title: test for token.py, and consistency tests for tokenize.py
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29812>
_______________________________________