[issue9974] tokenizer.untokenize not invariant with line continuations

[Terry J. Reedy]

Terry J. Reedy added the comment:

One could argue that "The guarantee applies only to the token type and token string as the spacing between tokens (column positions) may change." covers merging of lines, but total elimination of needed whitespace is definitely a bug.

----------
nosy: +terry.reedy
stage:  -> patch review
versions: +Python 3.3, Python 3.4 -Python 2.6, Python 3.1

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9974>
_______________________________________