[issue9974] tokenizer.untokenize not invariant with line continuations

[nick caruso]

nick caruso <ngvrnd at gmail.com> added the comment:

--------------------------
import StringIO
import tokenize

tokens = []
def fnord(*a):
    tokens.append(a)

tokenize.tokenize(StringIO.StringIO("a = 1").readline, fnord)

tokenize.untokenize(tokens)
----------------------------------

Generates the same assertion failure, for what it's worth.  No line continuation needed.

This does not happen in 2.5 on my machine.

----------
nosy: +nick.caruso

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9974>
_______________________________________