[issue34428] tokenize

[Luke Kenneth Casson Leighton]

Luke Kenneth Casson Leighton <lkcl at lkcl.net> added the comment:

ahh darn-it, autopep8 is passing in tokens line-by-line,
to be parsed one at a time.... oh and of course it's losing
state information that tokenizer critically relies on.

i *think* that's what's going on.... so it's highly unlikely
to be a python tokenize bug... can we wait to see what the
autopep8 developer says?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34428>
_______________________________________