[issue33899] Tokenize module does not mirror "end-of-input" is newline behavior

[Gregory P. Smith]

Gregory P. Smith <greg at krypto.org> added the comment:

FYI, An example of other fallout from this change - patsy broke and needed this fix:

https://github.com/pydata/patsy/commit/4f53bbaf58c0bf1a9bed73fc67c7c6d0aa7f4e20#diff-53c70e68c6dfd4fe9b08427792cb2bd6

----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33899>
_______________________________________