[issue37621] Line continuation of blank line emits NEWLINE

[Dominik Miedziński]

New submission from Dominik Miedziński <dominik at mdzn.pl>:

Both C and Python tokenizers emit NEWLINE tokens on continued blank lines. Because of this it is possible to emit repeated NEWLINEs, which some tools don't expect to happen.

I've attached example source file which is tokenized into 3 NEWLINE tokens in a row.

----------
components: Interpreter Core
files: linecont.py
messages: 348117
nosy: miedzinski
priority: normal
severity: normal
status: open
title: Line continuation of blank line emits NEWLINE
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file48488/linecont.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37621>
_______________________________________