[issue19120] shlex.shlex.lineno reports a different number depending on the previous token

[Cheryl Sabella]

Cheryl Sabella <cheryl.sabella at gmail.com> added the comment:

There was a parameter `punctuation_chars` added to the shlex.shlex class with issue 1521950 (implemented for 3.6).  Although the comma is not one of the default punctuation characters (setting the parameter to punctuation_chars=True won't change the behavior), you can use `punctuation_chars=","` to see the results reported in this issue.


>>> second = shlex.shlex('word1 word2,\nword3', punctuation_chars=',')
>>> second.get_token()
'word1'
>>> second.lineno
1
>>> second.get_token()
'word2'
>>> second.lineno
1
>>> second.get_token()
','
>>> second.lineno
2
>>>


Closing this as a duplicate of #1521950.

----------
nosy: +cheryl.sabella
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> shlex.split() does not tokenize like the shell

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue19120>
_______________________________________