[issue19120] shlex.shlex.lineno reports a different number depending on the previous token

[Daniel]

New submission from Daniel:

See the example below (also attached).

First example: The lineno reported just after "word2" is pulled is 2.
Second example: The lineno reported just after "," is pulled is still 1.

This behaviour seems inconsistent. The lineno should increment either when the last token of a line is pulled, or after the first token from the next line (in my opinion preferably the former). It should not have different bahaviour depending on what type of token that is (alpha vs. comma).

I have repeated this on 

Also, does Issue 16121 relate to this?


#!/usr/bin/env python
import shlex

first = shlex.shlex("word1 word2\nword3")
print (first.get_token())
print (first.get_token())
print ("line no", first.lineno)
print ("")

second = shlex.shlex("word1 word2,\nword3")
print (second.get_token())
print (second.get_token())
print (second.get_token())
print ("line no", second.lineno)


# Output:
# word1
# word2
# line no 2
#
# word1
# word2
# ,
# line no 1

----------
files: shlex_line.py
messages: 198561
nosy: daniel-s
priority: normal
severity: normal
status: open
title: shlex.shlex.lineno reports a different number depending on the previous token
type: behavior
versions: Python 2.6, Python 2.7, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file31900/shlex_line.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19120>
_______________________________________