[issue16121] shlex.shlex.error_leader() reports incorrect line number

[Birk Nilson]

Birk Nilson added the comment:

After investigating the issue I have a couple of proposals.

Although a bit vague, the documentation of shlex.lineo seems to suggest that it should be incremented immediately on finding a newline character. Changing this to allow wrapped lines within a token without incrementing the line number changes the existing shlex API. Something I believe should be avoided since netrc relies on the existing behavior and third-party modules might too.

Instead I recommend the following steps.
Step #1: Fix the immediate issue of getting different line numbers for the same input depending on whether posix=(True|False), but keep the current - greedy - behavior of shlex.lineo.
Step #2: A separate patch introduces shlex.wrapped_lineo which does not increment the lineno immediately, but prior to reading the next token - as introduced in my previous patch.

Step #2 should arguably be introduced in a separate issue - if at all - since it is a new feature to the shlex API.

I will provide a patch for #1 within the next day or two along with one for #2 if you guys think it is a good idea.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16121>
_______________________________________