[New-bugs-announce] [issue40546] Inconsistencies between pegen and traceback SyntaxErrors

[Lysandros Nikolaou]

New submission from Lysandros Nikolaou <lisandrosnik at gmail.com>:

Since traceback is programmed to match the SyntaxErrors emitted by the old parser, there are some inconsistencies between how it formats SyntaxErrors and how the new parser does it. One example for such behaviour is the following:

New Parser:
╰─ ./python.exe
Python 3.9.0a6+ (heads/master:4638c64295, May  7 2020, 16:47:53)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> x = 1 | 2 |
  File "<stdin>", line 1
    x = 1 | 2 |
               ^
SyntaxError: invalid syntax

Old parser and traceback module:
╰─ ./python.exe -X oldparser
Python 3.9.0a6+ (heads/master:4638c64295, May  7 2020, 16:47:53)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> x = 1 | 2 |
  File "<stdin>", line 1
    x = 1 | 2 |
              ^
SyntaxError: invalid syntax

We actually think that the new parser does a better job here, since there is nothing wrong with the `|` itself, the error is that a newline follows it and that's where the caret should point to.

Tests that are testing this are currently skipped. I think we should change traceback to match the new parser, wherever we think the new parser does a better job, and "unskip" these tests.

----------
components: Library (Lib)
messages: 368353
nosy: gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Inconsistencies between pegen and traceback SyntaxErrors
type: behavior
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40546>
_______________________________________