[New-bugs-announce] [issue43985] lib2to3 fails on a slash('/') after positional_only paramter in a function signature

[MisterY]

New submission from MisterY <cloud2037 at gmail.com>:

When I try to format a code snippet with yapf, there comes a error.
It seems lib2to3 can't reconised SLASH.

echo "def foo(posonly1, posonly2, /, positional_or_keyword): pass" | yapf 

Traceback (most recent call last):
  File "d:\program files\python38\lib\site-packages\yapf\yapflib\pytree_utils.py", line 115, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "d:\program files\python38\lib\lib2to3\pgen2\driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "d:\program files\python38\lib\lib2to3\pgen2\driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "d:\program files\python38\lib\lib2to3\pgen2\parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=17, value='/', context=(' ', (1, 28))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program files\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "d:\program files\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\Program Files\Python38\Scripts\yapf.exe\__main__.py", line 7, in <module>
  File "d:\program files\python38\lib\site-packages\yapf\__init__.py", line 362, in run_main
    sys.exit(main(sys.argv))
  File "d:\program files\python38\lib\site-packages\yapf\__init__.py", line 104, in main
    reformatted_source, _ = yapf_api.FormatCode(
  File "d:\program files\python38\lib\site-packages\yapf\yapflib\yapf_api.py", line 147, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "d:\program files\python38\lib\site-packages\yapf\yapflib\pytree_utils.py", line 121, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "d:\program files\python38\lib\lib2to3\pgen2\driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "d:\program files\python38\lib\lib2to3\pgen2\driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "d:\program files\python38\lib\lib2to3\pgen2\parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=17, value='/', context=(' ', (1, 28))

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 392396
nosy: cloud-yu
priority: normal
severity: normal
status: open
title: lib2to3 fails on a slash('/') after positional_only paramter in a function signature
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43985>
_______________________________________