[issue43163] codeop prematurely raises SyntaxError when ']' is needed

[Terry J. Reedy]

Terry J. Reedy <tjreedy at udel.edu> added the comment:

IDLE is a Python program that wraps the Python interpreter.  Before blaming IDLE for how code is executed, one should first execute the same code directly in Python.  The example above shows that one should also try code.interact, especially for compilaition issues. (Xie, thank you for this reminder.) IDLE and code.interact use the same code and codeop infrastructure. Most relevant here is codeop.CommandCompiler, which should act the same as codeop.compile_command in the absence of __future__ imports.

The codeop functions, including _maybe_compile, are intended to imitate in python the C REPL code that decides whether to execute, get more input, or raise.  It doesn't always.  (I think it would be good if a C expert compared it to the *current* C code.)

>>> CC = codeop.CommandCompiler
>>> cc("print([1,")  # Returns None, meaning, 'Get more input'.
>>> cc("print([1,\n2,")  
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "F:\dev\3x\lib\codeop.py", line 132, in compile_command
    return _maybe_compile(_compile, source, filename, symbol)
  File "F:\dev\3x\lib\codeop.py", line 106, in _maybe_compile
    raise err1
  File "F:\dev\3x\lib\codeop.py", line 93, in _maybe_compile
    code1 = compiler(source + "\n", filename, symbol)
  File "F:\dev\3x\lib\codeop.py", line 111, in _compile
    return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
  File "<input>", line 1
    print([1,
          ^
SyntaxError: '[' was never closed

If the opening '[' is removed, then ']' is replaced with (.  If '[' is replaced with '{' and the items adjusted, ']' is replaced with '}'.  In all three cases, CC should return None to ask for more, as the REPL does.

----------
components:  -IDLE
nosy: +pablogsal
title: code.interact() unexpectedly raises exception when there may be more code -> codeop prematurely raises SyntaxError when ']' is needed

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43163>
_______________________________________