[New-bugs-announce] [issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input"

[Mateusz Loskot]

New submission from Mateusz Loskot <mateusz at loskot.net>:

Something has changed in Python 3.7 through 3.10 (I'm observing it in 3.10) in behaviour of the Python C API function Py_CompileString such that for an incomplete input it no longer raises

SyntaxError: "unexpected EOF while parsing"

but

IndentationError: expected an indented block after ...

The new behaviour makes the sample program from the "How do I tell “incomplete input” from “invalid input”?" at https://docs.python.org/3/faq/extending.html no longer work as described there.

For example:

```
for i in []:
```

raises

IndentationError: expected an indented block after 'for' statement on line 1


```
if True:
```

raises

IndentationError: expected an indented block after 'if' statement on line 1

instead of 

SyntaxError: unexpected EOF while parsing

This effectively makes it impossible to detect incomplete input using the Py_CompileString in applications where it is not possible to use PyRun_InteractiveLoop.

I have failed to identify what could be related changes in the release notes and the documentation does not seem to offer any update on that.
So, I'm assuming the new behaviour is not desired or expected.

Attached, is the VS 2022 screenshot with debugging session of the sample program from the FAQ presenting the difference in the behaviour between Python 3.6 and 3.10 on Windows.

----------
components: C API
files: Py_CompileString-Py36-vs-Py310.png
messages: 411484
nosy: mloskot
priority: normal
severity: normal
status: open
title: Py_CompileString no longer allows to tell "incomplete input" from "invalid input"
versions: Python 3.10
Added file: https://bugs.python.org/file50582/Py_CompileString-Py36-vs-Py310.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46502>
_______________________________________