Error in Extending/Embedding FAQ, point 16: How do I tell "incomplete input" from "invalid input"?

[Dietrich Bollmann]

On Wed, 2008-04-23 at 00:12 +0900, Dietrich Bollmann wrote:
> The following code for example:
> 
>   >>> eins = [1,
>   ...     2,
>   ...     3]
>   >>> 
> 
> is accepted without any problem by the Python shell.
> 
> When using the code from the FAQ and entering it line by line 
> already the second line causes a simple "invalid syntax" error:
> 
>   >>> eins = [1,
>   ...     2,
>     File "<stdin>", line 2
>       2,
>        ^
>   SyntaxError: invalid syntax

By the way - isn't this error message / error code just "wrong" in
the given situation and therefor kind of a "bug"?

An "end of file" or "incomplete input" error at least would 
describe the situation much better - and be a better base for
functionality which is based the error code also.

---

I also thought that I should explain a little bit more exactly, 
what I am intending to do with the code based on 
paragraph 16 (How do I tell "incomplete input" from "invalid input"?)
of the Extending/Embedding FAQ:

I am using Python as scripting language in an application (blender).
In order to interface this application from other programs
I programmed a python command port / command socket 
for this application.

Between other clients I also wrote a shell client which connects via 
the command port to the application.  My goal is to make it as similar
to a normal python shell as possible - and therefor I try to also mimic
the "intelligent" way of the Python shell to react to Python input:

  - when entering a line which is a complete input,
    it is immediately evaluated by the shell and the 
    result is printed.

  - when the last entered line is erroneous, 
    an error message is printed immediately

  - when the input is incomplete, Python waits
    for other lines to complete the input

  - when the line is part of a function definition etc.
    python waits until an empty line is entered
    before accepting the input as complete.

My problem is to understand when an input is erroneous and
when it is incomplete - which is impossible with an error message
like "invalid syntax"...

So here again my question: How can I make the difference
between an incomplete and an erroneous input?

The code examples in the FAQ worked fine until now - but do not
anymore for the current Python implementation.

Thanks, Dietrich

By the way:  Does anybody know who is responsible for the FAQ
and could adapt the examples to the current Python version
by changing the code / annotating it? 


On Wed, 2008-04-23 at 00:12 +0900, Dietrich Bollmann wrote:
Hi, 
> 
> Both code examples from paragraph 16 from the Python Extending /
> Embedding FAQ - 'How do I tell "incomplete input" from "invalid
input"?'
> -
>
( http://www.python.org/doc/faq/extending/#how-do-i-tell-incomplete-input-from-invalid-input ) do not work with the current state of Python anymore.
> 
> In the second code example, the error message returned by Python is
> checked in order to differentiate errors caused by an incomplete input
> from other syntax errors:
> 
>    if (PyArg_ParseTuple (val, "sO", &msg, &obj) &&
>             !strcmp (msg, "unexpected EOF while parsing")) /* E_EOF */
> 
> In the current Python version there are more error messages indicating
an 
> incomplete Python input and I could make the code work for a while 
> by adding the following strings to the condition:
> 
> 		/* error messages indicating an incomplete input */
> 		if (PyArg_ParseTuple(error, "sO", &message, &obj) &&
> 			(!strcmp(message, "unexpected EOF while parsing") ||
> 			 !strcmp(message, "expected an indented block")   ||
> 			 !strcmp(message, "EOF while scanning triple-quoted string")
> 			 )
> 			) { /* E_EOF */
> 
> but recently there are also cases which generate error messages
> which are too general to be added to this list.
> 
> The following code for example:
> 
>   >>> eins = [1,
>   ...     2,
>   ...     3]
>   >>> 
> 
> is accepted without any problem by the Python shell.
> 
> When using the code from the FAQ and entering it line by line 
> already the second line causes a simple "invalid syntax" error:
> 
>   >>> eins = [1,
>   ...     2,
>     File "<stdin>", line 2
>       2,
>        ^
>   SyntaxError: invalid syntax
> 
> which is to general to be integrated into the list of tested 
> error messages as it might be caused also by code like:
> 
>   >>> one two
>     File "<stdin>", line 1
>       one two
>             ^
>   SyntaxError: invalid syntax
> 
> which generates an "invalid syntax" error even in the Python shell.
> 
> I also tried the first code example of paragraph 
> '16   How do I tell "incomplete input" from "invalid input"?'
> of the FAQ in order to see if it could be used to make the 
> difference between syntax errors and incomplete code errors.  
> But - as in the case before - the returned error
> code is E_SYNTAX (14 = Syntax error) and not E_EOF (11 = End Of File)
> as should be expected.
> 
> Is there anybody who has an idea how to differentiate the 
> first case from the second in order to mimic the behaviour of 
> the Python shell from c code?  
> 
> If this shouldn't be possible lists split into different lines
> couldn't be accepted anymore or the feature of the Python shell 
> to described in paragraph 16 of the faq:
> 
>   Sometimes you want to emulate the Python interactive interpreter's 
>   behavior, where it gives you a continuation prompt when the input 
>   is incomplete (e.g. you typed the start of an "if" statement or you 
>   didn't close your parentheses or triple string quotes), but it
gives 
>   you a syntax error message immediately when the input is invalid.
> 
> would have to be given up and every entered line of code would have
to 
> be terminated by an empty line before evaluation :(
> 
> Thanks for any help, Dietrich
> 
> 
> 
>