Error in Extending/Embedding FAQ, point 16: How do I tell "incomplete input" from "invalid input"?

[Steve Holden]

Dietrich:

The web maintainers list is pydotorg at python.org. Your message will be 
held for moderation, but will be seen by the team who maintain the web site.

regards
  Steve

PS: Anyone who wants to *help* maintain the web site should also email 
that list. It helps if you are already known to other members of the 
Python community, but it's not essential.

Dietrich Bollmann wrote:
> Hi,
> 
> I found a solution thanks to another posting on c++-sig and an answer by
> Andreas Klöckner :)
> 
> Thank you, Andreas!
> 
> The thread is here: 
>   http://mail.python.org/pipermail/c++-sig/2008-April/thread.html#13470
> 
> I would like to inform the responsible of the Python Extending/Embedding
> FAQ, http://www.python.org/doc/faq/extending/ about the broken code in
> the FAQ and the solution I found.
> 
> I hope this might prevent other people from the frustration I found
> myself in this morning  (...but unfortunately also, at least partly,
> from the joy I am experiencing now, after finding the new solution :).
> 
> Does anybody know how to contact the person in charge?
> 
> Thanks, 
> 
> Dietrich
> 
> PS:
> 
> Of course, I still wonder about the "invalid syntax" error message /
> code I wrote about. But ok, I hope there will be some more adequate
> error message / code some day in the future :)
> 
> On Wed, 2008-04-23 at 01:09 +0900, Dietrich Bollmann wrote:
>> On Wed, 2008-04-23 at 00:12 +0900, Dietrich Bollmann wrote:
>>> The following code for example:
>>>
>>>   >>> eins = [1,
>>>   ...     2,
>>>   ...     3]
>>>   >>> 
>>>
>>> is accepted without any problem by the Python shell.
>>>
>>> When using the code from the FAQ and entering it line by line 
>>> already the second line causes a simple "invalid syntax" error:
>>>
>>>   >>> eins = [1,
>>>   ...     2,
>>>     File "<stdin>", line 2
>>>       2,
>>>        ^
>>>   SyntaxError: invalid syntax
>> By the way - isn't this error message / error code just "wrong" in
>> the given situation and therefor kind of a "bug"?
>>
>> An "end of file" or "incomplete input" error at least would 
>> describe the situation much better - and be a better base for
>> functionality which is based the error code also.
>>
>> ---
>>
>> I also thought that I should explain a little bit more exactly, 
>> what I am intending to do with the code based on 
>> paragraph 16 (How do I tell "incomplete input" from "invalid input"?)
>> of the Extending/Embedding FAQ:
>>
>> I am using Python as scripting language in an application (blender).
>> In order to interface this application from other programs
>> I programmed a python command port / command socket 
>> for this application.
>>
>> Between other clients I also wrote a shell client which connects via 
>> the command port to the application.  My goal is to make it as similar
>> to a normal python shell as possible - and therefor I try to also mimic
>> the "intelligent" way of the Python shell to react to Python input:
>>
>>   - when entering a line which is a complete input,
>>     it is immediately evaluated by the shell and the 
>>     result is printed.
>>
>>   - when the last entered line is erroneous, 
>>     an error message is printed immediately
>>
>>   - when the input is incomplete, Python waits
>>     for other lines to complete the input
>>
>>   - when the line is part of a function definition etc.
>>     python waits until an empty line is entered
>>     before accepting the input as complete.
>>
>> My problem is to understand when an input is erroneous and
>> when it is incomplete - which is impossible with an error message
>> like "invalid syntax"...
>>
>> So here again my question: How can I make the difference
>> between an incomplete and an erroneous input?
>>
>> The code examples in the FAQ worked fine until now - but do not
>> anymore for the current Python implementation.
>>
>> Thanks, Dietrich
>>
>> By the way:  Does anybody know who is responsible for the FAQ
>> and could adapt the examples to the current Python version
>> by changing the code / annotating it? 
>>
>>
>> On Wed, 2008-04-23 at 00:12 +0900, Dietrich Bollmann wrote:
>> Hi, 
>>> Both code examples from paragraph 16 from the Python Extending /
>>> Embedding FAQ - 'How do I tell "incomplete input" from "invalid
>> input"?'
>>> -
>>>
>> ( http://www.python.org/doc/faq/extending/#how-do-i-tell-incomplete-input-from-invalid-input ) do not work with the current state of Python anymore.
>>> In the second code example, the error message returned by Python is
>>> checked in order to differentiate errors caused by an incomplete input
>>> from other syntax errors:
>>>
>>>    if (PyArg_ParseTuple (val, "sO", &msg, &obj) &&
>>>             !strcmp (msg, "unexpected EOF while parsing")) /* E_EOF */
>>>
>>> In the current Python version there are more error messages indicating
>> an 
>>> incomplete Python input and I could make the code work for a while 
>>> by adding the following strings to the condition:
>>>
>>> 		/* error messages indicating an incomplete input */
>>> 		if (PyArg_ParseTuple(error, "sO", &message, &obj) &&
>>> 			(!strcmp(message, "unexpected EOF while parsing") ||
>>> 			 !strcmp(message, "expected an indented block")   ||
>>> 			 !strcmp(message, "EOF while scanning triple-quoted string")
>>> 			 )
>>> 			) { /* E_EOF */
>>>
>>> but recently there are also cases which generate error messages
>>> which are too general to be added to this list.
>>>
>>> The following code for example:
>>>
>>>   >>> eins = [1,
>>>   ...     2,
>>>   ...     3]
>>>   >>> 
>>>
>>> is accepted without any problem by the Python shell.
>>>
>>> When using the code from the FAQ and entering it line by line 
>>> already the second line causes a simple "invalid syntax" error:
>>>
>>>   >>> eins = [1,
>>>   ...     2,
>>>     File "<stdin>", line 2
>>>       2,
>>>        ^
>>>   SyntaxError: invalid syntax
>>>
>>> which is to general to be integrated into the list of tested 
>>> error messages as it might be caused also by code like:
>>>
>>>   >>> one two
>>>     File "<stdin>", line 1
>>>       one two
>>>             ^
>>>   SyntaxError: invalid syntax
>>>
>>> which generates an "invalid syntax" error even in the Python shell.
>>>
>>> I also tried the first code example of paragraph 
>>> '16   How do I tell "incomplete input" from "invalid input"?'
>>> of the FAQ in order to see if it could be used to make the 
>>> difference between syntax errors and incomplete code errors.  
>>> But - as in the case before - the returned error
>>> code is E_SYNTAX (14 = Syntax error) and not E_EOF (11 = End Of File)
>>> as should be expected.
>>>
>>> Is there anybody who has an idea how to differentiate the 
>>> first case from the second in order to mimic the behaviour of 
>>> the Python shell from c code?  
>>>
>>> If this shouldn't be possible lists split into different lines
>>> couldn't be accepted anymore or the feature of the Python shell 
>>> to described in paragraph 16 of the faq:
>>>
>>>   Sometimes you want to emulate the Python interactive interpreter's 
>>>   behavior, where it gives you a continuation prompt when the input 
>>>   is incomplete (e.g. you typed the start of an "if" statement or you 
>>>   didn't close your parentheses or triple string quotes), but it
>> gives 
>>>   you a syntax error message immediately when the input is invalid.
>>>
>>> would have to be given up and every entered line of code would have
>> to 
>>> be terminated by an empty line before evaluation :(
>>>
>>> Thanks for any help, Dietrich
>>>
>>>
>>>
>>>
>>
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/