[IronPython] IPython is breathing but there's a compile() problem

[Michael Foord]

Dino Viehland wrote:
>
> This is probably a dup of 12907 
> http://ironpython.codeplex.com/WorkItem/View.aspx?WorkItemId=12907
>
> That being said the more test cases for this the better – this option 
> isn’t really documented. Also if anyone can provide insight into 
> exactly how this is supposed to behave that’d be great. But my guess 
> is that we’ll have a tail of bugs around this until we can nail down 
> the behavior and get it right. Anyway, I’ll take a look at fixing the 
> existing issues here. Wanting to implement a REPL from Python seems 
> like it’s the thing everyone likes to try J
>

I wonder if it isn't in fact caused by the fact that the repr of an 
IronPython syntax error doesn't change if the you add new lines to the 
source code you are compiling. As you can see the code below is actually 
comparing the error repr with newline added to see what *kind* of 
SyntaxError it is (horrible I know but this is what the code module does).

There is a codeplex issue for this specifically but I can't find it.

Michael

> *From:* users-bounces at lists.ironpython.com 
> [mailto:users-bounces at lists.ironpython.com] *On Behalf Of *Mike Krell
> *Sent:* Monday, May 25, 2009 10:00 AM
> *To:* Discussion of IronPython
> *Subject:* [IronPython] IPython is breathing but there's a compile() 
> problem
>
> Now that 2.6B1 has frames support, I've started playing with 
> IronPython under IPython again. I've managed to get a command prompt 
> up (some modules are missing, but the only crucial one is codeop, 
> which I stole from the standard distribution).
>
> However, there's a problem with entering multiline code snippets 
> interactively. With CPython, this looks like:
>
> In [21]: if 1:
> ....: if 1:
> ....:
>
> (The indentation looks wrong without a fixed-width font, but you get 
> the idea.)
>
> With IronPython, the second "if 1:" line blows up with a syntax error. 
> This boils down to a difference in the way the compile() builtin works 
> as used by the codeop module.
>
> I've written this up as a bug at codeplex. Please vote for the bug here:
>
> http://ironpython.codeplex.com/WorkItem/View.aspx?WorkItemId=22692
>
> It would be awesome if we could have a good IronPython + IPython story 
> before 2.6 is released!
>
> Below are more details about the problem as described in the bug 
> description.
>
> Mike
>
> --------------------------------------------------------------------------------
>
> bug description at codeplex follows
>
> compile() behaves differently than in CPython in the presence of 
> incomplete multiline code snippets. Fixing this incompatiblity is 
> necessary for running IronPython under IPython.
>
> Here is a sample program illustrating the problem. The program is a 
> modification of the code used in the standard codeop module by IPython 
> to determine when to provide a continuation prompt for a multiline 
> snippet.
>
> --------------------------------------------------------------------------------
>
> def testcompile(source, flags):
>
> err = err1 = err2 = None
> code = code1 = code2 = None
>
> try:
> code = compile(source, "dummy", "single", flags, 1)
> except SyntaxError, err:
> pass
>
> try:
> code1 = compile(source + "\n", "dummy", "single", flags, 1)
> except SyntaxError, err1:
> pass
>
> try:
> code2 = compile(source + "\n\n", "dummy", "single", flags, 1)
> except SyntaxError, err2:
> pass
>
> print "for source = '%s' and flags = %d" % (source, flags),
>
> if code:
> print "Syntax valid"
> elif not code1 and repr(err1) == repr(err2):
> print "Syntax error!"
> print
> print "err1:", repr(err1)
> print "err2:", repr(err2)
> else:
> print "Continue on next line"
> print
> print "err1:", repr(err1)
> print "err2:", repr(err2)
>
> print
>
> # 0x200 is PyCF_DONT_IMPLY_DEDENT
>
> testcompile("if 1:", 0x200)
> testcompile("if 1:", 0)
>
> testcompile("if 1:\n if 1:", 0x200)
> testcompile("if 1:\n if 1:", 0)
>
> --------------------------------------------------------------------------------
>
> Under CPython (2.6.1) the output is:
>
> --------------------------------------------------------------------------------
>
> for source = 'if 1:' and flags = 512 Continue on next line
>
> err1: SyntaxError('unexpected EOF while parsing', ('dummy', 1, 6, 'if 
> 1:\n'))
> err2: IndentationError('expected an indented block', ('dummy', 2, 1, 
> '\n'))
>
> for source = 'if 1:' and flags = 0 Continue on next line
>
> err1: SyntaxError('unexpected EOF while parsing', ('dummy', 1, 6, 'if 
> 1:\n'))
> err2: IndentationError('expected an indented block', ('dummy', 2, 1, 
> '\n'))
>
> for source = 'if 1:
> if 1:' and flags = 512 Continue on next line
>
> err1: IndentationError('expected an indented block', ('dummy', 2, 8, ' 
> if 1:\n'))
> err2: IndentationError('expected an indented block', ('dummy', 3, 1, 
> '\n'))
>
> for source = 'if 1:
> if 1:' and flags = 0 Continue on next line
>
> err1: IndentationError('expected an indented block', ('dummy', 2, 8, ' 
> if 1:\n'))
> err2: IndentationError('expected an indented block', ('dummy', 3, 1, 
> '\n'))
>
> --------------------------------------------------------------------------------
>
>
> In all cases the code correctly outputs "Continue on next line" since 
> both snippets are incomplete but otherwise valid python.
>
> For IronPython 2.6 Beta 1 the output is:
>
> --------------------------------------------------------------------------------
>
> for source = 'if 1:' and flags = 512 Continue on next line
>
> err1: IndentationError("unexpected token '<eof>'", ('dummy', 2, 1, ''))
> err2: IndentationError("unexpected token '<eof>'", ('dummy', 3, 1, ''))
>
> for source = 'if 1:' and flags = 0 Continue on next line
>
> err1: IndentationError("unexpected token '<eof>'", ('dummy', 2, 1, ''))
> err2: IndentationError("unexpected token '<eof>'", ('dummy', 3, 1, ''))
>
> for source = 'if 1:
> if 1:' and flags = 512 Syntax error!
>
> err1: IndentationError("unexpected token '<eof>'", ('dummy', 2, 8, ' 
> if 1:\n'))
> err2: IndentationError("unexpected token '<eof>'", ('dummy', 2, 8, ' 
> if 1:\n'))
>
> for source = 'if 1:
> if 1:' and flags = 0 Syntax error!
>
> err1: IndentationError("unexpected token '<eof>'", ('dummy', 2, 8, ' 
> if 1:\n'))
> err2: IndentationError("unexpected token '<eof>'", ('dummy', 2, 8, ' 
> if 1:\n'))
>
> --------------------------------------------------------------------------------
>
> The second snippet is misinterpreted as being a syntax error instead 
> of merely incomplete.
>
> This is very similar to the issue described here:
>
> http://www.voidspace.org.uk/python/weblog/arch_d7_2007_09_22.shtml#e834
>
> But it apparently doesn't have to do with PyCF_DONT_IMPLY_DEDENT per 
> se because the CPython output is the same whether this flag is passed 
> or not.
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog