Semi-static analysis (was Re: Partial Parsing of Python)

[Peter Eckersley]

On Mon, Jul 01, 2002 at 10:45:13AM +0200, holger krekel wrote:
> Peter Eckersley wrote:
> > [me]
> > > Actually the REAL problem to solve first is to have Multi-Line editing. 
> > > Then all sorts of partial parsers *and* partial evaluation.  Python is
> > > just too damn dynamic to have that one easily.
> > > 
> > 
> > Oh, which reminds me of something else.  I also have problems with
> > side-effects ("heisenberg" effects)-- it will be necessary to get rid of
> > these.  
> 
> But you certainly know that this isn't generally possible? Take

Yep, because in Python you can't know the return type of all functions.
My proposal was to add notes to lots of functions, and handle the cases
where you _do_ know the return type.
> 
>     somestrangefile.read().<tab>
> 
A perfect example!

If <somestrangefile> is a builtin file object, it should have a read()
method annotated as follows:

pure == False (calling the method may cause side effects - do not
               eval() it in vain!)

return_type == types.StringType (so we know we can complete using normal string
                                methods)

If <somestrangefile> is a user defined class, the user should be able
to easily provide the above information.  Even if it this hasn't been done, 
there should be *some* cases where we can analyse the python code
to extract this information[*].  If a method is impure, and we don't
know the return type, then we just give up and don't evaluate it. 

> for example.  The next best bet (read: hack) is to fork off a process
> which does the evaluation of an Subexpression.  This way you'd catch
> 90% of in-place modifications.  Doesn't work for files.  But then how
> often are you doing dynamic network/file communication on the commandline?   
> This might change once we have multiline editing.  But one after three
> others :-) 
> 
> > My feeling was that this could be done using purity analysis and
> > return-type annotation.  Many of the important cases can be handled by
> > just adding information about the builtin modules, but it might also
> > be possible to do generalised python purity analysis.  Has anyone tried
> > this?
> 
> Not me. I don't even now exactly what you are talking about :-)
> 
I was asking about the analysis marked [*] above.

> > Hmmm, I agree that one needs to escape readline/curses - but SDL sounds
> > like overkill, since I imagine you'll have to re-implement lots of basic
> > widgets on top of it.  I picked GTK, and have found it generally
> > suitable, despite a few quirks.
> 
> Reflecting our EuroPython discussion, GTK or QT seems to be heavyweight.  
> We don't need widgets but the ability to put scalable fonts on a screen.  
> Having sprites and game-level performance can't hurt either :-)
> It's very worth a try. 

I don't know enough about GTK's implementation to know if it's efficient
at what it does (I was assuming it would be, but I could easily be
wrong).  I've certainly found lots of the widget behaviour very useful,
but it might not be all that hard that hard to re-implement.

-- 
Peter Eckersley
Department of Computer Science   &                  mailto:pde at cs.mu.oz.au 
IP Research Institute of Australia             http://www.cs.mu.oz.au/~pde
The University of Melbourne