[pypy-dev] Re: Project suggestions

[Aurélien Campéas]

Michael Hudson a écrit :
> RPython is a subset of python with the main constraint being able to
> do some level of static type analysis (for full python, the amount of
> static ananlysis you can do is really very small, you can read Brett
> Cannon's thesis about this:
> 
> http://www.ocf.berkeley.edu/~bac/thesis.pdf).
> 
> So, when translating, the code that implements the interpreter
> (roughly interpreter/*.py, objspace/std/*.py) is imported and a flow
> graph built from it.  This is then annotated (code in annotator/*.py),
> a fairly hairy process.  This (for the C and LLVM backends, at least)
> is then turned into a graph containing low level operations (like
> "dereference this pointer").
> 
> Python is just the language all of the above happens to be implemented
> in (and also the interpreter/*.py code is involved in making a flow
> graph which includes itself, but this isn't that important -- just
> confusing :).

So the same graph can exist in three states : raw python, annotated 
python (possible whenever the raw python matches rpython), low-level stuff ?

Then, to translate unrestricted python, I have to work on the first 
pass/state of the graph.


>>>>In fact, the distance from de-sugared python to the opcode is
>>>>unknown to me (even though I suspect there is no 1:1 mapping between
>>>>the two). Whatever, that could be the start of a strategy to
>>>>translate python towards other high-level languages (ruby, or js as
>>>>in the other thread) without paying the full price of opcode
>>>>interpretation in the target (that is : parts which are semantically
>>>>similar could be easily translated, others would be -costily-
>>>>emulated).
>>>
>>>Thing is, I don't know how feasible this is.  It's pretty hard,
>>>without some kind of type inference, to translate, say this Python:
>>>    a + b
>>>into anything significantly more efficient than this Common Lisp:
>>>    (py:add a b)
>>
>>The mere fact that it will be compiled will make it more efficient I
>>guess. I mean, not on CLISP, but with a real lisp compiler.
> 
> 
> Not much.  The "interpreter overhead" for Python is usually estimated
> at about 20%, though it obviously depends what code you're running to
> some extent.

hmmmm, 20% related to what ? Is this an observed quantity from 
benchmarking, say, C-translated rpython vs CPython ?


Also, "on average" can, as you say, cover a lot of differences depending 
on what we have to do.

>  
>>>And making type inference possible is what RPython is all about.
>>
>>Sure, but then, it is a restricted subset of Python, and I like python
>>completely unrestricted ;)
> 
> 
> Well, so do we all, but then you can't have type inference.  There is
> no simple answer to this.

Has the possibility to extend python with optional type annotations been 
studied ? (I guess the pypy-dev archives could answer this)

> 
> 
>>>You could make #'py:add a generic function and see if a given CLOS
>>>implementation is fast enough to give a useful speed (but I think the
>>>coercion rules would probably drive you insane first).
>>
>>In this case, CLOS would add overhead. In fact the python add operator
>>(and some arith. ops) seems close enough to the lisp one that one can
>>be tempted to translate in the lisp equivalent with only minor
>>adaptation (like the printer appending an "L" on bignums, etc.).
> 
> 
> Er... no.  #'cl:+ is not that much like python's + (e.g. the latter
> operates on strings and you can't overload the former).

Thanks for reminding me that.

> 
>>I am not sure I would use CLOS at all, in fact (at least for a first
>>attempt at producing a lisp backend).
>  
> Fair enough.
> 

... but the more I think about it, it looks like CLOS might help with 
these overloaded operators ...

> 
>>BTW, what's this "insanity with coercion rules" that you mention -
>>can you expand a little on this ?
> 
> 
> Think about things like 2L**135 < 1.0e40 or range(10)*3 or... mixed
> type operations are not that simple in Python.
> 

... and coercion rules ...

Well, for now I will go on tweaking a CL backend based on pyrex's one. I 
guess I will discover the real problems as I progress.

Thanks also to Bert and Samuele,
Aurélien.