[pypy-dev] Re: [pypy-sprint] vilnius sprint planning progress

[Armin Rigo]

Hi Holger,

On Fri, Oct 08, 2004 at 12:18:44PM +0200, holger krekel wrote:
> Armin, Samuele, it would be great if especially the two of you
> (or at least one of you) think hard about how to split up work
> for the translate-pypy goal so that multiple teams can work on
> it.  If you have ideas please post them to pypy-dev.  

As far as I can see now, this goal can be divided in three relatively
independent tasks:

A. Obtain the complete flow graph of the RPython part of PyPy.
B. Perform type inference and optimizations on the flow graph.
C. Generate C code from the flow graph.


A. PyPy -> FlowGraph
====================

See goal/translate_pypy.py; run it and fix things until this script
processes the whole of PyPy.

This requires updating things in PyPy when they are not RPythonic enough.
In particular, there are some more efforts to be done on "caches": lazily
built objects.  Generally, the code to build such objects is not RPython;
so for RPython, the objects must be built in advance.  The flow space must
force these caches to be completely built.  This part can be done
independently. The goal would be to get a complete graph, which the
existing genc.py can (mostly) already translate to C and run, for testing.
(This will not be extremely fast because genc.py doesn't use type
information now, but it should be a bit faster than the pure Python py.py.)


B. FlowGraph -> Optimized FlowGraph
===================================

Still open to discussion.  What exactly should be done here, and how?
An idea is that we could provide a set of rules that transform some
operations according to the type inferred for their arguments.  This would
introduce new operations that work on individual fields of PyObjects
instead of just calling PyObject_Xxx() on them.  Global analysis can
further discover when PyObject can be inlined into a parent, when they
don't need reference counters, etc.


C. Optimized FlowGraph -> C code
================================

See genc.py.  This not-too-long piece of code translates a regular flow
graph into C code, and it seems to work fine, mostly.  There are a few
missing RPython constructions (e.g. exception handling) that A will
generate.  In parallel, the optimizations introduced in B will produce
flow graphs with new kinds of operations whose support must then be added
to genc.py.  So if you'd like to work on genc.py, people working on A and
B will keep throwing at you new kind of flow graphs and optimization-related
data to support.


Armin