[pypy-dev] Work plan for PyPy

[Armin Rigo]

Hi all,

Here is a summary of what we discussed today on #pypy-sync.  We tried to
list the tasks that now lie ahead of us.  There are a number of
relatively independent tasks, but there is also a knot of dependent
tasks that we'll have to untie bit by bit.

RCtypes and the extension compiler are somewhat of a blocker for other
tasks.  A controversial solution is to drop them.  This means:

* To support calling external libraries, we would instead shift the
  focus on supporting the full ctypes at app-level.  This means that we
  could support out of the box the same ctypes-based pure Python modules
  as CPython.  There are issues here (e.g. is the regular ctypes really
  suitable to wrap large libraries?) but they are not worse than with
  rctypes.
  
* Of course, we keep the general support for compiling stand-alone
  RPython programs, even if rctypes dies.

* In RPython, fijal worked on a more basic replacement for rctypes,
  called rtti (almost finished already).  It's a PyPy-specific interface
  to call external functions in RPython.  Calls written in this
  interface can be compiled to good C code, but testing is a bit less
  convenient than with rctypes because rtti calls cannot be directly
  executed on top of CPython.

* The CPy Object Space is going away initially too because it is based
  on rctypes.  It should not be too hard to port it to rtti in order to
  continue supporting compiling extension modules for CPython.  We need
  to think a bit more here - how can we compile ext modules for Jython
  and IronPython? - but it's likely not a high priority at the moment.

Any comments or objections?  Are there people that already make heavy
use of rctypes, or the extension compiler?


The graph of dependent tasks ("A --- B"  means that B depends on A):


  merge rtti branch ---- app-level ctypes ---- kill rctypes and extcompiler
                    \                        /   \
                     \-- port our modules ---    kill keepalives
                      \                            \
                       -- pyrepl                    -- moving GCs
                                                   /
                   _______________________________/
                  /
                 /
  malloc cleanup ----- gc transform polishing
                     /           \
  fix weakrefs -----/             \
                                   \
                                    -- c/llvm backend sharing


Additionally, some of the independent tasks mentioned:

* move some app-level parts of the Python interpreter to interp-level
* complete the support for Python 2.5
* switch to LLVM 2.0
* port the stackless transform to ootypesystem
* JIT work


A bientot,

Armin.