[pypy-dev] Roadmap draft

[Martijn Faassen]

Hey,

The most important point I want to make is this one. The current 
discussion is about a roadmap towards CPython replacement (or other 
interpreter replacements). This is a *different* goal from having PyPy 
being useful in *some* production settings. The latter is a more modest 
goal. It also shifts the focus: instead of aiming for parity with 
CPython, you aim to be *better* than CPython at *some* things, while not 
having parity in other areas.

So, you could have an interpreter that is not yet a full replacement for 
CPython being very useful for some tasks. This is where my primarily 
interest lies right now: it'll take a while before I can use a 
PyPy-derived interpreter instead of CPython for many tasks, but I'm 
interested in using it for at least *some* tasks.

Being better at some things than existing interpreters in production 
will validate the project in the minds of many, and should increase the 
mindshare of PyPy tremendously and attract enough contributors to reach 
the final goal of being better at *everything*. I don't want to detract 
from that goal of parity with existing interpreters (and beyond). I 
don't want to take away from that. I just want to frame the discussion 
in a different light, and highlight other goals. You don't need parity 
in all respects in order to go beyond existing interpreters in some.

I'll first note one way in which PyPy is already better than existing 
interpreters: flexibility. Unfortunately this benefit by itself is not 
going to be very useful in production itself. The idea is to exploit 
this flexibility. Flexibility is a goal of the project, but it's only a 
means, not a goal, of a production-interpreter project.

As to the goal of being a CPython replacement, one important realisation 
is that you might in fact reach replacement of CPython through a 
completely other route than the one sketched out. Who is to say that a 
CPython replacement has to be run through a C compiler, after all? The 
only requirement is for it to run Python code. If that Python code runs 
on top of .NET or JVM, so be it, as long as it runs, right? Of course as 
an *end* goal you might want third-party library support, but you could 
get quite far as a replacement of CPython in other ways without covering 
those.

So, what intermediate goals could you set that would make PyPy better 
than other interpreters in production settings? These are the main 
routes that I can see right now:

You could aim for compliance with CPython (and library support) - but 
then actually replacing CPython is a goal you can't compete on soon! 
Your compliance will shine the most on other backends, like .NET or JVM, 
and the feature would be access to libraries on those platforms, so 
integration features would be the most important. Any performance or 
JIT-related task is less important on the near term, while integration 
tasks are most important.

You could make compliance less important and aim for performance. In 
this case you could complete with CPython right away, even if your 
compliance and library support isn't perfect yet. You might only support 
Python 2.4 features and implement a limited set of libraries, and 
limited extension facilities, and still attract people. Performance and 
JIT related tasks are the most important.

You could also target one specialist platform (some embedded 
architecture, or some dedicated VM architecture, like the Google Android 
platform). You could make PyPy work way better than any competition on 
that. (Multi)platform support beyond Linux, OS/X and Windows becomes way 
more important then.

What your initial focus is would influence which tasks you focus on 
first a lot. I think this is important to keep in mind during the 
roadmap discussions. I'd of course also be curious to find out which 
ones you're most interested in as initial goals. Saying "all of them, 
right now" is not good enough. :)

Regards,

Martijn