[Python-Dev] Towards selective compilation to native code?

[Vladimir Marangozov]

There was a cool seminar the other day here, focusing on embedded &
real-time Java, so I'll throw here some impressions while they're fresh.
One of the presenters talked extensively about compilation strategies and
techniques for Java bytecode to native code, and while I know that Java
is not Python, it would be interesting to, at least, try to project some
of the ideas to Python in the long run, where applicable.

Needless to say, the guy & his environment (ex-OSF, now Silicomp) are
experts in compiler technology and I'm not in a position to comment on
everything, but you may find the second half of the slides enlightening: 
http://cgi-serv.inrialpes.fr/InTech/2000-06-15/ejava-silicomp.ppt

So, the interesting part of the talk was about a "new" compiler
architecture, built upon the concept of "Flash compiling" (quite
unappropriate name), which produces a mixture of the following:

  1. Bytecode
  2. Native code
  3. Native code is compiled selectively on demand and/or ahead of time
     (!= JIT) for "expensive" methods.
  4. Expensive methods are, on one hand, detected by dynamic profiling
     of the bytecode, and OTOH, determined ahead of time according to the
     following 2 simple creteria:
       - methods containing loops
       - methods calling native code from bytecode
           (to avoid bytecode to native and vice versa arg conversion paths
            which are expensive -- btw Python has the same perf. bottleneck)
  5. Everything relies on dyn loading & late binding of native code
     (front-end loads dynamically the back-end, native code is stored
     in the code stream, but linked incrementally & dynamically during
     execution)
  6. The whole thing is designed to avoid code explosion and to preserve
     the advantage of bytecode == small code size, and the presented
     preliminary results are quite optimistic -- the figures differ depending
     on the code size / speed tradeoff.

Of course, this is Java and there's static typing in there, but it
would be interesting to see whether Python can steal something from these
ideas. For instance, I'm trying to foresee what kind of static typing
(i.e. how much static info) would be appropriate in Python from this
point of view but I fail to obtain any realistic image :).
I see most parts of the dyn loading / late binding aspect though --
they are typical for Python.

Hm. Sounds like I'll be harrassing Tim to make him finally write something
valuable for Python ;-)

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252