[pypy-dev] Miasma: x86 machine code generation in Python

[Darius Bacon]

holger krekel <hpk at trillke.net> wrote:

[x86 code emission]
> Sure interesting! Except i don't know enough Scheme.  Could you
> (or somebody) else explain what the challenges were and how
> you solved them? 

The Scheme code generates Python code from a table of instruction
descriptions; you can just use the pregenerated Python directly
without bothering about the Scheme, as long as you're happy with the
interface it gives you.  I wouldn't have used Scheme if I were
starting a Python project from scratch, but most of the code was
already written.

I don't expect this to be right for Psyco as is, because the code
generator I originally built this for emitted instructions back to
front, in the reverse order of execution -- like this:

def prolog():
    x86.push_gv(edi)
    x86.push_gv(esi)
    x86.push_gv(ebx)
    x86.mov_gv_ev(ebp, reg(esp))
    x86.push_gv(ebp)

which emits the conventional function prolog

    push ebp
    mov ebp, esp
    push ebx
    push esi
    push edi

Changing it to work forwards instead, like Psyco, would require
rebuilding the whole 2000 lines or so of generated code, so you would
want to mess with the Scheme for that.  It also needs more addressing
modes and a way of doing backpatching.

The challenges, well, the main one was just working through the Intel
reference manual and discovering the occasional error in it --
checking the output against gas was a big help, though one of the
differences turned out to be a bug in gas instead.  Internally,
there's a table with instruction descriptions in a little language
modeled after Intel's documentation, and we generate code for each
instruction emitter from that.  This was all done years ago for a Lisp
OS project.  Later I made an attempt to factor out the general logic
from language-specific code emission stuff, which is why the Python
version only took a few hours hacking to get running.  It's not very
fancy but enough to start on.

Darius