[pypy-dev] Hildesheim sprint report 4 + 5

[Carl Friedrich Bolz]

Hi PyPy-dev,

somre more news from the Hildesheim sprint (I try to do this in some 
sort of chronological order although the days are kind of mixed in my mind):

Thursday:
    - Holger and Richard worked on refactoring the mess that is the
      option handling of PyPy. Now it is possible to. Now the handling
      of options is unified in one place. It is now possible to tell
      PyPy exactly which modules to fake and for which to import our
      own implementation.

    - Armin and me completed our implementation of the math module, which
      is now completely translatable.

    - One of the big problems of the last days for the translation effort
      was that we had a bootstrap problem: our interplevel compiler and
      parser are not entirely done yet, so we need applevel version.
      These need to be imported using a parser and a compiler. Damn.
      At first we tried to get pyc-file support working, which involved
      finishing marshal. This approach was dropped because marshal used
      codecs which lives on applevel and therefore cannot be imported.
      Instead Armin came up with a hack: Basically we cheat during
      bootstrapping and use the underlying compiler and parser. Later
      (before translation) it is replaced by a real applevel
      implementation. Armin worked on this the basically the whole
      Thursday afternoon. In the evening it turned out that this approach
      leads to *exactly* the same problem. Armin, Samuele, Christian and
      me fixed Thursday evening. It kind of worked in the end but it
      introduced strange translation problems that Armin and Samuele
      fixed Thursday night (last checkin message: 3:45am).

Friday:
    - Holger worked on py.test's stdout capturing because after some
      recent refactoring it didn't work any more for PyPy applevel tests.

    - Christian started reimplementing marshal on interplevel.

    - Richard tried once more to get LLVM prebuilt constants to work.

    - basically the day was spent trying to translate pypy
      (which involves watching PyPy: The Movie -- Two and a Half Years
      Later. We sit and watch the translator running on the beamer until
      it crashes). After a few typer errors and a bug in exception
      handling we actually managed to reach the point where
      the translator generated C code! To reach this you
      need quite a lot of RAM: something around 1GB just for the
      translator, annotator, rtyper and genc. The whole generated code is
      something like 120 Mb big (all in one file!). Surprisingly the
      compiler even manages to do something with it (which just means
      that it produces errors and does not just crash). While trying to
      actually translate PyPy we found various ommissions in various
      places: Some external functions were missing. While doing this the
      following compiler error was found:
          "testin_1.c:306344: error: 
'Is_Unsigned_Division_Really_Useful'
           undeclared (first use this function)"
      well, obviously it is :-)

    - Armin worked on fixing an order problem: Some struct declarations
      have to be written to the C file in the right order because they
      depend on each other.

    - Samuele and me tried to run the flowgraph of PyPy on the
      llinterpreter (yay, yet another level!) which turned up some
      bugs in the RTyper and in the PyPy interpreter but mostly
      space operations that were still missing from the llinterpreter,
      mostly overflow and division by zero checking.


Well, I'm pretty sure that I forgot some things but I'm too tired now.

G'night,

Carl Friedrich