Jeremy Hylton : weblog : 2004-03-20

PyCon sprints, day 1

Saturday, March 20, 2004

PyCon sprints started today. There were several people working in each room by the time I arrived at GWU on Saturday morning. I did a quick headcount after lunch; it looked like we had about 35 people working then.

The core sprint was well attended. There was a big contingent of locals, including Andrew, Barry, Fred, Neal, Neil, and Tim. Nick Bastin and some other folks from OPNET helped out.

My project for the weekend was the AST branch. Tim Peters observed that we will have to finish the AST branch this year or make it the annual sprint topic. Neal Norwitz, Neil Schemenauer, and I got to work on the list of open bugs. Fixed: doc strings, name mangling (__variables), partial fix for encoding declarations.

It was a great help to have some many core developers around. Martin von L÷wis helped Neil with Unicode issues, and Neil and Armin Rigo looked at stack depth calculation. They concluded that we would be better off calculating stack depth incrementally, just like the old compiler. Opcodes like MAKE_CLOSURE have stack effects that depend on the previous value on the stack, making a separate calculation difficult. (The compiler package does a conservative approximation of the correct depth.)

The major effort for the AST branch is done. Code generation works for nearly every source construct. The remaining work is slow going, though. The bug tend to be subtle and take a long time to fix. We started on fixing some problems with closures, but didn't get very far by the end of the day.

  • The representation for function doc strings seems to magical: It is the first element of co_consts, unless that element is None. Why not have an explicit co_docstring slot?
  • The line number table is so delicate and restricted. Armin Rigo suggested a scheme for mapping ranges of bytecodes to ranges of lines.

    It was easy to get distracted with so many core sprints in one place. Nick Bastin was working on a change to the profiler so that it tracks calls to builtin and C extension functions. Armin and Martin looked at a variety of ways to speedup Python functions. They wanted to streamline frame allocation. One possibility discussed on python-dev was to cache unused frames on the code object so that there was less initialization to do; another was to make frames smaller. I think they concluded that most of the benefit was achieved by replacing memset() with loops. Ha!

    We discussed a lot of ideas for making frames smaller, which I still think is a good idea. A smaller frame uses less memory. If it is small enough to use pymalloc, we might not need a custom freelist for frames. Some options for shrinking frames: Eliminate the block stack or allocate only the space that is needed for it. Remove fields that can be calculated from the code object. Get rid of the thread state.

    I think we concluded that it might be possible to remove the block stack completely. The block stack is used for try/except, try/finally, and for breaking out of loops. We can save space by eliminating the block stack. We could also eliminate the opcodes to setup loops and exceptions if we could determine all the details at compile time, e.g. an exception handling table.