Fun Python Projects

Wednesday, October 29, 2003, 3:39 p.m.

Brett Cannon asked for ideas for a master's thesis related to Python. He got some interesting responses. Neil Schemenauer came up with most of the topics listed here, but his topics are dear to my heart. (Neil's suggestions are in italics.)

• Macros for Python. There are lots of difficult issues here. One of the great strengths of macros in Lisp is quasiquotation. You need some other mechanism for manipulating the program as an AST (Maya) or concrete syntax (JSE).
• Maybe macros aren't exactly the right thing. The compiler package had two original goals; one was to provide a tool for developing Python variants. It's been used that way in Quixote's Python Template Language (PTL) and Zope's Restricted Python.

  The key problem is that there's no good way to extend the syntax of the base language and still get it parsed. The compiler package doesn't have a parser. It looks like Polyglot realizes for Java what I had hoped to do with the compiler package.

  Question: Is there a boundary between what can be accomplished by macros and what requires other mechanisms for extending the language? Could the Polyglot extensions be implemented in Maya?

• Martin von Löwis has a suggestion that could benefit from the previous tools. Design a decent way to handle attributes and declarations in Python code. What is a clear and concise way to write that a method is static?
• Build a refactoring code editor that works using the AST. Refactoring tools where the other impetus for work on the compiler package and later the AST branch. Perhaps the refactor tools code be integrated with Eclipse.
• Optimize access to global variables and builtins. PEP 267 describes my ideas. See also my Faster namespaces talk from developer's day at the 10th Python conference.
• My suggestion to Brett was to look at Erik Meijer's work on integrated language support for XML and SQL.
• Implement an object system that supports multiple dispatch. You can look at Dylan and Goo for ideas. (Note to self: Object-Oriented Programming in Common Lisp has been sitting on your bookshelf for a year now.)
• Analyze the memory usage of some interesting Python programs. Berger, Zorn, and McKinley argue that state of the art, general purpose memory allocators usually outperform custom allocators. Python is full of custom allocators. Does it really need them? How about a serious test of the Boehm-Demers-Weiser collector.
• Look at making the GC mark-and-sweep. You will need to provide it explict roots. Is it worth doing? Mark-and-sweep would require changes to extension modules since they don't expose roots to the interpreter.
• More radically, look at Chicken and it's GC. Henry Baker's "Cheney on the M.T.A" is very clever, IMHO, and could be used instead of Python's reference counting. Build a limited Python interpreter based on this idea and evaluate it.
• Dennis Allison suggests getting rid of the GIL. It sounds like a fun project, but a big one. The PyPy folks are headed in that direction, although there is lots of jargon about object spaces that makes it hard to follow. If it were up to me, I'd start with a small, fast VM (perhaps Armin's IVM) and build a stackless Python on top of it. I'd punt on C extensions (FFI) at first and see how far you can get with pure Python code.
• Martin von Löwis suggestions working on persistent system images for Python, ala Smalltalk.