[Python-Dev] [ANN] "compiler" package resurrected

[Paul Sokolovsky]

Hello,

I'm sorry for posting here and not to python-announce, somehow I think
(perhaps naively) that it may be of interest to people who are
interested in Python development. At the very least, creation of the
original package is (very likely, I didn't trace up to that) was
discussed on python-dev, its removal was discussed on python-dev, so
why revival of it can't be noted here?

Because, turns out, in old good times, there was a bytecode compiler
written in Python, and even as a part of stdlib. Well, actually it's
still there in the latest 2.7 release, with a proud notice: "Remove
in Python3". The point is that I'm with Python since 1.5 times, and
never knew about this package. I'd generally consider that to be
something to be ashamed and hush of, but unfortunately I found that to
be recurring pattern: people interested in playing with a Python
compiler discover "by a chance" and "suddenly" that they should look no
beyond the stdlib for their (usually pretty simple for starters) needs
- oftentimes, after they already started on the more effortful path
(I faithfully spent 2 days on trying to extract a bytecode compiler
from PyPy first). 

With that intro, here's it - the port of Python2 compiler package
(https://docs.python.org/2/library/compiler.html) to Python3:

https://github.com/pfalcon/python-compiler

Currently, it generates bytecode compatible with CPython3.5, and is
able to compile its entire Lib/ (which includes not just stdlib modules
per se, but also tests, and the real "teeth-cracking" stuff is there of
course), except for test_super.py, for which current implementation's
teeth are indeed too weak yet.  

Now that it passes the compile-stdlib test, the idea is to refactor
it into something which can be easily played with and extended. We'll
see how it goes.

As one of the example of what's intended to be easily done with it, see
thread
https://mail.python.org/pipermail/python-dev/2019-January/155991.html .
I started it when updating the codegen for Python3, but also shows the
intended purpose - it would easy to analyze if an except handler body
contains "del exc" and if not, skip generating superfluous bytecode.  

-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com