Topy news - still about pretty-printing

[François Pinard]

Hi, gang!  Here are a few more Topy news, for those who are interested.

Quick reminder: Topy is a tool meant to help converting from other languages
to Python.  This is a project under development, it is not usable yet.

Two weeks ago, I asked this list about pretty printers for Python.  Finding
none, I finally made a good stab on this problem, or at least, I think so.
The line wrapper which I temporarily put in the back-end is now gone.

Generating pretty printed Python sources has been an interesting problem
in Topy.  There were two main difficulties.  The first is that, for doing
a good job, Topy needed a more extensive representation of expressions,
right in the tree structure allowing front-ends to communicate with the
back-end, so I extended those structures as needed.  The second is that, to
stay speedy enough, generation should be interrupted as soon as we go over
line width, and when one wanders in trees while maintaining stacked state,
properly and efficiently backtracking everything required some thought :-).

Other news are that the Perl front-end now uses PLY instead of SPARK,
(the other front-ends still use SPARK).  I made small stunts so that PLY
diagnostics are precise to the column, to be compatible with the rest.
I also merged `miltopy', `pltopy' and `scmtopy' into a single `topy'.

Speaking of, I thought for a moment that if Topy had a Python front-end
(the back-end is already for Python), then Topy would offer a Python
pretty printer for free.  This would requires two things.  First, the
structure of trees for communicating between front-ends and the back-end
has be boosted, once more.  Second, I need a scanner and parser for Python.
I thought merely using the tokenizer/parser modules provided with Python,
and even experimented a bit with them.  The resolution is rough to lines,
Not columns, so some hackery wound be needed here as well (this is not
only for diagnostics, but for proper re-insertion of comments).  Also,
ASTs produced by the Python parser are surprisingly noisy, Topy would need
more code and overhead for cleaning these.  This all means that having a
Python reformatter in Topy would require more work than I expected.  Sigh!

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard