Recommendation of a parser generator

[Andrew Dalke]

Fortepianissimo:
> I'm about to start a new project which will be mostly written in
> Python. The first task is to parse some formula-like expressions into
> an internal data structure so they can be evaluated.

How close is this formula language to Python's?  For other projects
I've punted the heavy work to Python's own parser, then filled in
the bits I needed.  For example, suppose you have the expression

   a.b + c + s.find('d')

>>> import compiler
>>> from compiler import visitor
>>> s = "a.b + c + s.find('d')"
>>> class GetNames(visitor.ASTVisitor):
...    def __init__(self):
...        self.names = {}
...    def visitName(self, obj):
...        self.names[obj.name] = 1
...

>>> a = compiler.parse(s)
>>> names = compiler.walk(a, GetNames()).names.keys()
>>> names
['a', 'c', 's']
>>>

Then get the values for a, c, and s, put them into a dict, and

>>> class A:
...     b = 5
...
>>> eval(s, {"a": A, "c": 3, "s": ""})
7
>>>

(Assuming I didn't make any mistakes - it's modified from an earlier
exchange Alex and I had in c.l.py, titled "classes derived from dict
and eval" and I didn't test all the changes.)

Failing that, I've been happy with SPARK as a parser generator,
but as you read in the paper, it's slow compared to the other parsers
that were benchmarked.

> This parser would be run extensively in the future, so speed is a
> consideration,

Why is the parser performance the problem?  Most of the time
is spent evaluating the result, right?  That's post-parsing.

The only time to worry about parsing performance is if you have a
lot of different expressions coming in.  Otherwise, just cache the
results, as Python does with .pyc files.

> I'd appreciate very much some expert suggestions from the group, like
> on the speed, flexibility, portability, and the future prospect (like
> to be adopted as the standard etc.).

I too would like a standard parser generator for Python.  I don't know
the status of that activity.  As it is, SPARK is small enough that it's
easy for me to include in my projects.

                    Andrew
                    dalke at dalkescientific.com