[Types-sig] "Mobius2" -- high performance Python-extensible Python parser

[Jeff Epler]

On Thu, Apr 12, 2001 at 11:19:26AM -0700, Paul Prescod wrote:
> This sounds really cool. The docs look pretty old so perhaps you could
> give us a quick overview of how you pulled off this neat trick!

The docs apply to "Mobius1", the SPARK-based version.  So, yeah,
they're woefully outdated (and sparse even for what they describe)

> Doesn't Python do parser generation at build time? Is Mobius Python
> a parser generator that depends on a compiler or does it allow you to
> load new parsers at runtime with a plain old Python installation?

The Python build process constructs a parser and writes it out as a
static table.  However, when in memory the table is immediately usable
as a parser without the intervening step of being written into a C file 
and compiled.

Only a few changes are needed to support the creation of the table at
runtime (and it's a fast process, 0.04 seconds to create the standard
grammar on a P2-350, or .25 seconds on a 486-75) and to wrap it as a
Python object with a .parse() method.  (calls PyParser_ParseString)

> I mean if Mobius parsers Python code faster than Python and it is
> runtime extensible then it seems like something that should be destined
> to replace Python's current parser! What's the downside?

Well, I'm confused by any difference in speed, given that the parser
code is Python's internal parser, and the tables generated should be
identical when using the standard grammar.  The only other difference
is that the tables are on the heap instead of with static variables.
Why this accounts for a 15% speed difference is beyond me.  The data
is laid down in a more advantageous manner than it is in the generated
"graminit.c" file?  Cache effects?

Jeff