speed

[Oliver Fromme]

Peter Kleiweg <in.aqua.scribis at nl.invalid> wrote:
 > I implemented a lexer in Pylly and compared it to the version I
 > had written in Flex. Processing 219062 lines took 0.9 seconds in
 > C (from Flex), and 5 minutes 54 second in Python (from Pylly), a
 > ratio of 393 to 1.
 > 
 > Is this normal for Python, or does Flex produce better parsers
 > than Pylly? I have been looking at the code produced by Flex to
 > see if I could translate it to Python automaticly. But it has a
 > lot of goto statements, and I haven't figured out how to
 > translate those to Python efficiently.
 > 
 > What are the average times used for text processing of Python
 > compared to C?

I don't know Pylly, but I guess it generates a parser using
a finite automaton -- just like lex/flex, except it handles
every single character in Python, wheres lex/flex will lead
to compiled C code.  That would explain the speed difference.

When I have to parse something in Python, I try to do that
using things like string.split(), string.find(), the "re"
module etc.  Those things are written in C, therefore they
are fast enough for most applications.  There are also some
modules for specialized cases, such as "ConfigParser" and
"shlex".  See the Python Library Reference.

Best regards
   Oliver

-- 
Oliver Fromme, Konrad-Celtis-Str. 72, 81369 Munich, Germany

``All that we see or seem is just a dream within a dream.''
(E. A. Poe)