regular expression: perl ==> python

[Fredrik Lundh]

Nick Craig-Wood wrote:

> I take your point. However I don't find the below very readable -
> making 5 small regexps into 1 big one, plus a game of count the
> brackets doesn't strike me as a huge win...

if you're doing that a lot, you might wish to create a helper function.

the undocumented sre.Scanner provides a ready-made mechanism for this
kind of RE matching; see

    http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1614344

for some discussion.

here's (a slight variation of) the code example they're talking about:

    def s_ident(scanner, token): return token
    def s_operator(scanner, token): return "op%s" % token
    def s_float(scanner, token): return float(token)
    def s_int(scanner, token): return int(token)

    scanner = sre.Scanner([
        (r"[a-zA-Z_]\w*", s_ident),
        (r"\d+\.\d*", s_float),
        (r"\d+", s_int),
        (r"=|\+|-|\*|/", s_operator),
        (r"\s+", None),
        ])

    >>> print scanner.scan("sum = 3*foo + 312.50 + bar")
    (['sum', 'op=', 3, 'op*', 'foo', 'op+', 312.5, 'op+', 'bar'], '')

</F>