Need suggestion to speed up code...

[greg jorgensen]

On Thu, 10 May 2001, Roy Smith wrote:

> I need to split up a string into a list of fields.  The strings are value
> lists from SQL statements, and look something like this:
>
> (1, 'foo', 'bar', 34, 3.14159, 'an imbedded comma, this sting has', 'this
> one isn''t so easy either')

Here's my humble offering:

# parses a string containing comma-separated values, which may be single-quoted
# returns list of values
def parseit(s):
    result = ''
    value = ''
    state = 'n'     # n=none, v=unquoted value,
                    # q=quoted string, e=embedded or end quote seen

    for c in s:
        if state == 'e' and c == "'":
            state = 'q'         # handle embedded paired quotes ''
            value += c
            continue
        elif state == 'e':
            state = 'n'         # end of quoted string
            result += value + '",'

        if state == 'n' and c in ' \t,':
            pass                # ignore whitespace & commas outside of value
        elif state == 'n' and c == "'":
            state = 'q'         # begin quoted string
            value = '"'
        elif state == 'n':
            state = 'v'         # begin unquoted value
            value = c
        elif state == 'v' and c == ',':
            state = 'n'          # end unquoted value
            result += value + ','
        elif state == 'v':
            value += c          # continue gathering value
        elif state == 'q' and c == "'":
            state = 'e'         # possible end of quoted string
        elif state == 'q':
            value += c          # continue gathering quoted string

    # finish off open state, if any
    if state == 'v':
        result += value
    elif state == 'q' or state == 'e':
        result += value + '"'

    return eval('[' + result + ']')


Exception handling deliberately left out so you can determine how to
handle execeptions.

Greg Jorgensen
Portland, Oregon, USA
gregj at pobox.com