Tokenize

[Alan Kennedy]

Andrew Dalke wrote:

> Another option is the little-known 'shlex' module, part of the standard
> library.

[snip]

> Also, with __iter__ in newer Pythons, if these module were useful
> then it would be nice if "for token in shlex..." worked.

Python 2.3b1 (#40, Apr 25 2003, 19:06:24)

>>> import shlex
>>> import StringIO
>>> s = StringIO.StringIO("""ls -lart "have space.*" will travel""")
>>> for ix, tok in enumerate(shlex.shlex(s)):
...     print "%d: %s" % (ix, tok)
...
0: ls
1: -
2: lart
3: "have space.*"
4: will
5: travel
>>>

Thanks to Gustavo Niemeyer, who is credited in the code for
"Lib/shlex.py" as follows

# Posix compliance, split(), string arguments, and
# iterator interface by Gustavo Niemeyer, April 2003.

regards,

-- 
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/mailto/alan