Parsing parameters with quotes

[Alex Martelli]

Giovanni Bajo wrote:

> 'foo "this is one" and this not':
> 
> and I want to output:
> 
> ["foo", "this is one", "and", "this", "not"]
> 
> Basically, a string.split() but must take into account quotes used to
> group as a single word (no escaping is supported within quotes). Now, is
> there already something in the python library to do this? My code is a bit

I think that the tokenizers that Python provides are all oriented to
considering the second token as *including* its delimiting quotes, e.g.:

import cStringIO, shlex
si = cStringIO.StringIO

xx = 'foo "this is one" and this is not'
print list(iter(shlex.shlex(si(xx)).get_token, ''))

emits:

['foo', '"this is one"', 'and', 'this', 'is', 'not']

and similarly for module tokenize:

import tokenize
print [t[1] for t in tokenize.generate_tokens(si(xx).readline) if t[1]]

emits:

['foo', '"this is one"', 'and', 'this', 'is', 'not']

However, removing the quotes when present shouldn't be too hard, I think.


Alex