Regex help needed

[Michael Spencer]

rh0dium wrote:
> Hi all,
> 
> I am using python to drive another tool using pexpect.  The values
> which I get back I would like to automatically put into a list if there
> is more than one return value. They provide me a way to see that the
> data is in set by parenthesising it.
> 
...

> 
> CAN SOMEONE PLEASE CLEAN THIS UP?
> 

How about using the Python tokenizer rather than re:

  >>> import cStringIO, tokenize
  ...
  >>> def get_tokens(source):
  ...     allowed_tokens = (tokenize.STRING, tokenize.OP)
  ...     src = cStringIO.StringIO(source).readline
  ...     src = tokenize.generate_tokens(src)
  ...     return (token[1] for token in src if token[0] in allowed_tokens)
  ...
  >>> def rest_eval(tokens):
  ...     output = []
  ...     for token in tokens:
  ...         if token == "(":
  ...             output.append(rest_eval(tokens))
  ...         elif token == ")":
  ...             return output
  ...         else:
  ...             output.append(token[1:-1])
  ...     return output
  ...
  >>> def parse(source):
  ...     source = source.splitlines()
  ...     original, rest = source[0], "\n".join(source[1:])
  ...     return original, rest_eval(get_tokens(rest))
  ...
  >>> sources = [
  ...     'someFunction\r\n "test" "foo"\r\n',
  ...     'someFunction\r\n "test  foo"\r\n',
  ...     'getVersion()\r\n"@(#)$CDS: icfb.exe version 5.1.0 05/22/2005 23:36 
(cicln01) $"\r\n',
  ...     'someFunction\r\n ("test" "test1" "foo aasdfasdf"\r\n "newline" 
"test2")\r\n']
  >>>
  >>> for data in sources: parse(data)
  ...
  ('someFunction', ['test', 'foo'])
  ('someFunction', ['test  foo'])
  ('getVersion()', ['@(#)$CDS: icfb.exe version 5.1.0 05/22/2005 23:36 (cicln01) 
$'])
  ('someFunction', [['test', 'test1', 'foo aasdfasdf', 'newline', 'test2']])
  >>>

Cheers

Michael