lexing nested parenthesis

[Bengt Richter]

On Sun, 28 Jul 2002 23:57:21 -0400, Dave Cinege <dcinege at psychosis.com> wrote:

>It's nice when you open up your reference book, and immediately
>find a section on what you want to do. 
>
>It sucks large donkey rocks when that section is titled:
>"Difficulties and Impossiblies" (Mastering Regular Expressions)
>
>I've been playing with shlex, and think in my application I'm
>better off minimizing or avoiding it. 
>
>Primarily I want to do search, evaluate, modify, and replace
>on a line by line basis. The diffculty comes with matching
>arbitrary nested constructs, IE nested parenthesis.
>
>IE
>	if 1 and (var1 or ?(-d /etc/)):
>
If you have defined the syntax of the above (I assume it's literal text),
perhaps you would gain by adding quotes, and using a symbol instead of '?', e.g.,

	if 1 and (var1 or qm('-d /etc/')):

would already be legal Python.

>I want to find ?(.*), but not runneth under or over.
>Evaluating a token at a time will become disgusting. It also
>seems redundant as you can see it's the same parsing rules as
>Python has. (And in fact that is what the line will be
>converted to.)
Maybe a few examples with un-nested and nested parens (and nested ?(...) constructs??)
together with what Python you would like to have them transformed to would get
you some useful help.

>
>Can anyone recommend a 'Better Way'? I'm thinking maybe subclassing
>generate_tokens.
'Better' is a waste of time unless we're working on the real problem ;-)

Regards,
Bengt Richter