Nested function scope problem

[danielx]

Bruno Desthuilliers wrote:
> Josiah Manson a écrit :
> > I found that I was repeating the same couple of lines over and over in
> > a function and decided to split those lines into a nested function
> > after copying one too many minor changes all over. The only problem is
> > that my little helper function doesn't work! It claims that a variable
> > doesn't exist. If I move the variable declaration, it finds the
> > variable, but can't change it. Declaring the variable global in the
> > nested function doesn't work either.
> >
> > But, changing the variable in the containing scope is the whole purpose
> > of this helper function.
> >
> > I'm new to python, so there is probably some solution I haven't
> > encountered yet. Could you please suggest a nice clean solution? The
> > offending code is below. Thanks.
> >
> > def breakLine(s):
> > 	"""Break a string into a list of words and symbols.
> > 	"""
> > 	def addTok():
> > 		if len(tok) > 0:
>
>                  if tok:
>
> An empty sequence evals to False in a boolean context.
>
> > 			ls.append(tok)
> > 			tok = ''
> >

I can't figure out why Josiah's breakLine function won't work either. I
know Josiah has had his problem resolved, but I'd still like to know
why his func won't work. I'd like to redirect this discussion in that
direction, if I may.

>
> First point: the nested function only have access to names that exists
> in the enclosing namespace at the time it's defined.

Coming from lisp, that doesn't make very much sense, and I'm not sure
that's true. If you move the def for addTok bellow the lines that
initialize the locals of breakLines, you still get the same problem.

>
> Second point: a nested function cannot rebind names from the enclosing
> namespace. Note that in Python, rebinding a name and modifying the
> object bound to a name are very distinct operations.

I'm not sure that's the problem, because when I ran the debugger, the
problem is with the line that says if len(tok), not the one bellow it
which says tok = "". Regardless, my above issue seems to be overriding
this one.

>
> Third point : functions modifying their environment this way are usually
> considered bad form.

Again, this is coming from lisp, but I don't see anything wrong with
that :P.

***

After some experimentation, I am completely baffeled as to why
breakLine won't work. Here is an example of one of the things I did,
which I believe exactly mimics what breakLine does:

>>> def outer():
... 	def inner():
... 		if outerLocal:
... 			return "I hear you, 'hello world'."
... 		else:
... 			return "Come again?"
... 	outerLocal = "hello world"
... 	return inner()
...
>>> outer()
"I hear you, 'hello world'."

As I said, I believe the line which sets tok should break (quietly),
but not the line which tests tok. My experiment seems to confirm
this...

One thing I can understand is why the line tok = "" in addTok won't
work. This is because when Python sees that line, it should create a
new local variable in the scope of addTok. Once addTok returns, that
variable is lost. That's pretty deep, now that I've thought about it...

>
> Here's a possible solution - but note that there are probably much
> better ways to get the same result...
>
> def breakline(line):
>    """Break a string into a list of words and symbols."""
>    class TokenList(list):
>      def append(self, token):
>        if token:
>          list.append(self, token)
>        return ''
>
>    tokens = TokenList()
>    token = ''
>    splitters = '?()&|:~,'
>    whitespace = ' \t\n\r'
>    specials = splitters + whitespace
>
>    for char in line:
>      if char in specials:
> 	token = tokens.append(token)
>          if char in splitters:
>            tokens.append(char)
>      else:
>        token += char
> 
>    tokens.append(token)
>    return list(tokens)
> 
> (snip)