Unexpected (by me) exec behavior

[Bengt Richter]

On 8 Jul 2003 08:18:06 -0700, mwright at pro-ns.net (Mark Wright) wrote:

>I have a script that I use to control our build process.  It is a
>general purpose script that exec's other scripts that contain project
>specific python code.  In one of those other, project-specific,
>scripts I exec a third script.  That third script is failing because
>it can't seem to 'import' successfully.  It seems that if one 'exec's
>a string that in turn 'exec's another string, the 'import's don't work
>in the second string.  I'm assuming that I'm misunderstanding
>something about Python namespaces, but here's an example that
>illustrates the problem:
>
I feel sure there is a better way to do what you want to do, though
I'm not clear on what that is ;-)

You're right that it's a name space thing. Choose [1], [2], [3], or [4] or [5] ;-)
>----------------------------
># filename = t.py
>s1 = """
>s2 = \"\"\"
>import socket
# moving the import above to [1] will make it work
>def xyz():
#def xyz(socket=socket): #[2] chg above to this will also bind the socket to a socket local to xyz
        #import socket #[1]
>	print socket.gethostbyname('somehost')
#                                  ^^^^^^^^^^-- suggest 'localhost' for quick result instead of error
>if __name__ == '__main__':
>	xyz()
#       xyz(socket) #[3] another way to grab the imported socket and pass it to xyz where
#                   # you could call it something different, e.g.,
                    # def xyz(skt): print skt.gethostbyname('localhost')
>\"\"\"
>
>def abc():
>	exec s2
#       exec s2 in globals() #[4] this would bind socket from import globally (w.r.t. t.py) for xyz to see
>if __name__ == '__main__':
>	abc()
>"""
>
>exec s1
>-----------------------------
>
>Traceback (most recent call last):
>  File "t.py", line 24, in ?
>    exec s1
>  File "<string>", line 13, in ?
>  File "<string>", line 11, in abc
>  File "<string>", line 6, in ?
>  File "<string>", line 4, in xyz
>NameError: global name 'socket' is not defined
>
>Can anyone explain this to me?
>
I suspect that to generate code referring to a variable in a nested scope outside
the local but not global -- like socket in your example, which is outside xyz but
not global since it winds up bound local to abc -- the nesting needs to be part of
the same compilation. In your example, s2 is not compiled until abc is executed,
so the compilation of xyz is separate from abc, even though the resulting bindings
are injected into abc's local space. However, when the compilation happens, the
nesting is not visible, so I think you get code that either refers to locals in a
defined function like xyz, or global, but not free. But the def code can assemble
the function when it executes and binds the name to the constructed function. That
includes a designated global directory, which can be abc's local directory if you
specify exec s2 in locals() there. Updating locals will probably not work, so
global declarations in xyz and assigning them would not be good, but looking for
socket in an apparent global that is the abc locals() dict should work. The code
example get another problem though, because __name__ is no longer t.py's __main__,
so xyz won't be called unless you change the if test (the name apparently becomes
__builtin__, so if you wanted to have a script you could execute like that as
well as from the command line directly, you might make the test
[5] ;-)

    if __name__ in ['__main__', '__builtin__']:
        xyz()

and execute it something like

    xdir = {}
    exec s2 in xdir

and potentially access results in xdir afterwards, to rebind them or print
status messages or whatever wasn't already a side effect of the exec.

The above is the way I'm guessing nested variable references work, but I haven't
dug into the compiler etc to verify it, so I'd wait to hear from someone who can
say for sure. I wonder if there are changes coming in name searching and rebinding etc.

Regards,
Bengt Richter