C-api: Executing several scripts as __main__: globals are gone

[Hartmut Goebel]

Hello,

in PyInstaller we execute several Python scripts one after each other.
The primary use of this is to run some setup prior to the actual
appication. Up to now all scripts shared the same global variables,
which worked well for 15 years, but now showed an error. The new code
(scratched below) now deletes sys.modules['__main__'], so the next
script will get a fresh module and fresh globals. This leads to an
obscure error:

If a setup-script hooks into something, this hook does not see it's own
script's/module's globals. Thus running the scripts below (setup first,
then main), fails with:

Traceback (most recent call last):
  File "main-script", line 2, in <module>
  File "setup-script", line 3, in do_it
NameError: global name 'sys' is not defined

Same effect for any other identifier defined globally in setup-script,
e.g global functions (which is worse then just a missing import).

I tried keeping a reference to the module (to avoid garbage-collection)
both in the C-code and in the setup-script. But this did not solve the
issue, the effect is the same. I also read through the CPython source
but did not spot anything useful. Additionally, this issue only occurs
with Python 2.7 and 3.3, Python 3.4 and up are okay.

Any ideas?

....8<------ C-code scratch!! pseudo-code ---
sys_modules = PyImport_GetModuleDict();
for each entry in archive {
    data = Get_data_out_of_the_archive()
    code = PyMarshal_ReadObjectFromString(data)
    /* execute as '__main__ for compatibility */
    module = PyImport_ExecCodeModule("__main__", code);
    Py_DECREF(module);
    /* remove '__main__' from sys.modules */
    PyObject_DelItem(sys_modules, "__main__");
}
......8<-----------------------

Now if a have these two scripts (these are examples:

......8<---- setup-script ------
import sys
def do_it()
   print(sys.modules)
sys.do_it = do_it
......8<-------------------------

and

......8<---- main-script ------
import sys
sys.do_it()
......8<-----------------------


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel at crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |