A thread import problem

[Bruce Sherwood]

Dieter Maurer commented the following on my question about a thread
import problem:

----------------------
In a recent discussion in this list someone mentioned that
on module import, you should not start a thread. The reason: apparently,
Python uses some kind of locking during import which can interfere
with "import"s in the started thread.

You can (in principle) easily avoid starting the thread on module import.
Instead of starting the thread as a side effect of the import,
put the start in a function, import the module and then call
the thread starting function.
-----------------------

Thanks for the helpful information. I tried the experiment of altering
the source code to be exec-ed to import a different module and then
call a function from that module to start a thread, but that didn't
help, perhaps because in this chain of events, which necessarily
involve exec's in a thread, it's the equivalent of the problem you
identify.

I actually do have a solution but it's inelegant. In the module
imported by the user's program, I find all import statements in that
user's program and execute them (then comment them out), thereby
adding their symbols to the module's globals, and I pass the module's
globals to the exec statement.

I'll state the problem in a more general way in the hopes that someone
will see a solution I've missed. The VPython API permits the following
short program, which displays a 3D cube moving to the right, and you
can rotate and zoom the camera with the mouse:

from visual import box, rate
b = box()
while True:
    rate(100) # no more than 100 iterations per second
    b.pos.x += .01

This works because a GUI environment is invoked by the visual module
in a secondary thread (written mainly in C++, connected to Python by
Boost). The OpenGL rendering of the box in its current position is
driven by a 30-millisecond timer. This works fine with any environment
other than Mac Cocoa.

However, the Mac Cocoa GUI environment and interact loop are required
to be the primary thread, so the challenge is to have the visual
module set up the Cocoa environment, with the user's program running
in a secondary thread. Any ideas?

Bruce Sherwood

On Wed, Jul 18, 2012 at 5:03 PM, Bruce Sherwood
<bruce.sherwood at gmail.com> wrote:
> I'm trying to do something rather tricky, in which a program imports a
> module that starts a thread that exec's a (possibly altered) copy of
> the source in the original program, and the module doesn't return.
> This has to do with an attempt to run VPython in the Mac Cocoa
> context, in which Cocoa is required to be the primary thread, making
> it necessary to turn the environment inside out, as currently VPython
> invokes the Carbon context as a secondary thread.
>
> I've created a simple test case, displayed below, that illustrates
> something I don't understand. The module reads the source of the
> program that imported it, comments out the import statement in that
> source, and performs an exec of the modified source. The module then
> enters an infinite loop, so that there is no return to the original
> program; only the exec-ed program runs, and it runs in a secondary
> thread.
>
> The puzzle is that if there is any later import statement in the exec
> source, the exec program halts on that import statement, with no error
> message. I saw a discussion that suggested a need for the statement
> "global math" to make the math import work, but that doesn't fix the
> problem. I've tried with no success various versions of the exec
> statement, with respect to its global and local environment.
>
> Can anyone explain why the math import statement causes a problem?
> Thanks for any advice you can give.
>
> Bruce Sherwood
>
> ---------------------------
> The main program:
>
> from import_test import *
> print('exec this file')
> global math
> from math import sin
> print(sin(3.14159/6))
>
> -----------------------------
> Contents of import_test:
>
> from threading import Thread
> from time import sleep
> import sys
>
> prog = open(sys.argv[0]).read()
> prog = '#'+prog # comment out the import statement
> print(prog)
>
> class worker(Thread):
>     def run(self):
>         print('start thread')
>         exec(prog)
>
> w = worker()
> w.start()
>
> while True:
>     sleep(1)