Looking for advice: supporting multiple embedded interpreters

[Josiah Carlson]

> Some background first - we have some software that embeds a Python 
> interpreter into a host application. Scripts are loaded dynamically and 
> used. But we want to support the ability to edit scripts while the app is 
> running. This means we have to "unload" the script and cause a reload. 
> Normal module reloading tricks don't work because if you try to reimport a 
> script that imports another script, and if the nested script is changed, it 
> might not be reloaded itself. Also, we have 2 parts of the software, each 
> using Python for its own set of scripts. We don't want scripts loaded into 
> each parts to "see" scripts loaded into the other part. Ideally, we would 
> like multiple "instances" of a Python interpreter to manage this.
> 
> So, for Python 2.2, I came up with a system that works. When we need a new 
> self-contained Python interpreter, I use Py_NewInterpreter(), swap it in 
> using PyThreadState_Swap, load my built in modules, and swap it back out. 
> When I need to run some code in that interpreter, I swap it back in, load 
> the module I need, call methods in it, and swap it back out. When I'm done 
> with the interpreter, I swap it back in and call Py_EndInterpreter.
> 
> When I want to force a reload of all the script code in a given 
> interpreter, I just delete the interpreter, create a new one, and load the 
> scripts into that one. This has worked flawlessly. And each "part" of my 
> application can use a different interpreter without modules and globals in 
> each one interfering with the other.
> 
> Now, with Python 2.3, this code doesn't seem to work anymore. Someone told 
> me it is likely because of the extensive rewrite of GUSI or whatnot. It is 
> important to note that I'm NOT really doing any threading. I just was 
> self-contained interpreters for the above reasons.
> 
> What I am wondering is if there a reliable method in 2.3 that does what I 
> need?
> 
> It has recently come to my attention that Lutz Paelike is in exactly the 
> same situation I am in, so I don't think this is a fringe concept.

My advice: use the 'reload' builtin to reload your python modules.

 - Josiah