[C++-sig] Expanding embedded code

[Stefan Seefeld]

Robin Gilks wrote:
> A difficult question to put a title to so I'll paint a brief scenario...
>
> The application is written in c++
> It receives a data message and instantiates a message decoder class. 
> Based on the contends that are extracted into the class variables, a 
> response is built by instantiating an encoder class and setting its 
> class variables with setter methods. At this point I want to pass 
> control over to a Python script so that it can modify the encoder 
> class variables (via the accessors) if required before the response is 
> actually sent. These accessors would form an extension to the 
> interpreter. Note that doing more work in the Python script is not an 
> option - too much data in too short a time!!

You say "The application is written in c++" which I read as the above 
already exists, and is not subject to (re-)design.
Otherwise I would suggest to do most of this inside Python itself, given 
the highly dynamic nature of the problem.

>
> I've got as far as embedding the interpreter and calling the correct 
> method in a script but I've hit a problem in extending the interpreter 
> to access the decoder and encoder classes in my main app.

Some nit-picking: You are not extending the interpreter. You are adding 
to the runtime (environment).

>
> Note that I want to access the specific instances of the classes to 
> manipulate the class variables that were set before entering the 
> Python domain. Now the questions...
>
> * My understanding is that extending Python *always* uses a 
> dynamically loaded library (i.e. a run time linkage) as part of the 
> import mechanism - is there something in the C/API that will override 
> this? If I load an extra library, even one built from the same source 
> as my classes in the main app, I won't be accessing the same instance 
> of them.

Indeed, there is no reason to having to dlopen() (or however your 
platform does this). You can entirely operate within the same 
compilation unit. For an example, see the 
boost/libs/python/test/exec.cpp test that defines an 'embedded_hello' 
module, then injects it directly into the Python runtime module map via 
'PyImport_AppendInitTab()'.

See https://svn.boost.org/trac/boost/browser/trunk/libs/python/test/exec.cpp

>
> * Since the class has already be instantiated in the main app, how do 
> I pass the instance pointer to Python so that it can pass it back when 
> it makes the extension call to my class methods (assuming I can get 
> past the previous point!!). I would assume that it involves a 
> PyCObject but none of the code examples I have found give much clue as 
> to how the extension is invoked from the Python side with the instance 
> pointer apart from when using a new_class constructor call.

I hope the following is intuitive enough. If not, please ask:

-8<---------------------------------

using namespace boost;
...
python::class_<Type> type("Type", ...); // create wrapper
...
python::object instance = type(); // instantiate it
python::object main = import("__main__");
python::object global = main.attr("__dict__");
global["instance"] = ptr(instance); // ptr() makes sure the object is 
passed by-reference.

python::object result = python::exec_file(script, global, global);

-8<---------------------------------

At this point, the script will have had its opportunity to interact with 
your 'instance' object directly.
But, it may also have stored other stuff into the 'global' dictionary, 
which you can extract now that the script has returned:

-8<---------------------------------

python::object something = global["something"];
extract<SomethingElse &> e(something);
if (e.check()) // is this a SomethingElse instance ?
{
  SomethingElse &something_else = e(); // extract it...
  ... // ...and use it.
}

-8<---------------------------------

You get the idea...

Regards,
       Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...