Using a dictionary to pass data to/from embedded python functions

[Alex Martelli]

wardm <wardm66 at gmail.com> wrote:

> I have created a Dict object in a C++ App that calls (embedded) Python
> functions.
  [[snip snip]]
> This python code throws an exception when it attempts to access the 
> "VarDictionary".
> Does anyone know why this fails ?

It fails due to some code of yours that you have not included: in other
words, the minimal application which embeds Python using _only_ the code
you show us does not exhibit the failure.

I wrote the following file za.c:

#include "Python.h"
#include "stdio.h"
#include "stdlib.h"

int main()
{
    printf("start\n");
    putenv("PYTHONPATH=.");
    Py_Initialize();
    printf("inited\n");
    PyObject* m_pVarDictionary = PyDict_New();
    printf("dict is %p\n", m_pVarDictionary);
    PyObject* m_pInterfaceModule =
PyImport_AddModule("InterfaceModule");
    printf("modu is %p\n", m_pInterfaceModule);
    int status = PyModule_AddObject(m_pInterfaceModule, "VarDictionary"
,
                                    m_pVarDictionary);
    printf("stat is %d\n", status);
    PyObject* m_pScriptModule = PyImport_ImportModule("MyScriptModule");
    printf("impo is %p\n", m_pScriptModule);

    PyObject* func = PyObject_GetAttrString(m_pScriptModule,
"functionName");
    printf("func is %p\n", func);
    if (func && PyCallable_Check(func)) {
       PyObject* ret = PyObject_CallObject(func, NULL);
       printf("retu is %p\n", ret);
    }
    printf("done\n");
    return 0;
}

and the following file MyScriptModule.py:

import InterfaceModule

def functionName():
    print "hello"
    print dir(InterfaceModule)
    print "that's all"
    return

and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

and observed exactly the kind of output I predicted [[Note: the exact
addresses printed of course do not matter]]:

start
inited
dict is 0x51c780
modu is 0x520230
stat is 0
impo is 0x5202d0
func is 0x513770
hello
['VarDictionary', '__doc__', '__name__']
that's all
retu is 0xe57c0
done
brain:~/pyex alex$ 

As you see, in particular, VarDictionary is right up there in
InterfaceModule's dir.


There's a well-known essay by Eric Raymond, "How to ask questions the
smart way", at <http://catb.org/~esr/faqs/smart-questions.html> -- it
seems to me that you, quite commendably, follow most of Eric's advice,
but it's still worth reading -- the key point by which you could help us
to help you is what Eric mentions as "If you have a large, complicated
test case that is breaking a program, try to trim it and make it as
small as possible".

In this case, you should try to trim your code down to the smallest
program using this approach, which you believe should work in a certain
way but actually doesn't.  The exact code you posted plus the minimal
additions to make it a compilable program does work the way you appear
to desire, as I show above; therefore, there must be something else in
your code that is breaking things.  Do tiny steps of addition and
restructuring to move this minimal skeleton towards the direction of
your bigger program (that does not behave this way) until you've found
exactly the "largest" version that still succeeds, and the minisculely
larger "smallest" version that fails -- the tiny difference between the
two must then be the root of the problem, and if the reason is not clear
at that point then posting here again is exactly the right thing to do.

Perhaps you're falling afoul of the fact that (as documented e.g. at
<http://docs.python.org/api/moduleObjects.html>) PyModule_AddObject is a
"convenience function" that steals a reference to the value -- so you
end up with just one reference to the dictionary object, and if for some
reason you decref it, the object gets garbage collected.  But, that's
just a wild guess on my part, since you have not shown us any code
performing (for example) any decrefs.


Alex