From tlyons at ivenue.com Tue May 14 17:45:58 2013 From: tlyons at ivenue.com (Todd Lyons) Date: Tue, 14 May 2013 15:45:58 -0000 Subject: [capi-sig] Embedded python, adding a path to sys.path before import, strange observations Message-ID: I was well on my way to sending a help request about "why can I not add a path to sys.path" in an embedded python instance (that I'm adding to a c app). I have test code from *three* different methods that should add a path to the sys.path object, but none of them _seemed_ to work. I highlight that word because the way I was testing is what seems to be flawed. All three methods turn out to work, but the way I was testing appears to be flawed. I was testing by comparing the output of Py_GetPath() before I adjusted the path, and then after I adjusted the path. It always shows the same. On a whim, I added a call to load a quickie python script in that extra subdirectory after I adjusted the path and I was FLOORED when loaded. I've been working on this for days thinking that it wasn't working because the output of Py_GetPath() did not indicate the new path I was adding. Question A: Is it normal that Py_GetPath() doesn't show the adjusted path? Method 1: PySys_GetObject("path"), PyList_Insert(), PySys_SetObject() with the new object Method 2: PyRun_SimpleString("import sys" "sys.path.append('blah')") Method 3: Py_GetPath(),append the syspath and new path together, PySys_SetPath() with the new string Question B: All three seem to work equally well, is there a reason to choose one over the other? All positive guidance and suggestions are welcome. Testing C code is quoted below, as well as the sample output from running it and an strace trimmed output to show that it does appear to be loading the module. Thanks for any comments, observations, or explanations. ...Todd py_testc: #include int main (int argc, char *argv[]) { char *path, *eximpath = "/etc/exim"; PyObject *syspath, *pName, *pModule; Py_SetProgramName("application"); Py_Initialize(); if (!Py_IsInitialized()) { printf("Error initializing Python interpreter\n"); return 1; } path = Py_GetPath(); printf("Python search path is:\n%s\n", path); syspath = PySys_GetObject("path"); if (syspath == 0) printf("Python failed to import sys\n"); printf("Adding %s\n", eximpath); pName = PyString_FromString(eximpath); if (PyList_Insert(syspath, 0, pName)) printf("Error inserting extra path into sys.path list\n"); if (PySys_SetObject("path", syspath)) printf("Error setting sys.path object\n"); pName = PyString_FromString("mod_python"); pModule = PyImport_Import(pName); Py_DECREF(pName); if (pModule == NULL) printf("Unable to import module\n"); else printf("Loaded module\n"); /* Still shows wrong path! But module still loads above. */ path = Py_GetPath(); printf("Modified Python search path is:\n%s\n", path); Py_DECREF(syspath); Py_Finalize(); return 0; } // vim:sw=2 ts=2 expandtab [todd at tlyons ~/python]$ ./py_test Python search path is: /usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload Adding /etc/exim Loaded module Modified Python search path is: /usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload [todd at tlyons ~/python]$ strace ./py_test 2>&1 | grep mod_python stat64("/etc/exim/mod_python", 0xbf9dfa58) = -1 ENOENT (No such file or directory) open("/etc/exim/mod_python.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) open("/etc/exim/mod_pythonmodule.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) open("/etc/exim/mod_python.py", O_RDONLY|O_LARGEFILE) = 3 open("/etc/exim/mod_python.pyc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) unlink("/etc/exim/mod_python.pyc") = -1 ENOENT (No such file or directory) open("/etc/exim/mod_python.pyc", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC|O_LARGEFILE, 0100644) = -1 EACCES (Permission denied) -- The total budget at all receivers for solving senders' problems is $0. If you want them to accept your mail and manage it the way you want, send it the way the spec says to. --John Levine From tlyons at ivenue.com Tue May 21 16:27:09 2013 From: tlyons at ivenue.com (Todd Lyons) Date: Tue, 21 May 2013 07:27:09 -0700 Subject: [capi-sig] Constructing variable arg tuples Message-ID: Hi all, I'm embedding python into a c app (exim, the MTA). I've gotten code in to initialize python, read in the embedded script, call various functions within that script, and appropriately return data from those functions back to the exim call. One thing that does not make sense yet though, is how to assemble the data to be passed to the python functions. Imagine the following simple function: def simple_math(a,b): return a+b In this case, a and b are integers or long integers. How can I format that incoming data without knowing in advance that there are two arguments to this function and that are both are integers? If the reason for this question is not clear, imagine that the sysadmin goes and edits the python script so that it now is: def simple_math(name,a,b): // do something to log "name" to a logfile return a+b Now argument index 0 is a string, and indexes 1 and 2 are the integers. How does a calling c application deal with the fact that the variables to function can be, well, of variable types, and not known beforehand? It seems like there should be a python api function available which says "for $FUNC, are $ARGS valid arguments" or "for $FUNC, convert $ARGS to what $FUNC expects". Am I missing something simple here? ...Todd -- The total budget at all receivers for solving senders' problems is $0. If you want them to accept your mail and manage it the way you want, send it the way the spec says to. --John Levine