Embedding Python in a MS Windows app

[Edward K. Ream]

This message summarizes my last two (too long) posts on the wx-users
mailing list.  I will also post this to comp.lang.python for review.
        
The process of embedding the Python interpreter in a MS Windows app is
almost completely undocumented.  I recommend that the following be added
to Python's web site immediately, and to other Python documentation when
feasible.  Without this information, I have spent several weeks on what
could have been a routine task.

Embedding the Python interpreter in a Windows app can be summarized as
follows:

1. Do _not_ build Python into your .exe file directly.  On Windows,
Python must be a DLL to handle importing modules that are themselves
DLL's.  (This is the first key undocumented fact.) Instead, link to
python15.dll; it is typically installed in c:\Windows\System.  

You can link to Python statically or dynamically.  Linking statically
means linking against python15.lib The drawback is that your app won't
run if python15.dll does not exist on your system.

General note: python15.lib is the so-called "import lib" corresponding
to python.dll.  It merely defines symbols for the linker.

Borland note: convert python15.lib to OMF format using Coff2Omf.exe
first.

Linking dynamically greatly simplifies link options; everything happens
at run time.  Your code must load python15.dll using the Windows
LoadLibraryEx routine.  The code must also use access routines and data
in python15.dll (that is, Python's C API's) using pointers obtained by
the Windows GetProcAddress routine.  Macros can make using these
pointers transparent to any C code that calls routines in Python's C
API.

2. If you use SWIG, it is easy to create a Python "extension module"
that will make the app's data and methods available to Python.  SWIG
will handle just about all the grungy details for you.  The result is C
code that you link _into your .exe file_ (!)  You do _not_ have to
create a DLL file, and this also simplifies linking.

3.  SWIG will create an init function (a C function) whose name depends
on the name of the extension module.  For example, if the name of the
module is leo, the init function will be called initleo().  If you use
SWIG shadow classes, as you should, the init function will be called
initleoc().  This initializes a mostly hidden helper class used by the
shadow class.

The reason you can link the C code in step 2 into your .exe file is that
calling the initialization function is equivalent to importing the
module into Python! (This is the second key undocumented fact.)

4. In short, you can use the following code to initialize the Python
interpreter with your extension module.

    #include "python.h"
    ...
    Py_Initialize();  // Initialize Python.
    initmyAppc();  // Initialize (import) the helper class. 
    PyRun_SimpleString("import myApp") ;  // Import the shadow class.
        
5. There are two problems with Python's C API which will become apparent
if you use a compiler other than MSVC, the compiler used to build
python15.dll.

Problem 1: The so-called "Very High Level" functions that take FILE *
arguments will not work in a multi-compiler environment; each compiler's
notion of a struct FILE will be different.  Warnings should be added to
the Python documentation!  From an implementation standpoint these are
very _low_ level functions.

Problem 2: SWIG generates the following code when generating wrappers to
void functions:

    Py_INCREF(Py_None);
    _resultobj = Py_None;
    return _resultobj;
        
Alas, Py_None is a macro that expands to a reference to a complex data
structure called _Py_NoneStruct inside python15.dll.  Again, this code
will fail in a mult-compiler environment.  Replace such code by:

        return Py_Build("");
        
It may be possible to use SWIG's %typemap command to make the change
automatically, though I have not been able to get this to work (I'm a
complete SWIG newbie.)
        
6. Using a Python shell script to put up a Python interpreter window
from inside your Windows app is not a good idea; the resulting window
will be independent of your app's windowing system.  Rather, you (or the
wxPythonWindow class) should create a "native" interpreter window.  It
is easy to connect that window to the Python interpreter.  You can
redirect Python's i/o to _any_ object that supports read and write, so
all you need is a Python object (defined in your extension module) that
contains read and write methods.

That's all.  Any comments, especially from Python or SWIG gurus would be
appreciated.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edream at tds.net
Leo: Literate Editor with Outlines
Leo: http://personalpages.tds.net/~edream/front.html
--------------------------------------------------------------------