[capi-sig] Embedded Python in C application

[Swapnil Talekar]

On Fri, Sep 26, 2008 at 11:18 PM, Gustavo Carneiro <gjcarneiro at gmail.com>wrote:

> 2008/9/26 Eljay Love-Jensen <eljay at adobe.com>
>
> > Hi everyone,
> >
> > First, my apologies if I'm in the wrong forum for my "embedding Python in
> a
> > C application" questions.  Please redirect me if I've wandered into the
> > wrong place.
> >
> > I have two needs for using Python in my application that I hope has an
> easy
> > answer without rewriting Python's internals.
> >
> > I need to use Python* in a multi-threaded application, where separate
> > threads may be working on very long lasting Python scripts, and other
> > threads may be involved in short Python scripts.  None of the Python
> > scripts
> > running concurrently have any shared state with any of the other Python
> > scripts running concurrently.  Number of threads is in the 100-1000
> range.
> >
> > I need to manage Python's use of the heap by providing a memory pool for
> > Python to use, rather than allowing Python to use malloc/free.  This is
> to
> > prevent memory fragmentation, and to allow easy disposal of a memory pool
> > used for a closed Python interpreter instance.
> >
> > A quick view of Py_Initialize() indicates that Python does not return
> some
> > sort of "Py_State" pointer which represents the entire state of a Python
> > interpreter.  (Nor some sort of Py_Alloc().)  Nor accepts a custom
> > malloc/free function pointers.  Hmmm.
>
>
> >Python already has its own highly optimized memory allocator, it does not
> >use malloc/free directly.  That's why the configure option
> >--without-pymalloc exists.
>
> >So I think your basic premise is wrong.  But in any case maybe you are
> >looking for PyInterpreterState_New().  But beware that going down that
> path
> >is going to be painful: multiple interpreter states and threading can lead
> >to many hours of debugging.  I would think thrice before deciding I really
> >need it.
>

>But If Eljay is trying to actually create multiple threads to run the
>scripts simultaneously, then I guess he has much more to worry
>about than just PyInterpreterState. The API does not take
>care of all the Python global variables for sure.



--Swapnil Talekar