Multithreaded Performance/Embedding

[Warren L. DeLano]

Thanks for responding!

Yes: straightforward calls into C from Python are running much slower (OpenGL
rendering in particular) in the imported vs. embedded case.  However, this code
does make some calls to the Python API.

Yes: I am using Python threads.  In the embedded case, the master thread
intializes Python, spans a few python threads at the interpreter level (using
threading), returns to C, unblocks python threads, and falls into an asynchronous
event/update loop.

In the "imported" case, several threads are spawned at the script level, and one
of these threads then calls the C code that unblocks threads and ends up in the
asynchronous event/update loop.

But hey - I tried a control experiment:  FORGET ABOUT THREADS

Just calling the C code directly from Python, I find that the same C code which
does OpenGL rendering through Mesa with GLUT simply runs 3X slower as a python
module than as a standalone C program.

Any ideas?  In each case, there C code excutes some python code as part of the
tight update/event loop, and both obtains and releases global interpreter locks.
Is there substantially more overhead for calling the operations from modules
versus embedded interpreters (perhaps involving namespaces or something?).

I'll continue to reduce the complexity of the problem...and hopefully find a
really simple example
of this behavior..

-Warren

Aahz Maruch wrote:

> In article <39110799.2415927D at sunesis-pharma.com>,
> Warren L. DeLano <warren at sunesis-pharma.com> wrote:
> >
> >I am writing a multithreaded molecular modeling program in C and Python
> >and I am finding a substantial difference in execution speed of the C
> >components (3-4X) depending on whether I embed python (fast) or import
> >my code in as a module (slow).  Is this normal?
> >
> >Since the C-code is running as a separate thread with Python threads
> >unblocked, I don't see why there should be any difference in performance
> >-- but the difference is huge! If the python interpreter is the master
> >thread (vs being embedded), it seems to add a 50-75% overhead onto my
> >C-code execution speed.
>
> Let's make sure we understand the problem first.  Your C code spawns its
> own threads?  And you're saying that if you do a straight Python call
> into your C code, your C code runs much slower?  You aren't using Python
> threads at all?  If the answer to any of these questions is "no", try
> converting them to "yes".
> --
>                       --- Aahz (Copyright 2000 by aahz at netcom.com)
>
> Androgynous poly kinky vanilla queer het    <*>     http://www.rahul.net/aahz/
> Hugs and backrubs -- I break Rule 6
>
> "Laws that restrict speech are an abomination for a simple reason.
> They're like a poison gas; they seem to be a good weapon to vanquish
> your enemy, but the wind has a way of shifting."  -- Ira Glasser

--
mailto:warren at sunesis-pharma.com
Warren L. DeLano, Ph.D.
Informatics Scientist
Sunesis Pharmaceuticals, Inc.
3696 Haven Ave., Suite C
Redwood City, CA 94063
(650)-556-8800 fax: (650)-556-8824