C replacement for Queue module

Jonathan Ellis jbellis at gmail.com
Fri Oct 21 18:44:03 EDT 2005


I'm working on an application that makes heavy use of Queue objects in
a multithreaded environment.

By "heavy" I mean "millions of calls to put and get, constituting ~20%
of the app's run time."  The profiler thinks that a significant amount
of time is spent in this code -- not just a consumer waiting for a
producer, but actual _empty, notify calls, etc.

Would it be worth the time to write a CQueue module with pthread_cond
instead of Python Condition objects, etc?  I don't really have a gut
feeling for how much bang-for-the-loc C optimization might give: twice
as fast?  5x as fast?

thanks,

-Jonathan




More information about the Python-list mailing list