[Python-Dev] Extension modules, Threading, and the GIL
James Henstridge
james@daa.com.au
Sun, 26 Jan 2003 19:15:22 +0800
This is a multi-part message in MIME format.
--------------090900070709010305040001
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
I originally sent this as private mail to David, but he suggested I
repost it here as well.
The Python GIL has been causing trouble getting threading to work
correctly in my Python bindings for GTK 2.x. Both Python and GTK have
an idea about how threading should work, and meshing the two together
isn't quite successful. It would be great if Python made this kind of
thing easier.
James.
--
Email: james@daa.com.au
WWW: http://www.daa.com.au/~james/
--------------090900070709010305040001
Content-Type: message/rfc822;
name="your threading/extensions posts on python-dev.eml"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="your threading/extensions posts on python-dev.eml"
Message-ID: <3E251157.3020809@daa.com.au>
Date: Wed, 15 Jan 2003 15:44:23 +0800
From: James Henstridge <james@daa.com.au>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2.1) Gecko/20021130
X-Accept-Language: en-au, en
MIME-Version: 1.0
To: dave@boost-consulting.com
Subject: your threading/extensions posts on python-dev
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Hi,
I am the author/maintainer of the Python bindings for GTK+ and GNOME, so
have run into a number of these problems as well. If changing Python
made threading more reliable in PyGTK, that would be great.
Neither Python or GTK+ are free-threaded, which makes juggling the locks
quite difficult (the thread support code in current 1.99.x releases of
PyGTK doesn't really work).
I don't know how different pygtk's requirements are to your Qt code are,
so I will give a quick description of the relevant bits of pygtk.
1. GTK uses an event loop, so the program will spend most of its time in
a call to gtk.main(). We obviously want to allow other threads to
run during this call (=> release python GIL).
2. GTK has a signal system to handle notifications. Arbitrary numbers
of callbacks can be attached to signals. PyGTK has a generic marshal
function to handle all signal callbacks, which uses introspection to
handle the different signatures.
Some signals will be emitted in response to method calls (ie. GIL is
held), while others are emitted in the event loop (GIL not held).
Some signals can be emitted in both conditions, so it isn't really
possible for the signal handler marshal code to know if the GIL is
being held or not.
3. Add to this mix GTK+'s global lock which must be held when
executing gtk functions. This lock is held during signal emissions,
but must be manually acquired if a thread other than the one running
the main loop wants to execute gtk functions. The functions used to
deallocate gtk objects count here, which means that the GTK lock must
be held while finalising the Python wrappers.
Having a recursive GIL in Python would help a lot with signal emission
issue (to handle cases where the signal emission occurs while the Python
GIL is not held, and while it is held).
I am sure that it would be possible to clear up a lot of these issues by
dropping the Python lock during every gtk API call, though this would
require a fair bit of work to the code generator and testing (there are
a few thousand gtk APIs).
Sorry for a long winded email, but if you are going to work on Python
threading, it would be good if the changes could help solve at least
some of these problems.
James Henstridge.
--
Email: james@daa.com.au | Linux.conf.au http://linux.conf.au/
WWW: http://www.daa.com.au/~james/ | Jan 22-25 Perth, Western
Australia.
--------------090900070709010305040001--