[C++-sig] RAII for the GIL in Boost.Python?

[Niall Douglas]

On 16 Dec 2012 at 20:37, John Zwinck wrote:

> There are many questions and example classes online for dealing with 
> Python's Global Interpreter Lock (GIL) using RAII.  I think Boost.Python 
> should provide this functionality, rather than having users who build 
> hybrid C++/Python applications copy-pasting code from various places.  I 
> was one of those users recently, and the code I developed to deal with 
> it is now public:
> 
> https://github.com/jzwinck/pccl/blob/master/InterpreterLockGuard.hpp
> https://github.com/jzwinck/pccl/blob/master/InterpreterLockGuard.cpp
> https://github.com/jzwinck/pccl/blob/master/test/InterpreterLockGuard.test.cpp
> [snip]
> I welcome any feedback regarding the suitability for inclusion in Boost 
> of this particular code and/or the concepts it implements.

Firstly, thanks for taking the time to write and submit this. So 
please don't take what I'm about to say too negatively.

GIL (un)locking isn't just a case of a quick RAII holder and you're 
done as your code has implemented. If that were the case, one of us 
would have submitted such a thing years ago. There are very, very 
good reasons why we have deliberately foresworn a GIL solution, and 
that's because getting it "right" probably means a refactoring of 
much of the BPL design.

I won't go into too much detail here (search this list's archives), 
but in short there isn't just GIL management but also interpreter 
management, and on top of that both of those have to work right as 
exception throws happen plus policies must be instituted for things 
like container iterators (e.g. do you release GIL each iteration, 
every 10 iterations etc) and policies for multiple interpreter 
interactions (e.g. do we timeslice interpreters, or wait for one to 
go to i/o sleep first?). In short, the code is complex and error 
prone. I know, as I wrote the only full BPL GIL management 
implementation I know of and no one has seen fit to attempt to port 
it to Boost yet, and I would assume no one ever will.

Now, I've had some discussion with Dave about the future of BPL and 
how it relates to some related infrastructure I'm currently 
prototyping, but I have to admit I don't think it likely that this 
infrastructure will get greenlit (we'll know before Spring 2013) and 
as a result, someone else will have to acquire the resources to get 
something done for a more permanent solution. Even if greenlit, we're 
talking 2014 easily before any open source release could be possible.

So, if you can't wait before then, feel free to dive in and implement 
a full solution. Start here:

https://github.com/ned14/tnfox/blob/master/Python/FXPython.h

https://github.com/ned14/tnfox/blob/master/Python/FXPython.cxx

And you'll need to modernise these patches to Boost:

https://github.com/ned14/tnfox/blob/master/Python/BoostPatches.zip

You'll see a similar, but much more complex, RAII system in there.

Good luck!

Niall

-- 
Any opinions or advice expressed here do NOT reflect those
of my employer Research In Motion Inc.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMime.p7s
Type: application/x-pkcs7-signature
Size: 6061 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20121216/3dd07797/attachment-0001.bin>