[C++-sig] boost::python and threads

[Paul Scruby]

Hiya,

That's fantastic, all I needed to do was to put PyGILState_Ensure(); before 
my virtual function calls into python from another thread and my program no 
longer crashes.  Problem solved, isn't boost::python great!

Many thanks,

Paul


"Renato Araujo" <renatox at gmail.com> wrote in message 
news:95291a80907041315k41b7ad88o32d2111ae8fe1e91 at mail.gmail.com...
Hi Paul

In my bindings I had a problem like this, to solve I created a simple
class like that:

class thread_locker
{
thread_locker()
{
    if (thread_support::enabled())
        m_gstate = PyGILState_Ensure();
}

~thread_locker()
{
    if (thread_support::enabled())
        PyGILState_Release(m_gstate);
}
};

then in my wrapper virtual implementation I did this:

...
void wrapper::virtual_func(..)
{
thread_locker lock;
.. my code ..
}
....

this solve my problems with call of virtual functions in thread enviroment.

BR




On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<wladwig at wdtinc.com> wrote:
> Whoops, I think this problem is a little uglier than I thought, since you 
> overrode the onTick() function in python with a call to print, which needs 
> access to the interpreter in your new thread. See the link Thomas posted 
> for dealing with the GIL (along with worrying about any possible garbage 
> collection issues). Now I remember why I kept my C++ threads isolated from 
> Python stuff....sorry, it's been a while....
>
> Bill
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of William 
> Ladwig [wladwig at wdtinc.com]
> Sent: Saturday, July 04, 2009 1:34 PM
> To: Development of Python/C++ integration
> Subject: Re: [C++-sig] boost::python and threads
>
> It looks to me like you have a garbage collection problem going on. If you 
> create a wrapped c++ object in python, then python is going to own the 
> object and will destroy it when its reference count goes to 0. In your 
> python example script at the bottom, you call the Ticker's run() function, 
> which from the python point of view, returns quickly and the script ends. 
> Python has no idea that you spawned off a new thread from the C++ side, so 
> when the script ends, python destroys the object and now you have a 
> problem. One way that you can check to see if this is what is going on is 
> to add this to the bottom of the test script and see if the crashes go 
> away:
>
> # Warning, you'll need to kill this script manually
> import time
> while True:
> time.sleep(1)
>
> Generally when I want to fire off a new C++ thread, I hold any objects 
> that the thread needs in an auto_ptr (see held type for class wrappers), 
> take ownership of them in C++ (see the FAQ in the documentation) and start 
> the thread from the C++ side. Or, you can also create C++ wrappers which 
> accept shared_ptr arguments, while holding your classes in shared_ptrs, 
> and this should handle the reference counting as well. Unfortunately, this 
> may require some interface changes to what you have already written (or 
> possibly some clever wrapping).
>
> Hope this helps,
> Bill
>
>
> ________________________________________
> From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org 
> [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul 
> Scruby [paul at gingernut.tv]
> Sent: Friday, July 03, 2009 6:15 AM
> To: cplusplus-sig at python.org
> Subject: [C++-sig] boost::python and threads
>
> I am having some problems using boost::python with boost::thread. I'm 
> using
> threads because I want to run some tasks in the background when I'm using
> the Python's interactive shell. However, when I use get_override() to call
> a Python method from another boost::thread it crashes internally. For
> example:
>
> #include <boost/python.hpp>
> #include <boost/thread/thread.hpp>
> #include <boost/thread/xtime.hpp>
>
> using namespace boost::python;
>
> class Ticker
> : public wrapper<Ticker>
> {
> private:
> bool run_;
> volatile bool * running_;
> boost::thread * thread_;
> boost::xtime xt_;
> public:
> Ticker() : running_(&run_) { *running_ = false; }
>
> void operator()()
> {
> while (*running_)
> {
> boost::xtime_get(&xt_, boost::TIME_UTC);
> ++xt_.sec;
> boost::thread::sleep(xt_);
> onTick();
> }
> }
>
> void run()
> {
> if (*running_ == false)
> {
> *running_ = true;
> thread_ = new boost::thread(*this);
> }
> }
>
> void stop()
> {
> if (*running_ == true)
> {
> *running_ = false;
> thread_->join();
> delete thread_;
> }
> }
>
> virtual void onTick() { get_override("onTick")(); }
> void default_onTick() {}
> };
>
> BOOST_PYTHON_MODULE(tick)
> {
> class_<Ticker, boost::noncopyable> ("Ticker")
> .def("run", &Ticker::run)
> .def("stop", &Ticker::stop)
> .def("onTick", &Ticker::default_onTick);
> }
>
> Here is a test script that which will crash when you import it into 
> Python's
> interactive shell.
>
> from tick import Ticker
>
> class MyTicker(Ticker):
> def onTick(self):
> print "Each second"
>
> myticker = MyTicker()
> myticker.run()
>
> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
> Windows XP.
>
> The call-stack in dbx on Solaris:
>
> >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
> PyErr_Restore at 0xfef38fa1
> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
> Current function is boost::python::override::operator()
> 99 detail::method_result x(
>
> (dbx) where
> current thread: t at 2
> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
> 0x80652fc), at 0xfef38fa1
> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
> 0xfef2bf02
> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
> "override.hpp"
> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
> 56 in "thread.hpp"
> [11] thread_proxy(0x810a288), at 0xfea78ce4
> [12] _thr_setup(0xfe670200), at 0xfee159b9
> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
> 0x80652fc, 0x80f1220), at 0xfee15ca0
>
> The call-stack in Visual Studio 2008:
>
> python26.dll!1e013595()
> [Frames below may be incorrect and/or missing, no symbols loaded for
> python26.dll]
> python26.dll!1e09ee7d()
> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
> C++
> 00f3fd64()
> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
> param=0x00245f30) Line 168 C++
> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
> kernel32.dll!7c80b729()
>
>
> Have a missed a trick using the wrapper, or does boost::python not support
> threading?
>
> Many thanks,
>
> Paul
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Renato Araujo Oliveira Filho