[C++-sig] boost::python and threads

[William Ladwig]

It looks to me like you have a garbage collection problem going on.  If you create a wrapped c++ object in python, then python is going to own the object and will destroy it when its reference count goes to 0.  In your python example script at the bottom, you call the Ticker's run() function, which from the python point of view, returns quickly and the script ends.  Python has no idea that you spawned off a new thread from the C++ side, so when the script ends, python destroys the object and now you have a problem.  One way that you can check to see if this is what is going on is to add this to the bottom of the test script and see if the crashes go away:

# Warning, you'll need to kill this script manually
import time
while True:
    time.sleep(1)

Generally when I want to fire off a new C++ thread, I hold any objects that the thread needs in an auto_ptr (see held type for class wrappers), take ownership of them in C++ (see the FAQ in the documentation) and start the thread from the C++ side.  Or, you can also create C++ wrappers which accept shared_ptr arguments, while holding your classes in shared_ptrs, and this should handle the reference counting as well.  Unfortunately, this may require some interface changes to what you have already written (or possibly some clever wrapping).  

Hope this helps,
Bill


________________________________________
From: cplusplus-sig-bounces+wladwig=wdtinc.com at python.org [cplusplus-sig-bounces+wladwig=wdtinc.com at python.org] On Behalf Of Paul Scruby [paul at gingernut.tv]
Sent: Friday, July 03, 2009 6:15 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] boost::python and threads

I am having some problems using boost::python with boost::thread.  I'm using
threads because I want to run some tasks in the background when I'm using
the Python's interactive shell.  However, when I use get_override() to call
a Python method from  another boost::thread it crashes internally.  For
example:

    #include <boost/python.hpp>
    #include <boost/thread/thread.hpp>
    #include <boost/thread/xtime.hpp>

    using namespace boost::python;

    class Ticker
        :    public wrapper<Ticker>
    {
    private:
        bool run_;
        volatile bool * running_;
        boost::thread * thread_;
        boost::xtime xt_;
    public:
        Ticker() : running_(&run_) { *running_ = false; }

        void operator()()
        {
            while (*running_)
            {
                boost::xtime_get(&xt_, boost::TIME_UTC);
                ++xt_.sec;
                boost::thread::sleep(xt_);
                onTick();
            }
        }

        void run()
        {
            if (*running_ == false)
            {
                *running_ = true;
                thread_ = new boost::thread(*this);
            }
        }

        void stop()
        {
            if (*running_ == true)
            {
                *running_ = false;
                thread_->join();
                delete thread_;
            }
        }

        virtual void onTick() { get_override("onTick")(); }
        void default_onTick() {}
    };

    BOOST_PYTHON_MODULE(tick)
    {
        class_<Ticker, boost::noncopyable> ("Ticker")
            .def("run", &Ticker::run)
            .def("stop", &Ticker::stop)
            .def("onTick", &Ticker::default_onTick);
    }

Here is a test script that which will crash when you import it into Python's
interactive shell.

    from tick import Ticker

    class MyTicker(Ticker):
        def onTick(self):
        print "Each second"

    myticker = MyTicker()
    myticker.run()

I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler on
Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008 on
Windows XP.

The call-stack in dbx on Solaris:

    >>> t at 2 (l at 2) signal SEGV (no mapping at the fault address) in
PyErr_Restore at 0xfef38fa1
    0xfef38fa1: PyErr_Restore+0x0031:       movl     0x00000028(%edi),%ecx
    Current function is boost::python::override::operator()
       99           detail::method_result x(

    (dbx) where
    current thread: t at 2
      [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
0x80652fc), at 0xfef38fa1
      [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
      [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
      [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
      [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
0xfef2bf02
      [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
    =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
"override.hpp"
      [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
      [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
      [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
56 in "thread.hpp"
      [11] thread_proxy(0x810a288), at 0xfea78ce4
      [12] _thr_setup(0xfe670200), at 0xfee159b9
      [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
0x80652fc, 0x80f1220), at 0xfee15ca0

The call-stack in Visual Studio 2008:

    python26.dll!1e013595()
    [Frames below may be incorrect and/or missing, no symbols loaded for
python26.dll]
    python26.dll!1e09ee7d()
  > tick.pyd!boost::python::override::operator()()  Line 103 + 0x16 bytes
C++
    00f3fd64()
    tick.pyd!Ticker::operator()()  Line 27 + 0xe bytes C++
    tick.pyd!boost::detail::thread_data<Ticker>::run()  Line 57 C++
    tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
param=0x00245f30)  Line 168 C++
    msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes C
    msvcr90d.dll!_threadstartex(void * ptd=0x00d46938)  Line 331 C
    kernel32.dll!7c80b729()


Have a missed a trick using the wrapper, or does boost::python not support
threading?

Many thanks,

Paul



_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig