[C++-sig] binding a userspace thread library

[Mathieu Lacage]

On Wed, 2006-01-04 at 11:20 -0500, Stefan Seefeld wrote:
> > [mlacage at chronos yans-current]$ ./bin/python/test-thread.py
> > before sleep
> > Exception exceptions.NameError: "global name 'time_s' is not defined" in 'garbage collection' ignored
> > Fatal Python error: unexpected exception during garbage collection
> > Aborted
> > [mlacage at chronos yans-current]$
> > 
> > This error message does not look quite right: it looks like a collision
> > of two error messages. Am I wrong ?
> 
> To me it looks plausible: The worker thread writes 'before sleep' and
> then stops. The error you are seeing then seems to be coming from
> the cleanup at the end of your script, i.e. it appears to be an entirely
> different issue.

I fail to see how the call to Simulator.destroy could generate such a
warning but you are likely to be right.

[snip]

> > Traceback (most recent call last):
> >   File "./bin/python/test-simulator.py", line 34, in ?
> >     simulator.run ()
> >   File "./bin/python/test-simulator.py", line 7, in notify
> >     print "my event class now=%f" % now_s ();
> > NameError: global name 'now_s' is not defined
> > 
> > I suspect that the c++ exception I ignore is interpreted correctly by
> > the boost bindings which re-generate the correct python error. Or maybe
> > not.
> 
> I'm not sure. Are you saying that the 'now_s()' call should actually succeed ?

no. I am merely concerned by the fact that I do not get the same warning
when using a Thread and when not using a Thread. 

I suspect that, in the case shown here, we have:
  - python interpreter calls Simulator.run which calls c++
Simulator::Run
  - c++ Simulator::run calls c++ EventWrap::notify which calls python
MyEvent.notify
  - MyEvent.notify generates python error because now_s is not defined
and returns.
  - c++ EventWrap sees python error, generates c++ exception.
  - c++ exception propagates to the outer of Simulator::run, that is the
boost binding for Simulator::run.
  - c++ boost binding eats the c++ exception, generates a python error.

In the other case I was looking at, I have:
  - python interpreter calls Simulator.run which calls c++
Simulator::Run
  - Simulator::Run calls some Event::notify which switches to another c+
+ Thread's stack (c++ Thread::run)
  - c++ ThreadWrap::run calls python Thread.run
  - python Thread.run generates python error because now_s is not
defined and returns.
  - c++ boost binding code generates c++ exception from python error.
  - c++ ThreadWrap::run sees exception, eats c++ exception, switches
back to main thread
  - c++ main thread (that is, Simulator::run) keeps on executing until
the end of the simulation.
  - at the end of the simulation, c++ Simulator::run returns to python
interpreter which keeps on executing
  - python interpreter runs Simulator.destroy (), triggers, another
errors, and displays the new error's message together with the old error
message.

So, I decided to work around this by forwarding the boost c++ exception
from my c++ thread to the c++ main thread such that the python
interpreter can generate the right error message as soon as possible. I
ended up doing this:

class ThreadWrap {
        void rethrow_exception (error_already_set &set) {
                throw set;
        }
        virtual void run (void) {
                try {
                        call_method<void>(m_self, "run");
                } catch (error_already_set &set) {
                        Simulator::insert_later (make_event (&ThreadWrap::rethrow_exception, this, set));
                }
        }
};

Which as the nice side-effect of now generating the correct error
message with a buggy script:
[mlacage at chronos yans-current]$ bin/python/test-thread.py
before sleep
Traceback (most recent call last):
  File "bin/python/test-thread.py", line 14, in ?
    simulator.run ()
  File "bin/python/test-thread.py", line 10, in run
    print "after sleep %f" % time_s ();
NameError: global name 'time_s' is not defined
[mlacage at chronos yans-current]$

Which is really uber-cool.

> May be you are right, and you need to clear the error flag first before
> terminating the worker thread. Try calling 'PyErr_Clear()' in your exception
> handler in the worker thread. (see http://docs.python.org/api/exceptions.html)

Nice. But I don't think I want to do this.

thanks for your comments,
Mathieu
--