Thread terminate

[Chris Kaynor]

On Thu, Aug 28, 2014 at 1:35 PM, Ervin Hegedüs <airween at gmail.com> wrote:

> On Thu, Aug 28, 2014 at 12:34:18PM -0700, Chris Kaynor wrote:

 depending on what you are doing with the first two arguments to self._exit,
>> the following might also work:
>>
>> def run(self):
>>     try:
>>         connect_to_db()
>>         do_another_thing()
>>     except:
>>         self._exit(*sys.exc_info())
>>         return True
>
>
> The first argument is a status, this is passed to the item, which passed
> to thread - so the thread sets that status, and the main loop knows,
> which item needs to pass to a thread. Eg. if the DB connection
> has failed, it needs to run again, at later. But if that item is
> parsed and finished (eg. table exists in db), then the item
> is deletable from the queue.
>

Mostly, I was wondering if you really need to differentiate types of
failures at that point. Would it be enough to merely know "it worked" vs
"it failed"? Even if you do need to behave differently, can you make the
decision based on the type of exception, rather than which stage you were
on? If so for either, you can remove the multiple try...except around
different statements, and put one big one around the entire run function.

This would likely simplify your handling code.


> One solution I did not mention before, but is plausible, is to
>> monkey-patch/override the threading module to change the behavior
>> of _bootstrap_inner or _bootstrap to behave differently. As these are on
>> the Thread class, you could override them in your subclass or a base
>> subclass (depending on the number of Thread subclasses you have) that
>> behaves differently. Perhaps by making it call sys.excepthook. At one
>> point, we discussed doing something similar at work as we normally print
>> exceptions to custom streams that provide color formatting to make the
>> exceptions easier to find and read (exception message printed in red,
>> traceback in a dark red), as well as log.
>
>
> oh, that would be a big gun - and I don't feel that knowledge to
> made that with security and stability.
>
> I think this solution will enough at first time :), I didn't used
> anytime the threads (in serious project).
>

Yah, we never did this either, mostly due to the fact that then we'd have
to update our code if Python's code was updated. This can be quite
problematic, especially as the functions that would need to be overridden
are private, and thus not put to the same backwards compatibility standards
as most of Python's API.

What we did do, however, was wrap threading (in some cases) where we used a
thread pool where the actual run function was wrapped in our own error
handling, and called out to the desired function. Basically, most of our
threads went though common code that looked something like (simplified
here):

def run(self):
    while True:
        func, args, kwargs = self.queue.pop() # Actually, returned an
instance of a class which provided some join() and cancel() methods, among
others.
        try:
            func(*args, **kwargs)
        except BaseExeception:
             # complex error handling here. Depending on conditions, this
might print the error, or might merely cache it in case the work item was
joined

Basically, the whole code block was a custom implementation of Python's
concurrent module, which we could not use as we had to support 2.6. It was
also written before 3.2 came out (if I remember the timing right).

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20140828/41a03296/attachment.html>