Directly calling threaded class instance methods and attributes

[David Bolen]

lights at cix.co.uk (Matthew Bell) writes:

>(...)
> My question is simply, can anyone see any issues with calling methods
> and/or attributes of a threaded class instance like this?  It looks ok
> to me but, as the docs never seem to mention using threads like this,
> I'm wondering if I've missed something important.  If it helps, I 
> know the basic considerations of threading, such as locking, exception
> handling and so on; I only really need advice on whether there could
> be issues with directly calling class instance attributes of a 
> running thread.

Others have focused more on the locking issues if the methods you use
access data that the separate thread is also accessing, so I'll try to
hit the general question of just sharing the instance itself.

Clearly directly accessing a non-callable attribute has the potential
requirement for locking and/or race conditions.  But for callables,
and if I understand what you might be getting at, the answer is
definitely yes.  There's absolutely no problem calling methods on a
thread object from separate threads, and even have methods used from
multiple threads simultaneously.  The execution flow itself is fine,
but as you note, you have to handle shared data access issues
yourself, to the extent that it applies.

I do think this could simplify your thread communication in some cases
because you can export a more typical "object" interface from your
thread object (at least in the forward direction) rather than having
the user of the thread have to handle queue management.

For example, it's very common for me to have thread objects structured
like (to extend your example):

    class SimpleThread(threading.Thread):
        def __init__(self):
            threading.Thread.__init__(self)
            # I often handle the communication queue to the background
            # thread internally, so callers need not be aware of it.
            self.queue = Queue.Queue()
            # Often my thread objects are self starting (so a user is just
            # instantiating the object and not knowing a thread is involved)
            self.start()

        #
        # Typically several methods used by the background thread during
        # processing.
        #
        def _spam(self):
            pass
        def _eggs(self):
            pass
        def _ham(self):
            pass
        
        #
        # The main thread itself - processes queue requests
        #
        def run(self):
            while 1:
                operation = self.queue.get()
                if operation is None:
                    return

                # Perform operation


        #
        # "Public" (not background thread) operations
        #
        def shutdown(self):
            self.queue.put(None)

        def someoperation(self, args):
            self.queue.put(("dosomething", args))


So to the outside party, they are just creating an instance of my
object, and using methods on it.  The methods happen to then use a queue
internally to get to the processing portion of the object which is in
a background thread, but you don't need to expose that to the caller.

Where this falls down a little is in the result of the processing.
Generally you need to provide for a query mechanism on your object
(which itself might be using an internal queue, or you could just
permit the caller to access an attribute which is the queue), or a
callback system, in which case the caller should clearly be made aware
that the callback will be executing in a separate thread.

Or, if you're using some async, event-driven approach even the thread
is probably completely hideable.  For example, with Twisted, your public
thread instance methods can just appear as deferrable methods, using
standard deferreds as return values.  Then when the result is ready in the
background thread, you have twisted fire the deferred in the main reactor
loop.

-- David