Modification to asyncore.py to support threaded event loop

Anand Pillai abpillai at gmail.com
Thu Nov 4 03:53:43 EST 2004


This is for folks who are familiar with asynchronous event handling in
Python using the asyncore module.

If you have ever used the asyncore module, you will realize that it's
event loop does not have a programmable exit condition. It keeps
looping till the channels in its socket map (a dictionary) are closed
and don't have any pending reads/writes.

If you are using Python threads in your application, by using either
the threading or thread modules, adding asyncore capability on top of
it is a bit tricky. The moment you call asyncore.loop after
initializing your asyncore based client/server, it takes over the main
thread. Hence, if you want to mix threaded classes (derived from
threading.Thread) and asyncore in one program, the only recourse is to
create and initialize all your thread objects first and call
asyncore.loop as the last function in your main thread.

However, with a small hack it is possible to get asyncore event loop
running in its own thread, as shown below.

The following is a mixin class, which derives from threading.Thread
and also overrides asyncore.loop in its 'run()' method.

-==========================================================================-
import threading

class AsyncoreThread(threading.Thread):
    """ Asyncore thread class """

    def __init__(self, timeout=30.0, use_poll=0,map=None):
        self.flag=True
        self.timeout=30.0
        self.use_poll=use_poll
        self.map=map
        threading.Thread.__init__(self, None, None, 'asyncore thread')
        
    def run(self):

        self.loop()

    def loop(self):

        # print 'In AsyncoreThread:loop...'
        if self.map is None:
            self.map = asyncore.socket_map

        if self.use_poll:
            if hasattr(select, 'poll'):
                poll_fun = asyncore.poll3
            else:
                poll_fun = asyncore.poll2
        else:
            # print 'Using asyncore.poll...'
            poll_fun=asyncore.poll

        while self.map and self.flag:
            poll_fun(self.timeout,self.map)

    def end(self):
        print 'Ending asyncore loop...'
        self.flag=False
        self.map=None

-============================================================================-
Now you can initialize your asyncore based clients/servers and instead
of calling 'asyncore.loop', do it as follows,

t=AsyncoreThread(timeout=mytimeout,...)
t.start()

Since the 'run()' method of this class calls 'loop()' which is exactly
the same as asyncore.loop, only that it gets the asyncore event
handler running in a separate thread, without monopolizing the main
thread of your Python app.

I have used this model successfully in my program, HarvestMan. I wrote
to Sam Rushing, author of asyncore module and here is what he said.

<QUOTE>
"I would recommend separating out your event loop and sending it to
the
Python folks for possible inclusion in the asyncore module.  I had
always intended the loop() function in asyncore.py to be a 'field
replaceable item', and in fact have over the years made many different
versions of it.  Though this isn't really talked about in asyncore.py,
it's clear if you look at Medusa, which is where the module came
from."
</QUOTE>

Perhaps this is worth a PEP for inclusion in Python 2.4?

Thanks & Regards

-Anand



More information about the Python-list mailing list