asyncore deficiencies

[David M. Wilson]

Josiah Carlson <jcarlson at nospam.uci.edu> wrote...

> I agree that there should probably be some sort of asyncore2; I have a 
> handful of boilerplate servers and clients that I use as a base any time 
> I need something.  Optimally, an asyncore2 library would have simple 
> clients and servers ready to be subclassed and customized as needed.

I would expect such things to be provided by modules that are
dependant on asyncore, eg. as asynchat is. It's beyond the scope of
what asyncore tries to do.


> I'm curious as to why you need to use a private map.  Most uses of 
> private maps I've seen haven't added any sort of meaningful 
> functionality that couldn't have been handled by the default map.  Yours 
> could be an exception, but I'm still curious.

I don't need a private map, however given that the other option was a
module variable, I naturally chose to provide my own to increase any
required future flexibility. Indeed there is little, if anything to be
gained from using your own map, but the option was there.


> Again, you need to use asyncore.poll.

The poll function is not documented in the manual, nor do any
docstrings suggest it is a public interface. Internally asyncore
chooses which of poll, poll2, or poll3 to use inside loop, so calling
these directly also bypasses some of asyncore's OS-independance
functionality.


> I've never used asyncore with UDP sockets, but maybe using 
> "handle_connect = handle_read" in your subclass definition could be 
> sufficient.

I see this as a workaround, and as such I think the point still
stands. :)


> That is exactly why polling is optional.  Select is available on every 
> platform, but poll is faster on platforms that support it...

That is not true at all. If you are feeling investigative, quite a
complete starting point would be <http://www.kegel.com/c10k.html>.
There is lots of information there regarding various OS
implementations async IO performance using poll, epoll, select, aio_*,
etc.


> The trick with doing this is that you end up interrupting other 
> potentially ready sockets with handling an exception.  If you are 
> feeling fiesty, I believe you can use the following and get the error 
> propagated back up:

>      def handle_error(self):
>          raise

That's a good point, but I don't have a problem restarting the poll
after handling an exception, a) because it shouldn't happen very
often, b) will probably mean shutting down the service rather than
trying to continue, and c) exceptions are generally expensive special
cases anyway.

If an exception is propogating up from the dispatcher, that means the
dispatcher's method did not handle the exception, which should mean it
gets passed back up the stack as it does with all other code,
eventually ending up with a traceback print if it wasn't handled.

I dislike the handle_error method since it is essentially redundant in
the face of Python's own exception handling mechanism. If a dispatcher
wants it's own function to handle exceptions, it should wrap it's
methods in a try block.

I assume handle_error was provided to allow graceful teardown of a
dispatcher that was erroneous, but it isn't granular enough to be
useful for tearing down complicated objects, given an exception that
could have been generated anywhere in a couple of screenfuls worth of
code, and an object that could be in any state.


> > - Decoupling from socket objects: removal of the __getitem__ magic
> > which causes screenfuls of abuse for a simple typo at the wrong stage
> > in execution.
> 
> Infinite recursion in asyncore?  Yikes, I've not hit that bug before. 
> Care to post some offending code?

It was entirely my fault, but here is the example anyway. :)

    py> from asyncore import *
    py> class D(dispatcher):
    ...     def __init__(self):
    ...             self.foo
    ...             dispatcher.__init__(self)
    ... 
    py> D()
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
      File "<stdin>", line 3, in __init__
      File "/usr/lib/python2.3/asyncore.py", line 365, in __getattr__
        return getattr(self.socket, attr)
      File "/usr/lib/python2.3/asyncore.py", line 365, in __getattr__
        return getattr(self.socket, attr)
      File "/usr/lib/python2.3/asyncore.py", line 365, in __getattr__
        return getattr(self.socket, attr)


> One could even have a framework for registering even handlers...yikes, 
> this is starting to sound like wxPython.  I don't know if that is a good 
> idea or not.

    NAME  asyncore
    Basic infrastructure for asynchronous socket service clients and
servers.

I think that answers the question.


Is anyone else interested in this? I have already made small
modifications to the asyncore module to fix a few things, but nothing
mentioned here. I could put together a proposed asyncore2 module if
there was interest. I may do it, out of need, anyway. :)


David.