[Python-ideas] The async API of the future

[Ben Darnell]

On Sun, Nov 4, 2012 at 7:26 AM, Guido van Rossum <guido at python.org> wrote:

> I've been thinking about this too. I can see the scalability issues with
> select(),  but frankly, poll(), epoll(), and even kqueue() all look similar
> in O() behavior to me from an API perspective. I guess the differences are
> in the kernel -- but is it a constant factor or an unfortunate O(N) or
> worse? To what extent would this be overwhelmed by overhead in the Python
> code we're writing around it? How bad is it to add extra
> register()/unregister() (or (modify()) calls per read operation?
>


The extra system calls add up.  The interface of Tornado's IOLoop was based
on epoll (where the internal state is roughly a mapping {fd: event_set}),
so it requires more register/unregister operations when running on kqueue
(where the internal state is roughly a set of (fd, event) pairs).  This
shows up in benchmarks of the HTTPServer; it's faster on platforms with
epoll than platforms with kqueue.  In low-concurrency scenarios it's
actually faster to use select() even when kqueue is available (or maybe
that's a mac-specific quirk).

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121104/3c206e0a/attachment.html>