[Python-ideas] Asynchronous IO ideas for Python

[Sturla Molden]

Paul Colomiets <paul at colomiets.name> wrote:

> I've written an article about how I perceive the future of
> asynchronous I/O in Python. It's not something that should directly be
> incorporated into python now, but I believe it's useful for
> python-ideas list.
> 
> https://medium.com/@paulcolomiets/the-future-of-asynchronous-io-in-python-ce200536d847

This approximately how asynchronous I/O is implemented on Windows (IOCP)
and Mac and FreeBSD (GCD): You have a thread-pool that operates independent
of the main program thread, and then you can enqueue I/O operations as work
tasks to the thread pool. You suggest to have a singleton "I/O kernel"
thread in Python, but it is actually not that different from having a pool
of worker threads.

In a very minimalistic way, one could implement something similar to the
set of I/O functions present in GCD. This API is already designed to have
the smallest possible complexity, and yet it is as powerful as Windows'
IOCP for most practical purposes.

https://developer.apple.com/library/mac/documentation/Performance/Reference/GCD_libdispatch_Ref/index.html

On Windows it could run on top of IOCP, on Mac and FreeBSD it could run on
top of GCD and thus use kqueue under the hood. Linux would actually be the
hardest platform, but IOCPs have been implemented with epoll and a
threadpool in Wine.


Sturla