[Python-ideas] An alternate approach to async IO

[Trent Nelson]

On Tue, Nov 27, 2012 at 10:30:10AM -0800, Richard Oudkerk wrote:
> On 27/11/2012 5:49pm, Trent Nelson wrote:
> >      Here's the "idea" I had, with zero working code to back it up:
> >      what if we had a bunch of threads in the background whose sole
> >      purpose it was to handle AIO?  On Windows/AIX, they would poll
> >      GetQueuedCompletionStatus, on Solaris, get_event().
> >
> >      They're literally raw pthreads and have absolutely nothing to
> >      do with Python's threading.Thread() stuff.  They exist solely
> >      in C and can't be interfaced to directly from Python code.
> >
> >      ....which means they're free to run outside the GIL, and thus,
> >      multiple cores could be leveraged concurrently.  (Only for
> >      processing completed I/O, but hey, it's better than nothing.)
> >
> >      The threads would process the completion port events via C code
> >      and allocate necessary char * buffers on demand.  Upon completion
> >      of their processing, let's say, reading 4096 bytes from a socket,
> >      they push their processed event and data to an interlocked* list,
> >      then go back to GetQueuedCompletionStatus/get_event.
> 
> But you have to allocate the buffer *before* you initiate an overlapped 
> read.  And you may as well make that buffer a Python bytes object (which 
> can be shrunk if it is too large).  That leaves no "processing" that can 
> usefully be done by a C level thread pool.

    I'm a little confused by that last sentence.  The premise of my idea
    is being able to service AIO via simple GIL-independent threads that
    really just copy data from A to B.  The simple fact that they don't
    have to acquire the GIL each time the completion port has an event
    seems like a big win, no?

    (So I'm not sure if you're saying that this wouldn't work because
    you may as well use Python bytes objects, and they can't be accessed
    willy-nilly from non-GIL threads... or if you're saying they can,
    but there's no benefit from a C-level thread copying data to/from
    buffers independent of the GIL.)

> Also, note that (at least on Windows) overlapped IO automatically makes 
> use of a hidden thread pool to do the IO.

    I don't think that detail impacts my general idea though, right?

        Trent.