[Python-ideas] An alternate approach to async IO

[Trent Nelson]

On Tue, Nov 27, 2012 at 01:25:55PM -0800, Sturla Molden wrote:
> 
> Den 27. nov. 2012 kl. 21:54 skrev Antoine Pitrou
> <solipsis at pitrou.net>:
> 
> > 
> > You don't need to acquire the GIL if you use the Py_buffer API in
> > the right way. You'll figure out the details :-)
> > 
> 
> And if he don't, this is what Cython's memoryview syntax will help him
> to do :-)
> 
> As for the rest of his idea, I think it is fundamentally flawed in two
> ways:
> 
> First, there is no fundamental difference between a non-registered
> thread and a Python thread that has released the GIL. They are both
> native OS threads running freely without bothering the interpreter. So
> given that the GIL will br released in an extension module, a list of
> threading.Thread is much easier to set up than use POSIX or Windows
> threads from C.
> 
> Second, using a thread pool to monitor another thread pool (an IOCP on
> Windows) seems a bit confused. IOCPs can fire a callback function on
> completion. It is easier to let the IOCP use the simplified GIL API
> from the callback code to acquire the GIL, and notify the Python code
> with a callback. Or en even simpler solution is to use twisted
> (twistedmatrix.org) instead of reinventing the wheel. 

    Hrm, neither of those points are really flaws with my idea, you're
    just suggesting two different ways of working with IOCP, both of
    which wouldn't scale as well as a GIL-independent approach.

    As for reinventing the wheel, I'm not.  I'm offering an idea for
    achieving high performance async IO.  In fact, Twisted would run
    like the dickens if there was an aio.events() API -- they wouldn't
    need any of the non-blocking magic behind the scenes like they do
    now to *simulate* an asynchronous, event-driven architecture.

        Trent.