[Python-ideas] Asynchronous IO ideas for Python

[Paul Colomiets]

Hi Trent,

On Wed, Nov 26, 2014 at 10:36 PM, Trent Nelson <trent at trent.me> wrote:
> Have you seen this?:
>
>         https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores
>
> I spend the first 80-ish slides on async I/O.
>
> (That was a year ago.  I've done 2-3 sprints on it since then and have gotten it to a point where I can back up the claims with hard numbers on load testing benchmarks, demonstrated in the most recent video: https://www.youtube.com/watch?v=4L4Ww3ROuro.)
>
>

Thanks. Have never seen it before. Do you have some detailed
description about how I/O subsystem works? From what I see, pyparallel
relies on offloading I/O operations to the IOCP. And it seems that I/O
kernel which I'm describing can replace IOCP on unix. And allowing the
C code to do more high-level stuff (like reconnecting and message
delimitation), would be of a benefit for pyparallel too, as it would
allow less heap allocations, smaller number of heap snasphot, etc.

Still PyParallel, is much bigger departure from current Python. As it
doesn't allow arbitrary code to run in threads, so it's much longer
perspective, than I/O kernel which will be just a library on PyPI.

-- 
Paul