[issue37373] Configuration of windows event loop for libraries

[Nathaniel Smith]

Nathaniel Smith <njs at pobox.com> added the comment:

Yeah, that's the question. I've dug into the AFD_POLL stuff more now, and... There's a lot of edge cases to think about. It certainly *can* be done, because this is the primitive that 'select' and friends are built on, so obviously it can do anything they can do. But it also has some very sharp corners that 'select' knows how to work around, and we might not, since the folks writing 'select' could look at the internals and we can't.

The good news is that libuv has been shipping a version of this code for many years, and trio started shipping a version yesterday, so we can treat those as field experiments to gather data for asyncio. (I think that rust's mio library is also moving this way, but I'm not as familiar with the details. And wepoll is a good reference too, but I don't know how widely-deployed it is.)

The libuv implementation is very conservative, and falls back to calling 'select' in a thread if anything looks the slightest bit weird. The author of that code told me that he now thinks that's too conservative, esp. since some if the issues they were worrying about in the win xp era are no longer relevant. So Trio's version is more aggressive. I'm very curious to see how it goes.

I do think the incompatibilities between the different aio event loops are really a problem and the ultimate goal should be to support the full feature set everywhere. The question is how to make that viable.

Getting more experience with AFD_POLL will help make it possible for aio to implement its own version, if that's the direction we want to go.

Maybe it would also be helpful to try to find the right folks inside Microsoft to get more information about this? My understanding is that their official position on AFD_POLL is "don't do that", but you can't put the genie back into the bottle...

Alternatively: it seems like this is really highlighting the downsides of aio maintaining its own written-from-scratch event loops. Would it make sense to promote uvloop to the One True Event Loop? I get that there are significant complications to doing that, but there are also significant benefits: we'd get a more mature event loop core that we don't have to maintain alone, and it would fix tornado.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37373>
_______________________________________