asyncio - run coroutine in the background

[Kevin Conway]

> getaddrinfo is a notorious pain but I think it's just a library issue; an
async version should be possible in principle.  How does Twisted handle
it?  Does it have a version?

I think we're a little outside the scope of OP's question at this point,
but for the sake of answering this:

There are a few cases that I know of where Twisted uses the standard lib
socket DNS methods. One is when resolving names to IPv6 addresses [1] when
creating a client connection to a remote source. The other is in the
default DNS resolver that is installed in the reactor [2]. Creating client
connections allows the call to 'getaddrinfo' to block without mitigation.
The default DNS resolver, unfortunately, dispatches calls of
'gethostbyname' to a thread pool.

Without seeing the commit history, I'd assume the use of 'socket' and
threads by default is an artifact that predates the implementation of the
DNS protocol in Twisted. Twisted has, in 'twisted.names' [3], a DNS
protocol that uses UDP and leverages the reactor appropriately. Thankfully,
Twisted has a reactor method called 'installResolver' [4] that allows you
to hook in any DNS resolver implementation you want so you aren't stuck
using the default, threaded implementation.

As far as asyncio, it also defaults to an implementation that delegates to
an executor (default: threadpool). Unlike Twisted, though, it appears to
require a subclass of the event loop to override the 'getaddrinfo' method
[5].

[1]
https://github.com/twisted/twisted/blob/trunk/twisted/internet/tcp.py#L622
[2]
https://github.com/twisted/twisted/blob/trunk/twisted/internet/base.py#L257
[3] https://github.com/twisted/twisted/tree/trunk/twisted/names
[4]
https://github.com/twisted/twisted/blob/trunk/twisted/internet/base.py#L509
[5]
https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L572

On Sat, Feb 20, 2016, 03:31 Marko Rauhamaa <marko at pacujo.net> wrote:

> Paul Rubin <no.email at nospam.invalid>:
>
> > I've just felt depressed whenever I've looked at any Python async
> > stuff. I've written many Python programs with threads and not gotten
> > into the trouble that people keep warning about.
>
> Programming-model-wise, asyncio is virtually identical with threads. In
> each, I dislike the implicit state concept. I want the state to stand
> out with big block letters.
>
> > I've just felt depressed whenever I've looked at any Python async
> > stuff. I've written many Python programs with threads and not gotten
> > into the trouble that people keep warning about. But I haven't really
> > understood the warnings, so maybe they know something I don't. I just
> > write in a multiprocessing style, with every mutable object owned by
> > exactly one thread and accessed only by RPC through queues, sort of a
> > poor man's Erlang. There's a performance hit but there's a much bigger
> > one from using Python in the first place, so I just live with it.
>
> Good for you if you have been able to choose your own programming model.
> Most people have to deal with a legacy mess. Also, maintainers who
> inherit your tidy code might not be careful to ship only nonmutable
> objects in the queues.
>
> Your way of using threads works, of course, with the caveat that it is
> not possible to get rid of a blocking thread from the outside. With
> asyncio, you can at least cancel tasks.
>
>
> Marko
> --
> https://mail.python.org/mailman/listinfo/python-list
>