[Async-sig] Warts both in asyncio way and greenlet-based approach

[Andrew Svetlov]

@cory I didn't want to say "re-implementing requests in asyncio way is
impossible".

I just want to say: it's hard.
Requires massive code rewriting.
Moreover, it's pain to support both Python 2 and 3 with asyncio inthe same
code base.
Really only after dropping python 2 support you are able to adopt requests
for asyncio.
Hopefully after 2020.
Requests library (as well as sqlalchemy) are very crucial scaffolds for
python community, I suspect you will be forced to support Python 2 up to
the official death of branch at least.

On Wed, Aug 3, 2016 at 5:27 PM Cory Benfield <cory at lukasa.co.uk> wrote:

>
> On 2 Aug 2016, at 19:39, Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
>
> Long story short: in async world every IO library should use asyncio.
> Unfortunately for the most  popular libraries (requests, django, flask,
> sqlalchemy etc.) it's impossible to rewrite the code keeping backward
> compatibility. Much easier to create a new library than rewrite existing
> one.
>
>
> Removing my hyper hat for a moment and replacing it with my Requests hat:
> we disagree. Some of the discussion has happened on GitHub at [1], but I
> can summarise it again here.
>
> One of the nice things with an event loop is that you can retrofit
> synchronous code on top of it. That is, if you have an event loop, you can
> always turn your evented code (code that returned a Deferred (or
> Deferred-alike) or a coroutine) into synchronous code by simply running the
> event loop in the synchronous call. That is, given a method signature like
> this:
>
> async def get(url):
>     return await do_the_work(url)
>
> It can be turned into this:
>
> def sync_get(url):
>     return some_event_loop.run(get(url))
>
> The reason Requests has not previously done this is that it required us to
> do one of two things. In the first instance, we could bundle or depend on
> an event loop (e.g. Twisted’s reactors). That’s annoying: it’s a lot of
> code that we don’t really care about in order to achieve a task that is
> fundamentally an implementation detail. The other option is to write
> backends for all possible event loops *and* a kind of event-loop-alike
> synchronous approach that can be run as a series of coroutines, but that is
> actually entirely synchronous under the covers (essentially, ship a
> coroutine runner that never waits for anything). Both of these were a lot
> of work for fairly minimal gain.
>
> However, with asyncio’s event loop becoming shipped with the language, and
> with the other event loop implementations shimming in compatibility layers,
> libraries like Requests have a perfect escape hatch. Now we only need *two*
> backends: one for the asyncio event loop, and one synchronous one. And if
> we’re prepared to depend on Python 3.4+, we only need *one* backend:
> asyncio, because we can rely on it being present.
>
> This drastic reduction in the space of event loops we need to support
> suddenly makes it much more viable to consider adjusting the way we do I/O.
> There’s still a *lot* of work there, and no-one has actually started
> sitting down to do the hard stuff, but it’s certainly not impossible any
> longer.
>
> Cory
>
> [1]:
> https://github.com/kennethreitz/requests/issues/1390#issuecomment-224772923
>
> --
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160804/b8f829ec/attachment.html>