[Async-sig] async/sync library reusage

Mon Jun 12 12:37:12 EDT 2017

Yes, but not co-existing, I hope!

On Mon, Jun 12, 2017 at 9:25 AM, Andrew Svetlov <andrew.svetlov at gmail.com>
wrote:

> Unit tests at least. Running every test in own loop is crucial fro tests
> isolation.
>
> On Mon, Jun 12, 2017 at 7:04 PM Guido van Rossum <guido at python.org> wrote:
>
>> Multiple loops in the same thread is purely theoretical -- the API allows
>> it but there's no use case. It might be necessary if a platform has a
>> UI-only event loop that cannot be extended to do I/O -- the only solution
>> to do background I/O might be to alternate between two loops. (Though in
>> that case I would still prefer a thread for the background I/O.)
>>
>> On Mon, Jun 12, 2017 at 8:49 AM, Pau Freixes <pfreixes at gmail.com> wrote:
>>
>>> And what about the rationale of having multiple loop instances in the
>>> same thread switching btw them. Im still trying to find out what patterns
>>> need this... Do you have an example?
>>>
>>> Btw thanks for the first explanation
>>>
>>> El 12/06/2017 17:36, "Guido van Rossum" <guido at python.org> escribió:
>>>
>>>> In theory it's possible to create two event loops (using
>>>> new_event_loop()), then set one as the default event loop (using
>>>> set_event_loop()), then run the other one (using run_forever() or
>>>> run_until_complete()). To tasks running in the latter event loop,
>>>> get_event_loop() would nevertheless return the former.
>>>>
>>>> On Mon, Jun 12, 2017 at 4:39 AM, Pau Freixes <pfreixes at gmail.com>
>>>> wrote:
>>>>
>>>>> Sorry a bit of topic, but I would like to figure out why older python
>>>>> versions, prior this commit [1], the get_event_loop is not considered
>>>>> deterministic
>>>>>
>>>>> does anybody know the reason behind this change?
>>>>>
>>>>>
>>>>> [1] https://github.com/python/cpython/commit/
>>>>> 600a349781bfa0a8239e1cb95fac29c7c4a3302e
>>>>>
>>>>> On Fri, Jun 9, 2017 at 6:07 PM, Ben Darnell <ben at bendarnell.com>
>>>>> wrote:
>>>>> > On Fri, Jun 9, 2017 at 11:51 AM Cory Benfield <cory at lukasa.co.uk>
>>>>> wrote:
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> My concern with multiple loops boils down to the fact that urllib3
>>>>> >> supports being used in a multithreaded context where each thread can
>>>>> >> independently make forward progress on one request. To establish
>>>>> that with a
>>>>> >> synchronous codebase you either need one event loop per thread or
>>>>> you need
>>>>> >> to spawn a background thread on startup that owns the only event
>>>>> loop in the
>>>>> >> process.
>>>>> >
>>>>> >
>>>>> > Yeah, one event loop per thread is probably the way to go for
>>>>> integration
>>>>> > with synchronous codebases. A dedicated event loop thread may
>>>>> perform better
>>>>> > but libraries that spawn threads are problematic.
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> Generally speaking I’ve not had positive results with libraries
>>>>> spawning
>>>>> >> their own threads in Python. In my experience this has tended to
>>>>> lead to
>>>>> >> programs that deadlock mysteriously or that fail to terminate in
>>>>> the face of
>>>>> >> a Ctrl+C. So I tend to prefer to have users spawn their own
>>>>> threads, which
>>>>> >> would make me want a “one-event-loop-per-thread” model: hence,
>>>>> needing a
>>>>> >> loop parameter to pass around prior to 3.6.
>>>>> >
>>>>> >
>>>>> > You can avoid the loop parameter on older versions of asyncio (at
>>>>> least as
>>>>> > long as the default event loop policy is used) by manually setting
>>>>> your
>>>>> > event loop as current before calling run_until_complete (and
>>>>> resetting it
>>>>> > afterwards).
>>>>> >
>>>>> > Tornado's run_sync() method is equivalent to asyncio's
>>>>> run_until_complete(),
>>>>> > and Tornado supports multiple IOLoops in this way. We use this to
>>>>> expose a
>>>>> > synchronous version of our AsyncHTTPClient:
>>>>> > https://github.com/tornadoweb/tornado/blob/
>>>>> 62e47215ce12aee83f951758c96775a43e80475b/tornado/httpclient.py#L54
>>>>> >
>>>>> > -Ben
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> I admit that my concerns here regarding libraries spawning their own
>>>>> >> threads may be overblown: after my series of negative experiences I
>>>>> >> basically never went back to that model, and it may be that the
>>>>> problems
>>>>> >> were more user-error than anything else. However, I feel
>>>>> comfortable saying
>>>>> >> that libraries spawning their own Python threads is definitely
>>>>> subtle and
>>>>> >> hard to get right, at the very least.
>>>>> >>
>>>>> >> Cory
>>>>> >> _______________________________________________
>>>>> >> Async-sig mailing list
>>>>> >> Async-sig at python.org
>>>>> >> https://mail.python.org/mailman/listinfo/async-sig
>>>>> >> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > Async-sig mailing list
>>>>> > Async-sig at python.org
>>>>> > https://mail.python.org/mailman/listinfo/async-sig
>>>>> > Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --pau
>>>>> _______________________________________________
>>>>> Async-sig mailing list
>>>>> Async-sig at python.org
>>>>> https://mail.python.org/mailman/listinfo/async-sig
>>>>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (python.org/~guido)
>>>>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Async-sig mailing list
>> Async-sig at python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>
> --
> Thanks,
> Andrew Svetlov
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20170612/2cebca82/attachment.html>