[Async-sig] async/sync library reusage

Mon Jun 12 12:57:29 EDT 2017

Yes, but with one exception: default event loop created on module import
stage might co-exist with a loop created for test.
It leads to mystic hangs, you know.
Please recall code like:
    class A:
         mongodb = motor.motor_asyncio.AsyncIOMotorClient()

On Mon, Jun 12, 2017 at 7:37 PM Guido van Rossum <guido at python.org> wrote:

> Yes, but not co-existing, I hope!
>
> On Mon, Jun 12, 2017 at 9:25 AM, Andrew Svetlov <andrew.svetlov at gmail.com>
> wrote:
>
>> Unit tests at least. Running every test in own loop is crucial fro tests
>> isolation.
>>
>> On Mon, Jun 12, 2017 at 7:04 PM Guido van Rossum <guido at python.org>
>> wrote:
>>
>>> Multiple loops in the same thread is purely theoretical -- the API
>>> allows it but there's no use case. It might be necessary if a platform has
>>> a UI-only event loop that cannot be extended to do I/O -- the only solution
>>> to do background I/O might be to alternate between two loops. (Though in
>>> that case I would still prefer a thread for the background I/O.)
>>>
>>> On Mon, Jun 12, 2017 at 8:49 AM, Pau Freixes <pfreixes at gmail.com> wrote:
>>>
>>>> And what about the rationale of having multiple loop instances in the
>>>> same thread switching btw them. Im still trying to find out what patterns
>>>> need this... Do you have an example?
>>>>
>>>> Btw thanks for the first explanation
>>>>
>>>> El 12/06/2017 17:36, "Guido van Rossum" <guido at python.org> escribió:
>>>>
>>>>> In theory it's possible to create two event loops (using
>>>>> new_event_loop()), then set one as the default event loop (using
>>>>> set_event_loop()), then run the other one (using run_forever() or
>>>>> run_until_complete()). To tasks running in the latter event loop,
>>>>> get_event_loop() would nevertheless return the former.
>>>>>
>>>>> On Mon, Jun 12, 2017 at 4:39 AM, Pau Freixes <pfreixes at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Sorry a bit of topic, but I would like to figure out why older python
>>>>>> versions, prior this commit [1], the get_event_loop is not considered
>>>>>> deterministic
>>>>>>
>>>>>> does anybody know the reason behind this change?
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/python/cpython/commit/600a349781bfa0a8239e1cb95fac29c7c4a3302e
>>>>>>
>>>>>> On Fri, Jun 9, 2017 at 6:07 PM, Ben Darnell <ben at bendarnell.com>
>>>>>> wrote:
>>>>>> > On Fri, Jun 9, 2017 at 11:51 AM Cory Benfield <cory at lukasa.co.uk>
>>>>>> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> My concern with multiple loops boils down to the fact that urllib3
>>>>>> >> supports being used in a multithreaded context where each thread
>>>>>> can
>>>>>> >> independently make forward progress on one request. To establish
>>>>>> that with a
>>>>>> >> synchronous codebase you either need one event loop per thread or
>>>>>> you need
>>>>>> >> to spawn a background thread on startup that owns the only event
>>>>>> loop in the
>>>>>> >> process.
>>>>>> >
>>>>>> >
>>>>>> > Yeah, one event loop per thread is probably the way to go for
>>>>>> integration
>>>>>> > with synchronous codebases. A dedicated event loop thread may
>>>>>> perform better
>>>>>> > but libraries that spawn threads are problematic.
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >> Generally speaking I’ve not had positive results with libraries
>>>>>> spawning
>>>>>> >> their own threads in Python. In my experience this has tended to
>>>>>> lead to
>>>>>> >> programs that deadlock mysteriously or that fail to terminate in
>>>>>> the face of
>>>>>> >> a Ctrl+C. So I tend to prefer to have users spawn their own
>>>>>> threads, which
>>>>>> >> would make me want a “one-event-loop-per-thread” model: hence,
>>>>>> needing a
>>>>>> >> loop parameter to pass around prior to 3.6.
>>>>>> >
>>>>>> >
>>>>>> > You can avoid the loop parameter on older versions of asyncio (at
>>>>>> least as
>>>>>> > long as the default event loop policy is used) by manually setting
>>>>>> your
>>>>>> > event loop as current before calling run_until_complete (and
>>>>>> resetting it
>>>>>> > afterwards).
>>>>>> >
>>>>>> > Tornado's run_sync() method is equivalent to asyncio's
>>>>>> run_until_complete(),
>>>>>> > and Tornado supports multiple IOLoops in this way. We use this to
>>>>>> expose a
>>>>>> > synchronous version of our AsyncHTTPClient:
>>>>>> >
>>>>>> https://github.com/tornadoweb/tornado/blob/62e47215ce12aee83f951758c96775a43e80475b/tornado/httpclient.py#L54
>>>>>> >
>>>>>> > -Ben
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >> I admit that my concerns here regarding libraries spawning their
>>>>>> own
>>>>>> >> threads may be overblown: after my series of negative experiences I
>>>>>> >> basically never went back to that model, and it may be that the
>>>>>> problems
>>>>>> >> were more user-error than anything else. However, I feel
>>>>>> comfortable saying
>>>>>> >> that libraries spawning their own Python threads is definitely
>>>>>> subtle and
>>>>>> >> hard to get right, at the very least.
>>>>>> >>
>>>>>> >> Cory
>>>>>> >> _______________________________________________
>>>>>> >> Async-sig mailing list
>>>>>> >> Async-sig at python.org
>>>>>> >> https://mail.python.org/mailman/listinfo/async-sig
>>>>>> >> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > Async-sig mailing list
>>>>>> > Async-sig at python.org
>>>>>> > https://mail.python.org/mailman/listinfo/async-sig
>>>>>> > Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --pau
>>>>>> _______________________________________________
>>>>>> Async-sig mailing list
>>>>>> Async-sig at python.org
>>>>>> https://mail.python.org/mailman/listinfo/async-sig
>>>>>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --Guido van Rossum (python.org/~guido)
>>>>>
>>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>> _______________________________________________
>>> Async-sig mailing list
>>> Async-sig at python.org
>>> https://mail.python.org/mailman/listinfo/async-sig
>>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>
>> --
>> Thanks,
>> Andrew Svetlov
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-- 
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20170612/f3ac6689/attachment-0001.html>