[Python-ideas] Better integration of multiprocessing with asyncio

[Dan O'Reilly]

Ah, right. I find the terminology confusing myself, and I probably got it
wrong. I meant "non-blocking" as in - "this won't block the event loop",
not "this won't block the coroutine". I'll try to clarify that. Thanks!

On Thu, Oct 9, 2014 at 12:03 AM, Guido van Rossum <guido at python.org> wrote:

> This looks neat. I skimmed the README, and I noticed you marked up most
> "yield from" epressions with "# non blocking". That feels confusing to me,
> because when I read asyncio code, I think fo "yield from" as blocking (the
> task, if not the world :-).  What do you think?
>
> --Guido
>
>
> On Tuesday, October 7, 2014, Dan O'Reilly <oreilldf at gmail.com> wrote:
>
>> For what it's worth, I ended up writing a module that takes the "threads
>> on top of regular multiprocessing" approach to integrate the two:
>> https://github.com/dano/aioprocessing.
>>
>> Re-implementing multiprocessing to use asyncio internally, while an
>> interesting exercise, would require a very large amount of effort both to
>> implement and maintain alongside the current multiprocessing module. I'm
>> not sure it's really worth it when using threads on top of multiprocessing
>> gives you the same effect without requiring you to basically re-implement
>> large parts of the multiprocessing module.
>>
>> Anyway, we'll see if it ends up getting much use...
>>
>> On Sat, Jul 26, 2014 at 11:48 PM, Ryan Hiebert <ryan at ryanhiebert.com>
>> wrote:
>>
>>>
>>> On Jul 26, 2014, at 10:43 PM, Guido van Rossum <guido at python.org> wrote:
>>>
>>> I'm going to go out on a limb here and say that it feels too early to
>>> me. First someone has to actually solve this problem well as a 3rd party
>>> package before we can talk about adding it to the asyncio package. It
>>> doesn't actually sound like Billiards has adapted to asyncio yet (not that
>>> I have any idea what Billiards is -- it sounds like a fork of
>>> multiprocessing actually?).
>>>
>>>
>>> Yep, Billiard is a fork of multiprocessing:
>>> https://pypi.python.org/pypi/billiard
>>>
>>
>>
>
> --
> --Guido van Rossum (on iPad)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141009/c73244a3/attachment-0001.html>