[IPython-dev] Parallel support for external wrapperkernels in master?

[Doug Blank]

On Tue, Sep 9, 2014 at 3:05 PM, MinRK <benjaminrk at gmail.com> wrote:

> There aren't currently plans to make IPython.parallel language agnostic
> (it will remain an IPython project, not a Jupyter one), because it doesn't
> make much sense if the Client and Kernel aren't in the same language, so
> making a kernel available from a new language should also mean writing a
> new Client object in order for it to be useful.  This is distinctly
> different from the regular kernel-client relationship, which mostly talk
> code-as-text, which doesn't require the frontend to know much about the
> kernel. IPython.parallel talks code-as-code (pickle), because it's meant
> for interacting with actual results, rather than just their representation.
>

Ok, I understand. Thinking about this a bit, it should be possible to wrap
up the Client so that we can start our Python-based kernels on each node in
the cluster, and then move values back and forth through some mechanism
(maybe just strings, or maybe a serialize/unserialize process). That way,
we can still use the IPython infrastructure, and our kernels as
well---without having to reinvent too much. Will report back...

-Doug


>
> -MinRK
>
> On Tue, Sep 9, 2014 at 11:47 AM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> On Tue, Sep 9, 2014 at 2:25 PM, Fernando Perez <fperez.net at gmail.com>
>> wrote:
>>
>>>
>>> On Tue, Sep 9, 2014 at 11:23 AM, Doug Blank <doug.blank at gmail.com>
>>> wrote:
>>>
>>>> Just wondering if parallel support for external kernels (based on
>>>> wrapperkernel) is working in master yet, or planned to work soon?
>>>>
>>>
>>> You always doing the crazy stuff ;)
>>>
>>> No, we haven't really looked into the intersection of the parallel
>>> machinery and alternate kernels.  So go forward and blaze trail! :)
>>>
>>
>> This isn't too crazy, now that y'all have the wrapperkernel nicely
>> factored out :)
>>
>> This must be pretty close to being able to be made to work, at least for
>> those kernels that are Python-based, right?
>>
>> Just can't wait to run the chatbot_kernel in parallel :)
>>
>> -Doug
>>
>>
>>>
>>> Cheers,
>>>
>>> f
>>> --
>>> Fernando Perez (@fperez_org; http://fperez.org)
>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140909/2d4280e7/attachment.html>