[IPython-dev] Environment in load balanced map_async?

[Doug Blank]

On Fri, Sep 12, 2014 at 7:21 PM, MinRK <benjaminrk at gmail.com> wrote:

> This is the single most common confusion users face when getting started
> with IPython.parallel: namespace management, which I have discussed on
> Stack Overflow a few
> <http://stackoverflow.com/questions/10857250/python-name-space-issues-with-ipython-parallel/10859394#10859394>
>  times <http://stackoverflow.com/a/12307741/938949>, and in the IPython.parallel
> tutorial
> <http://nbviewer.ipython.org/github/minrk/IPython-parallel-tutorial/blob/master/tutorial/Remote%20Execution.ipynb#working-with-the-engine-namespace>
> .
>
> The gist is that in your code, kernels is just a reference to
> __main__.kernels on the engine, which hasn’t been defined. You would
> solve the same problem with view['kernels'] = kernels, or (depending on
> what `kernels` is, this doesn’t always work for serialization reasons) view.map_async(kernels.do_function_direct,
> [1,2,3]).
>
Thanks for these many answers :) kernels couldn't be serialized, and
neither could the method do_function_direct. The interactive function
wrapper solved the problem.

-Doug


> -MinRK
> ​
>
> On Fri, Sep 12, 2014 at 2:17 PM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> Can you imagine why this would fail:
>>
>> view_load_balanced.map_async(
>>             lambda arg: kernels.do_function_direct(arg),
>>             [1, 2, 3])
>>
>> but that this ugly, terrible hack would work:
>>
>> view.execute("""
>> import os
>> os.kernels = kernels
>> """)
>> view_load_balanced.map_async(
>>             lambda arg: os.kernels.do_function_direct(arg),
>>             [1, 2, 3])
>>
>> Why would I not be able to find a variable in the environment, but I can
>> put it in a module, and find it there?
>>
>> -Doug
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140912/f6e04248/attachment.html>