[IPython-dev] IPython parallel and objects

[Moritz Beber]

Hi,

On Wed, Jan 28, 2015 at 1:40 PM, thwiouz <maidos93 at laposte.net> wrote:

> Hi guys,
>
> I'd like to parallelize the following loop:
>
> m1 = np.arange(0, 10, 100)
> m2 = np.arange(0, 10, 100)
>
> for m_ in m1:
>     for n_2 in m2:
>         tmp = optimizer.compute_expectation([m_, n_])
>         res.append(tmp)
>
> with type(optimizer) some class that I've defined. How could I handle it?
>
> I tried to do the same as in
>
> http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython
> so I created a lambda function f
>
> f = lambda x, y: optimizer.compute_expectation([x, y])
>
> but it's not working with a load balanced view, saying that optimizer is
> not
> defined.
>

IPython only automatically transmits the function definition used in a
`map` call or similar. Your optimizer is inside your lambda function and
thus unknown on the remote kernels. In order to overcome this, take a look
here:

http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#remote-function-decorators

and here:

http://ipython.org/ipython-doc/2/parallel/parallel_task.html#dependencies

this could be helpful, too:

http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#moving-python-objects-around


HTH,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150128/91860336/attachment.html>