[IPython-dev] Using IPython Parallel inside a class

[Doug Blank]

On Wed, Dec 3, 2014 at 12:37 PM, Rafał Skolasiński <rjtskolasinski at gmail.com
> wrote:

> Hi,
>
> It doesn't look like a solution. I still get bunch of errors. And I would
> like to use self.hamiltonian instance, not create more instances of it. the
> hamiltonian object in my real (not just a test) simulation code enters as
> the argument to the Simulation class constructor.
>

Your original code gave errors trying to serialize the Hamiltonian instance
in order to make copies for each machine in the cluster. Some objects can't
be serialized like that. In any event, each machine needs to have access to
the data that it will process. So, you need to somehow get that data (in
whole or in part) to each machine.

Hope that helps,

-Doug



>
> And I would also like to execute hamiltonian.get_matrix method in
> parallel. The version of that method I use in my real simulation code is a
> little more complex and also take some time to be executed. Not maybe as
> much as diagonalization but still.
>
> Cheers,
> Rafal
>
> 2014-12-03 18:28 GMT+01:00 Doug Blank <doug.blank at gmail.com>:
>
>> I haven't done much parallel IPython, but it looks like you just need to
>> define/import everything on the remote clients, something like:
>>
>> def get_energy(momentum):
>>     import scipy.sparse.linalg as sla
>>     from scipy.sparse import coo_matrix
>>     from scipy.sparse import csc_matrix
>>     import numpy as np
>>     class Hamiltonian:
>>         def get_matrix(self, k):
>>             # sleep(3.0)
>>             return k**2 * np.diag(range(100))
>>
>>     hamiltonian = Hamiltonian()
>>     mat = hamiltonian.get_matrix(momentum)
>>     mat = csc_matrix(mat)
>>     ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
>>     return ev[0]
>>
>> def compute_parallel(momenta):
>>     rc = parallel.Client()
>>     dview = rc[:]
>>     spectrum = dview.map_sync(get_energy,momenta)
>>     return spectrum
>>
>> -Doug
>>
>> On Wed, Dec 3, 2014 at 12:21 PM, Rafał Skolasiński <
>> rjtskolasinski at gmail.com> wrote:
>>
>>> Oh, in the file I attached there should ofc be line
>>> spectrum = dview.map_sync(self.get_energy,momenta)
>>> instead of
>>> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
>>>
>>> Sorry for mistake.
>>> Raphael
>>>
>>>
>>> 2014-12-03 18:05 GMT+01:00 Rafał Skolasiński <rjtskolasinski at gmail.com>:
>>>
>>>> Hi Guys,
>>>>
>>>> I hope this is the right place to ask this kind of question.
>>>>
>>>> I would like to use IPython parallel in my physical simulation.
>>>> In my mind I've got class Simulation, which contain as a field object
>>>> hamiltonian of class Hamiltonian.
>>>>
>>>> Method get_energy should call  hamiltonian.get_matrix(momentum),
>>>> diagonalize it with scipy method and return energies.
>>>>
>>>> I would like to execute it in parallel for a list of different momenta.
>>>>
>>>>
>>>> I tested it with a standard python 'map' function and it worked so I
>>>> thought that easiest way would be to use 'dview.map_sync' in the same way
>>>> but it doesn't looks like a proper way of doing it.
>>>>
>>>> I am attaching my testing code. I would be very grateful for any help
>>>> and ideas how to do it in a nice and clean way.
>>>>
>>>> Cheers,
>>>> Raphael
>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/2967fc07/attachment.html>