[Numpy-discussion] Numeric objects, os.spawnlp and pickle

[Aureli Soria Frisch]

Hi all,

I am trying to make run a numerical computation (with arrays) in 
different computers simultaneously (in parallel). The computation is 
done under Linux.

For that purpose a master organizes the process and send rexec 
(remote execute) commands to the different slaves via the python 
command spawnlp. The slaves execute the script specified through 
rexec.

Inside this script the slaves open a file with the arguments of the 
process, which were serialized via pickle, then make the numerical 
computation, and write the result (a NumPy array) again via pickle in 
a file. This file is opened by the master, which uses the different 
results.

I am having the problem that the master sometimes (the problem does 
not happen always!!!) open the result and load an object of <type 
'instance'> instead of the expected object of <type 'array'> (what 
then produces an error). I have tested the type of the objects in the 
slaves and it is always 'array'.

Has someone made similar experiences by 'pickling' arrays? Could it 
be a problem of the different computers running versions of Python 
from 2.0 to 2.2.1?  Or a problem of different versions of NumPy?

Is there any other way for doing such a parallel computation?

Thanks for the time...
Regards,
Aureli
-- 
#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail: aureli at ipk.fhg.de
fon: +49 30 39006-143
fax: +49 30 3917517
web: http://vision.fhg.de/~aureli/web-aureli_en.html
#################################