[SciPy-User] multiprocessing module

Zachary Pincus zachary.pincus at yale.edu
Mon Nov 1 17:36:46 EDT 2010


> I'm trying to get multiprocess to do a bunch of independent
> calculations and save the results in a file and I'm probably going
> about it the wrong way.  I have a function defined "computeEq" that
> does the calculation and writes the result to a file (outfile) and I
> call it using:
>
> po = Pool()
> po.map_async(computeEq, product(rules,repeat=N))
> po.close()
> po.join()
> outfile.close()
>
> This seems to work for the most part but I seem to lose the last few
> calculations.  Indeed, one of my writes is truncated before the write
> is complete.

Are you taking proper precautions so that multiple workers aren't  
trying to write to the file at the same time?




More information about the SciPy-User mailing list