[Numpy-discussion] Annoyance of memap rray with multiprocessing.Pool.applay_async
Nadav Horesh
nadavh at visionsense.com
Sat Apr 3 16:29:37 EDT 2010
The following script generate the following error on every loop iteration in the function average:
Exception AttributeError: AttributeError("'NoneType' object has no attribute 'tell'",) in <bound method memmap.__del__ of memmap(xxxxxx)> ignored
where xxxxxx is a scalar (the array sum).
I get this error with numpy1.4 on a linux64 (dual core) machine. A winXP/Pentium4 with 2GB Ram could not run it since it explode the memory.
Any idea what is the origin on the error (my interset is the linux box)?
BTW if in the function average the 2nd line is commented and the 3rd line is uncommented I get no error on linux, but the win32 problem pertains.
----------------------------------------------------------------
import numpy as N
import multiprocessing as MP
import sys
try:
count = int(sys.argv[1])
except:
count = 4
filename = '%dx100x100_int32.dat' % count
def average(cube):
return [plane.mean() for plane in cube]
# return [N.asarray(plane).mean() for plane in cube]
data = N.memmap(filename, dtype=N.int32, shape=(count,100,100))
pool = MP.Pool(processes=1)
job = pool.apply_async(average, [data,])
print job.get()
import numpy as N
import multiprocessing as MP
count = 4
filename = '%dx100x100_int32.dat' % count
def average(cube):
return [plane.mean() for plane in cube]
# return [N.asarray(plane).mean() for plane in cube]
data = N.memmap(filename, dtype=N.int32, shape=(count,100,100))
pool = MP.Pool(processes=1)
job = pool.apply_async(average, [data,])
print job.get()
----------------------------------------------------------------
Nadav
More information about the NumPy-Discussion
mailing list