Using Pool map with a method of a class and a list

Luca Cerone luca.cerone at gmail.com
Wed Aug 7 04:33:05 EDT 2013


Hi Joshua thanks!

> I think you might not understand what Chris said.
> Currently this does *not* work with Python 2.7 as you suggested it would.
> >>> op = map(A.fun,l)

Yeah actually that wouldn't work even in Python 3, since value attribute used by fun has not been set.
It was my mistake in the example, but it is not the source of the problem..

> This, however, does:
> >>> op = map(A(3).fun,l)
> 
> >>> op
> 
> [1, 3, 9, 27, 81, 243, 729, 2187, 6561, 19683]
> 
> 

This works fine (and I knew that).. but is not what I want...

You are using the map() function that comes with Python. I want
to use the map() method of the Pool class (available in the multiprocessing module).

And there are differences between map() and Pool.map() apparently, so that if something works fine with map() it may not work with Pool.map() (as in my case).

To correct my example:

from multiprocessing import Pool

class A(object):
    def __init__(self,x):
        self.value = x
    def fun(self,x):
        return self.value**x

l = range(100)
p = Pool(4)
op = p.map(A(3).fun, l)

doesn't work neither in Python 2.7, nor 3.2 (by the way I can't use Python 3 for my application).

> You will find that 
> http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-> > when-using-pythons-multiprocessing-pool-ma 
> explains the problem in more detail than I understand. I suggest 
> reading it and relaying further questions back to us. Or use Python 3 

:) Thanks, but of course I googled and found this link before posting. I don't understand much of the details as well, that's why I posted here.

Anyway, thanks for the attempt :)

Luca



More information about the Python-list mailing list