multiprocessing python2.6.1

km srikrishnamohan at gmail.com
Sun Feb 15 02:38:31 EST 2009


Dear all,


I have a problem with multiprocessing module usage with python2.6.1
Here Pool object is to be instantiated with 50 processes  and the method
'run' has to be called with pool object.
My actual requirement is to reuse the 50 processes for further processing
until 100 items.

#########################
import multiprocessing as mp

def add(a,b):
    return a+b


if __name__ == '__main__':
    p = mp.Pool(processes=50)
    y = 40
    for x in  range(100):
        p.apply_async(run, (x, y) )
##########################
The problem now is that it creates only 32 processes (and not more) and
exits with processing first 32 numbers  from range(100) .
Is there a cap on the number of subprocesses one can fork ?
Even then,  once these 32 processes r free, I would like to reuse them with
next 32 inputs (x) from  range(100 ) in that order.
how could i accomplish this ?

thanks in advance.
KM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090215/45574134/attachment.html>


More information about the Python-list mailing list