Persistent variable in subprocess using multiprocessing?

[Piet van Oostrum]

>>>>> mheavner <miheavner at gmail.com> (m) wrote:

>m> I'm using multiprocessing to spawn several subprocesses, each of which
>m> uses a very large data structure (making it impractical to pass it via
>m> pipes / pickling). I need to allocate this structure once when the
>m> process is created and have it remain in memory for the duration of
>m> the process. The way the multiprocessing module is set up, only the
>m> 'run' method runs within the subprocess - so creating a wrapper class
>m> with a constructor that allocates the structure in __init__ will not
>m> work, as far as I know, as this will still be within the parent
>m> process.

>m> If I were working in C/C++, I would declare the variable "static"
>m> within the function body - is there any way with the multiprocessing
>m> module to have persistent data members within subprocesses?

>m> Any ideas??

Your post is not entirely clear. Is `the process' the same as `the
subprocess'? 

Assuming it is, what is the problem? You can create the datastructure
first thing in the run method can't you?

Like this:

from multiprocessing import Process
from time import sleep
from random import random

class MyProcess(Process):

    def __init__(self, number):
        self.number = number
        Process.__init__(self)

    def run(self):
        print "Process %s started" % self.number
        self.data = range(self.number * 100000, (self.number + 1) * 100000)
        self.doit()

    def doit(self):
        for i in range(5):
            sleep(3 * random())
            self.data[i] += i
            print self.data[i]

processes = []
for k in range(10):
    p = MyProcess(k)
    p.start()
    processes.append(p)


for p in processes:
    p.join()

-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org