Open Multiples Files at same time with multiprocessing - How "declare" dynamically the var?

macm moura.mario at gmail.com
Thu Nov 11 06:28:45 EST 2010


Hi Folks

My approach to open multiples files at same time is:

    def openFiles(self,file,q):
        fp = open(file, 'rb')
        fp.seek(0)
        fcontent = fp.read()
        fp.close()
        q.put(fcontent)
        return

    def testOpen(self):
        L =
['file1.txt','file2.txt','file3.txt','file4.txt','file5.txt']
        d1 = []
        for x in L:
            z=L.index(x)
            q = Queue()
            m = Process(target=self.openFiles, args=(x,q,))
            m.start()
            d1.append(q.get()) # <= This get is locking ? It is mean:
"wait m.start(), like m.join()??"

        print list(d1)
        return

Is the best way? Is q.get() locking the loop?

I feel that q.get() is locking so I would like to "declare"
dynamically the var q{z} in python? is it possible? How?

    def testOpen(self):
        L =
['file1.txt','file2.txt','file3.txt','file4.txt','file5.txt']
        d1 = []
        for x in L:
            z=L.index(x)
            q{z} = Queue()
            m{z} = Process(target=self.openFiles, args=(x,q{z},))
            m{z}.start()

	for x in L:
            z=L.index(x)
	    d1.append(q{z}.get()) # <= So now I am sure that q{z}.get() isn't
lock

        print list(d1)
        return

I tried use list but didn't work look below one shot.

Best Regards

macm

I tried :

    def testOpen(self):
        L = ['file1.txt','file2.txt',
             'file3.txt','file4.txt',
             'file5.txt']
        d1 = []
        q = []
        m = []
        for x in L:
            z=L.index(x)
            q.insert(z, Queue())
            m.insert(z,Process(target=self.openFiles, args=(x,q[z])))
            m[z].start()

        # Now I am sure. Isnt lock
        for x in L:
            z=L.index(x)
            d1.append(q[z].get())

        print list(d1)
        return



More information about the Python-list mailing list