Python 2.6's multiprocessing lock not working on second use?

Frédéric Sagnes speedup at gmail.com
Fri Jan 16 11:41:21 EST 2009


Hi,

I ran a few tests on the new Python 2.6 multiprocessing module before
migrating a threading code, and found out the locking code is not
working well. In this case, a pool of 5 processes is running, each
trying to get the lock and releasing it after waiting 0.2 seconds
(action is repeated twice). It looks like the multiprocessing lock
allows multiple locking after the second pass. Running the exact same
code with threads works correctly.

Further down is the test code, output is great when running with
threads (the sequence of lock/unlock looks good), but the output gets
mixed up (mutliple locks in a row) when running with processes.

My setup is : Mac OS X 10.5 running Python 2.6.1 from MacPython

Did I do something wrong, or is there a limitation for multiprocessing
locks that I am not aware of?

Thank you for your help!

-- Fred

-------------------------------

#!/usr/bin/python
# -*- coding: utf-8 -*-

from multiprocessing import Process, Queue, Lock
from Queue import Empty
from threading import Thread
import time

class test_lock_process(object):
    def __init__(self, lock, id, queue):
        self.lock = lock
        self.id = id
        self.queue = queue
        self.read_lock()

    def read_lock(self):
        for i in xrange(2):
            self.lock.acquire()
            self.queue.put('[proc%d] Got lock' % self.id)
            time.sleep(.2)
            self.queue.put('[proc%d] Released lock' % self.id)
            self.lock.release()

def test_lock(processes=10, lock=Lock(), process=True, queue=None):
    print_result = False
    if queue == None:
        print_result = True
        queue = Queue()

    threads = []
    for i in xrange(processes):
        if process: threads.append(Process(target=test_lock_process,
args=(lock,i,queue,)))
        else: threads.append(Thread(target=test_lock_process, args=
(lock,i,queue,)))

    for t in threads:
        t.start()

    for t in threads:
        t.join()

    if print_result:
        try:
            while True: print queue.get(block=False)
        except Empty:
            pass

if __name__ == "__main__":
    #test_lock(processes=5, process=True)
    test_lock(processes=5)



More information about the Python-list mailing list