multiprocessing, what am I doing wrong?

Eric Frederich eric.frederich at gmail.com
Thu Feb 23 12:59:56 EST 2012


Below is some pretty simple code and the resulting output.
Sometimes the code runs through but sometimes it just freezes for no
apparent reason.
The output pasted is where it just got frozen on me.
It called start() on the 2nd worker but the 2nd worker never seemed to
enter the run method.

################### the code

#!/usr/bin/env python

import sys
import Queue
import multiprocessing
import time
todo = multiprocessing.Queue()

for i in xrange(100):
    todo.put((i, i+1, i+2))

def FOO(a, b, c):
    print 'foo', a, b, c
    return (a + b) * c

class MyWorker(multiprocessing.Process):
    def __init__(self, inbox, outbox):
        super(MyWorker, self).__init__()
        self.inbox = inbox
        self.outbox = outbox
        print >> sys.stderr, '1' * 80; sys.stderr.flush()
    def run(self):
        print >> sys.stderr, '2' * 80; sys.stderr.flush()
        while True:
            try:
                args = self.inbox.get_nowait()
            except Queue.Empty:
                break
            self.outbox.put(FOO(*args))

print >> sys.stderr, 'a' * 80; sys.stderr.flush()
result_queue = multiprocessing.Queue()

print >> sys.stderr, 'b' * 80; sys.stderr.flush()
w1 = MyWorker(todo, result_queue)
print >> sys.stderr, 'c' * 80; sys.stderr.flush()
w2 = MyWorker(todo, result_queue)

print >> sys.stderr, 'd' * 80; sys.stderr.flush()
w1.start()
print >> sys.stderr, 'e' * 80; sys.stderr.flush()
w2.start()
print >> sys.stderr, 'f' * 80; sys.stderr.flush()

for i in xrange(100):
    print result_queue.get()



################### the output

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
11111111111111111111111111111111111111111111111111111111111111111111111111111111
cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
11111111111111111111111111111111111111111111111111111111111111111111111111111111
dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
22222222222222222222222222222222222222222222222222222222222222222222222222222222
foo 0 1 2
foo 1 2 3
foo 2 3 4
foo 3 4 5
foo 4 5 6
22222222222222222222222222222222222222222222222222222222222222222222222222222222
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
2
9
20
35
54
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120223/1c1b62fe/attachment.html>


More information about the Python-list mailing list