multiprocessing, what am I doing wrong?

[Eric Frederich]

If I do a time.sleep(0.001) right at the beginning of the run() method,
then it completes fine.
I was able to run it through a couple hundred times without problem.
If I sleep for less time than that or not at all, it may or may not
complete.

On Mon, Feb 27, 2012 at 9:38 PM, MRAB <python at mrabarnett.plus.com> wrote:

> On 27/02/2012 16:57, Eric Frederich wrote:
>
>> Still freezing sometimes, like 1 out of 10 times that I run it.
>> Here is updated code and a couple of outputs.
>>
>>  [snip]
> I don't know what the problem is. All I can suggest is a slightly
> modified version.
>
> If a worker that says it's terminating without first saying that it's
> got nothing, then I can only assume that the worker had some uncaught
> exception.
>
>
>
> #!/usr/bin/env python
>
> import sys
> import Queue
> import multiprocessing
> import time
>
> def FOO(a, b, c):
>    print 'foo', a, b, c
>    return (a + b) * c
>
> class MyWorker(multiprocessing.**Process):
>    def __init__(self, name, inbox, outbox):
>        super(MyWorker, self).__init__()
>        self.name = name
>        self.inbox = inbox
>        self.outbox = outbox
>        print >> sys.stderr, 'Created %s' % self.name; sys.stderr.flush()
>    def run(self):
>        print >> sys.stderr, 'Running %s' % self.name; sys.stderr.flush()
>        try:
>
>            while True:
>                try:
>                    args = self.inbox.get_nowait()
>                    print >> sys.stderr, '%s got something to do' %
> self.name; sys.stderr.flush()
>                except Queue.Empty:
>                    print >> sys.stderr, '%s got nothing' % self.name;
> sys.stderr.flush()
>                    break
>                self.outbox.put(FOO(*args))
>        finally:
>            print >> sys.stderr, '%s is terminating' % self.name;
> sys.stderr.flush()
>
>
> if __name__ == '__main__':
>    # This file is being run as the main script. This part won't be
>    # run if the file is imported.
>
>    print >> sys.stderr, 'Creating todo queue'; sys.stderr.flush()
>    todo = multiprocessing.Queue()
>
>    for i in xrange(100):
>        todo.put((i, i + 1, i + 2))
>
>    print >> sys.stderr, 'Creating results queue'; sys.stderr.flush()
>    result_queue = multiprocessing.Queue()
>
>    print >> sys.stderr, 'Creating Workers'; sys.stderr.flush()
>    w1 = MyWorker('Worker 1', todo, result_queue)
>    w2 = MyWorker('Worker 2', todo, result_queue)
>
>    print >> sys.stderr, 'Starting Worker 1'; sys.stderr.flush()
>    w1.start()
>    print >> sys.stderr, 'Starting Worker 2'; sys.stderr.flush()
>    w2.start()
>
>    for i in xrange(100):
>        print result_queue.get()
> --
> http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120228/1940bcfd/attachment-0001.html>