Multiprocessing / threading confusion

[Dave Angel]

On 6/9/2013 14:27, Paul Pittlerson wrote:

f> Ok here is the fixed and shortened version of my script:
>
> #!/usr/bin/python
>
> from multiprocessing import Process, Queue, current_process
> from threading import Thread
> from time import sleep
>
> class Worker():
>     def __init__(self, Que):
>         self._pid = current_process().pid        
>         self.que = Que
>         self.que.put('started worker %s' % self._pid)
>         
>         for n in range(5):
>             self.que.put('%s tick %d' % (self._pid, n))
>             # do some work
>             sleep(0.01)
>             
>         self.que.put('%s has exited' % self._pid) 
>         
> class Debugger(Thread):
>     def __init__(self, q):
>         super(Debugger, self).__init__()
>         self.q = q
>         
>     def run(self):
>         while True:
>             
>             sleep(0.1)
>             
>             if not self.q.empty():
>                 print self.q.get()
>                 
>             else:
>                 break
>         #
>
> if __name__ == '__main__':
>     
>     debug_q = Queue()
>     debug = Debugger(debug_q)
>     debug.start()
>     
>     for i in range(5):
>         
>         d = Process(target=Worker, args=(debug_q,))
>         d.start()
>
> This works great on linux, but does not run on windows (7). The behavior was: I 
> opened it with double clicking and so a window appeared and disappeared (no 
> text) then I opened it with IDLE and ran it there, where it worked a couple 
> times. Then reopened it with IDLE and this time it did not work at all. After 
> that the script did not run either through IDLE or opening directly.
>
> What may be the reason it works on linux, but seems buggy on windows?

In Linux, the Process() class works very differently.  One effect is
that it's probably much quicker than in Windows.  But also, the
relationship between the original process and the extra 5 is different.

I wouldn't even try to debug anything else till I add join() calls for
both the extra thread and the 5 extra processes.  You could just stick a
raw_input() at the end to fake it, but that's just a temporary hack.

Untested:


if __name__ == '__main__':
    
    debug_q = Queue()
    debug = Debugger(debug_q)
    debug.start()

    processes = []    
    for i in range(5):
        
        d = Process(target=Worker, args=(debug_q,))
	processes.append(d)
        d.start()
    for proc in processes:
        proc.join()
    debug.join()


As for running it in various shells, you missed the only one worth
using:  run it in the cmd shell.  When you double-click, you can't
really be sure if that temporary cmd shell was really empty before it
vanished;  it might just not have bothered to update its pixels on its
way out.  And any IDE shell like IDLE has so many other glitches in it,
that bugs like these are as likely to be the IDE's fault as anything
else.


-- 
DaveA