Finding the source of an exception in a python multiprocessing program

[Dave Angel]

On 04/24/2013 08:00 PM, Oscar Benjamin wrote:
> On 25 April 2013 00:26, Dave Angel <davea at davea.name> wrote:
>> On 04/24/2013 05:09 PM, William Ray Wing wrote:
>>>
>>>
>>>    <SNIP>
>>>
>>
>> My question is why bother with multithreading?  Why not just do these as
>> separate processes?  You said "they in no way interact with each other" and
>> that's a clear clue that separate processes would be cleaner.
>
> It's using multiprocessing rather than threads: they are separate processes.
>

You're right;  I was completely off base.  brain-freeze.

>>
>>     <SNIP>
>
> It's state that is passed to it by the subprocess and should only be
> accessed by the top-level process after the subprocess completes (I
> think!).
>
>>
>> Separate processes will find it much more difficult to interact, which is a
>> good thing most of the time.  Further, they seem to be scheduled more
>> efficiently because of the GIL, though that may not make that much
>> difference when you're time-limited by network data.
>
> They are separate processes and do not share the GIL (unless I'm very
> much mistaken).

No, you're not mistaken.  Somehow I interpreted the original as saying 
multi-thread, and everything else was wrong as a result.  Now it sounds 
like a bug in, or misuse of, the Pool class.



-- 
DaveA