multiprocessing signal defect

Adam Skutt askutt at gmail.com
Sat Oct 30 07:34:59 EDT 2010


On Oct 29, 10:08 am, Adam Tauno Williams <awill... at whitemice.org>
wrote:
> No, I don't think so.  You're asking the module to over generalize
> behavior.  Reaping of the child is important, and that the child needs
> to be reaped may matter to the master child (why? did something go
> wrong?).
And such information is made available when you reap the child
process.  There are a ton of ways for the module to provide sane and
proper default behavior and still provide hooks for people who need to
modify the behavior or retrieve information.  However, I can't imagine
a use case where the end-developer cares about the process' exit
status from a multiprocess.Pool.

> Silently reaping them [which would reduce the size of the
> Pool? Or would it dynamically create a new worker?] might have
> unintended side effects.
Such as?  Personally, even if you do enumerate one, I'm not sure how
much it matters, as an unintended side effect has already occurred.
It's too late to avoid side effects, they've already happened.
Moreover, merely blithely continuing on invites all sorts of fun side
effects, like the lovely "undefined behavior" side effect.

> How would a 'traditional' thread pool behave if a thread abended?  [of
> course, that depends on the thread-pool implementation]  The correct
> behavior in case of an exception in a thread is a topic of some debate.
Yes, but we're not talking about an exception in a thread, we're
talking about a thread being terminated by the OS.  Which should be
treated like every other OS termination: as a fatal error.  There's
not way for the multiprocessing module to fully recover from such an
event to a known state, so the only sensible thing left is
termination.  Anything else runs the risk of undefined behavior.

Adam



More information about the Python-list mailing list