multiprocessing, pool and process crashes

[Jesse Noller]

On Wed, Apr 29, 2009 at 10:50 AM,  <dragonk at gmail.com> wrote:
> I want to use the multiprocessing.Pool object to run multiple tasks in
> separate processes.
>
> The problem is that I want to call an external C function (from a
> shared library, with help from ctypes) and this function tends to
> crash (SIGSEGV,SIGFPE,etc.) on certain data sets (the purpose of this
> thing is to test the behavior of the function and see when it
> crashes).
>
> My question is if the Pool object is able to spawn a new process in
> place of the crashed one and resume operation.  I killed with SIGSEGV
> the subprocesses and the script locked up :(
>
> Any ideas/alternatives?
>

You're going to want to use a custom pool, not the built in pool. In
your custom pool, you'll need to capture the signals/errors you want
and handle them accordingly. The built in pool was not designed for
your use case.

-jesse