interrupted system call w/ Queue.get

[Philip Winston]

On Feb 18, 10:23 am, Jean-Paul Calderone
<calderone.jeanp... at gmail.com> wrote:
> The exception is caused by a syscall returning EINTR.  A syscall will
> return EINTR when a signal arrives and interrupts whatever that
> syscall
> was trying to do.  Typically a signal won't interrupt the syscall
> unless you've installed a signal handler for that signal.  However,
> you can avoid the interruption by using `signal.siginterrupt` to
> disable interruption on that signal after you've installed the
> handler.
>
> As for the other questions - I don't know, it depends how and why it
> happens, and whether it prevents your application from working
> properly.

We did not try "signal.siginterrupt" because we were not installing
any signals, perhaps some library code is doing it without us knowing
about it.  Plus I still don't know what signal was causing the
problem.

Instead based on Dan Stromberg's reply (http://code.activestate.com/
lists/python-list/595310/) I wrote a drop-in replacement for Queue
called RetryQueue which fixes the problem for us:

from multiprocessing.queues import Queue
import errno

def retry_on_eintr(function, *args, **kw):
    while True:
        try:
            return function(*args, **kw)
        except IOError, e:
            if e.errno == errno.EINTR:
                continue
            else:
                raise

class RetryQueue(Queue):
    """Queue which will retry if interrupted with EINTR."""
    def get(self, block=True, timeout=None):
        return retry_on_eintr(Queue.get, self, block, timeout)

As to whether this is a bug or just our own malignant signal-related
settings I'm not sure. Certainly it's not desirable to have your
blocking waits interrupted. I did see several EINTR issues in Python
but none obviously about Queue exactly:
http://bugs.python.org/issue1068268
http://bugs.python.org/issue1628205
http://bugs.python.org/issue10956

-Philip