[Python-Dev] Signal-resistant code (was: Two random and nearly unrelated ideas)

[Guido van Rossum]

> > What if the operation is a select() call?  Is restarting the right
> > thing?  How to take into account the consumed portion of the timeout,
> > if given?
> 
> Some versions of select update the timeout structure to the remainder if 
> they are interrupted by a signal. It's probably not a good idea to rely 
> on this so gettimeofday could be used to calculate the remainder.

I like Neil's suggestion: simply return.  The timeout is a hint.

> > Or is this a stdio problem?  I believe that calls like fgets() and
> > getchar() don't lose data, but maybe I misunderstand your observation.
> 
> This is not the point - even if Python I/O calls were fully restartable
> would you actually expect people to check for EINTR and restart for
> *every* I/O operation in the program just in case some module happens to
> use signals?
> 
> Instead of
> 
>     for line in file:
>         do_something_with(line)
> 
> we would need to write
> 
>     while 1:
>         try:
>             line = file.next()
>         except IOError, exc:
>             if exc.errno == errno.EINTR:
>                 continue
>             else:
>                 raise
>         except StopIteration:
>             break
>         do_something_with(line)

OK, but you're changing your tune here.  I agree that this is bad, but
I still don't believe (or understand) your previous remark about
readline losing track of buffering.  But let's forget about this, I
trust that you really meant what you showed here.

> > As I said before, I'm very skeptical that making the I/O ops
> > EINTR-safe would be enough to allow the use of signals as
> > suggested by Skip
> 
> If it's good enough for other purposes it should be good enough for
> Skip's proposal, too.

Well, it has to be *perfect* for Skip's proposal, since it means we'd
be generating signals probably at a rate of 100 per second.

> > Skip, but that might still be useful for other purposes, *if* we can
> > decide when to honor EINTR and when not.
> 
> Only low-level functions like os.read and os.write that map directly
> to stdio functions should ever return EINTR.

Um, os.read/write are the ones that *don't* map to stdio.  Maybe you
meant "that map directly to file descriptors"?  But I doubt this would
be acceptable -- if we were generating 100 signals per second,
os.read/write become much harder to use if they could raise EINTR
(currently they only raise EINTR if the app uses signal handlers,
which isn't that common).

> To make Python signal-safe all other calls that can return EINTR
> should have a retry loop. On EINTR they should check if there are
> things to do and if so grab the GIL, make pending calls, release the
> GIL and retry the operation (unless an exception has been raised by
> the signal handler, of course).
> 
> This way I could finally write a Python daemon that reloads its 
> configuration files on getting the customary SIGHUP :-)

If you really want that, maybe you could see if you can produce a
working design and patch?  Even if it's not perfect enough to use
signals to replace the ticker, people who like to use signals would
probably be happy.

--Guido van Rossum (home page: http://www.python.org/~guido/)