Python & EPIPE: Handling Chg Proposal

[Randall Hopper]

     The gist of the problem is that any program which writes to stdout or
stderr really needs to handle EPIPE to avoid tripping python error
tracebacks, and doing so is non-trivial and obfuscates otherwise-simple
Python code.

Basically, this is buggy Python code:
    python -c "for i in xrange(100000): print 'Lots of output'" | head

The attached script demonstrates EPIPE handling in detail.  Run as:

    broken_pipe.py | head

==============================================================================

PROPOSAL:

    Add an "epipe_error" attribute to the file object.  It would determine
    whether EPIPE errors from read()/write() calls generate SystemExit
    or IOError exceptions.

==============================================================================


By default EPIPEs received from the stdin/out/err files would generate
SystemExit exceptions (epipe_error = 0).  All other files would default to
IOError exceptions (epipe_error = 1).

Comments?

-- 
Randall Hopper
aa8vb at yahoo.com
-------------- next part --------------
#!/usr/bin/env python
"""
------------------------------------------------------------------------------
   broken_pipe.py  -  Demonstrate how EPIPE handling really convolutes
                      Python code.  (R. Hopper, 2/2000)
 
    The jist of it is this:
       python -c "for i in xrange(100000): print 'Lots of output'" | more
    This is buggy Python code.

    Why?  Because any normal write() connected to stdout can generate
      EPIPE, which Python turns into an IOError exception, which
      materializes as a Python exception dump and error message on
      termination.
 
    Substitute 'more' with your favorite pager ('head', etc.).  Quit
    the pager after the first screenful appears to make the program fail.
 
    What about using xrange(10000), or xrange(100)?  Depends on the OS
      buffer sizes.

    So with the current EPIPE handling in Python , the only correct
      solution is for every program that writes to stdout (or stderr) to
      handle IOError exceptions for EPIPE explicitly.
 
    However, special-handling EPIPE IOErrors specifically really convolutes
      otherwise simple Python code:
------------------------------------------------------------------------------
"""

import sys, string, errno

def write_a_rec():
  print '<A record from the database>'

def write_recs():

  for i in xrange(100000):

    #--------------------------------------------------------------------------
    # In write_a_rec(), we could do some things that may generate different
    # exceptions.  Among them, IOErrors, and among those EPIPE errors
    # specifically.  Special-handling EPIPE is a real pain.
    #--------------------------------------------------------------------------

    # So we can't just call this method.  Unprotected stdout/err write()s
    #   to stdout and stderr can produce errors.

    ## write_a_rec()

    # We have to special-handle EPIPE IOErrors from write()s to stdout/err, but
    #   we want to preserve all the other exceptions.
    #
    # To avoid duplicating exception cleanup code, we use a nested try.
    #   This would be unnecessary if we could snag just EPIPE IOErrors with
    #   an except clause.
    try:
      try:
        write_a_rec()
      except IOError, x:
        if x.errno == errno.EPIPE:
          sys.stderr.write( "\nHandling EPIPE...exiting quietly.\n"
                            "  (Normally, nothing would be printed.)"
                            "\n\n" )
          sys.exit(1)
        else:
          raise
    except:
      # Do <NEEDED CLEANUP BLOCK A>
      raise
      

#
# MAIN
#
write_recs()


"""
------------------------------------------------------------------------------

Possible Solutions:

     1) Have Python not print a traceback and error if EPIPE IOErrors
        propagate up to the top stack level (what if this is a socket though;
        then you want the error).  Bad idea.
   
     2) Throw a separate exception for EPIPE (IOErrorEPIPE?) so you don't have
        to do double-try's to process them. (No, breaks existing code, and
        still demands EPIPE handling for all programs that write more than
        a few lines to stdout/stderr.

=>   3) Have a file descriptor attribute (such as "epipe_error") that
        selectively sets whether receiving EPIPE is turned into an IOError
        or a SystemExit if it propagates up to the top stack level.

        In other words, chose per file descriptor, whether EPIPE should
        be a hard error or not.  stdout/stderr's default could be "no",
        while for all other files, the default would be "yes".

        Alternatively, the flag could designate whether SystemExit or
        IOError is thrown to start with.

------------------------------------------------------------------------------
"""