[Python-bugs-list] Request - EPIPE Handling Proposal (PR#199)

[aa8vb@yahoo.com]

Full_Name: Randall Hopper
Version: 1.5.2
OS: IRIX 6.5
Submission from: ralph.rtpnc.epa.gov (134.67.66.44)


     The gist of the problem is that any Python program which writes to 
stdout or stderr needs to handle EPIPE to avoid tripping Python error
tracebacks, and doing so is non-trivial and obfuscates otherwise-simple
Python code.

Basically, this is buggy Python code:
    > python -c "for i in xrange(100000): print 'Lots of output'" | head

    Traceback (innermost last):                                                

      File "<string>", line 1, in ?                                            

    IOError: [Errno 32] Broken pipe    

The attached script demonstrates the difficulty of error catching and 
handling with EPIPE.  Run as:

    broken_pipe.py | head

==============================================================================

PROPOSAL:

    Add an "epipe_error" attribute to the file object.  It would determine
    whether EPIPE errors from read()/write() calls generate SystemExit
    or IOError exceptions.

==============================================================================


By default EPIPEs received from the stdin/out/err files would generate
SystemExit exceptions (epipe_error = 0).  All other files would default to
IOError exceptions (epipe_error = 1).  Of course, these settings can be
changed by the developer.

See the c.l.python thread "Python & EPIPE: Handling Chg Proposal" for a 
more discussion on this topic.  

FWIW, most C utilities don't have this problem since they don't ignore 
SIGPIPE and follow the default behavior of exiting when they receive it.
The proposed change would allow Python to behave similarly by default for 
stdin/stdout/stderr broken pipes, but leave the Python developer in full 
control of EPIPE handling, for all files.

Thanks,

Randall

========= Attachment 1 =======================================================
#!/usr/bin/env python
"""
------------------------------------------------------------------------------
   broken_pipe.py  -  Demonstrate how EPIPE handling really convolutes
                      Python code.  (R. Hopper, 2/2000)
 
    The jist of it is this:
       python -c "for i in xrange(100000): print 'Lots of output'" | more
    This is buggy Python code.

    Why?  Because any normal write() connected to stdout can generate
      EPIPE, which Python turns into an IOError exception, which
      materializes as a Python exception dump and error message on
      termination.
 
    Substitute 'more' with your favorite pager ('head', etc.).  Quit
    the pager after the first screenful appears to make the program fail.
 
    What about using xrange(10000), or xrange(100)?  Depends on the OS
      buffer sizes.

    So with the current EPIPE handling in Python , the only correct
      solution is for every program that writes to stdout (or stderr) to
      handle IOError exceptions for EPIPE explicitly.
 
    However, special-handling EPIPE IOErrors specifically really convolutes
      otherwise simple Python code:
------------------------------------------------------------------------------
"""

import sys, string, errno

def write_a_rec():
  print '<A record from the database>'

def write_recs():

  for i in xrange(100000):

    #--------------------------------------------------------------------------
    # In write_a_rec(), we could do some things that may generate different
    # exceptions.  Among them, IOErrors, and among those EPIPE errors
    # specifically.  Special-handling EPIPE is a real pain.
    #--------------------------------------------------------------------------

    # So we can't just call this method.  Unprotected stdout/err write()s
    #   to stdout and stderr can produce errors.

    ## write_a_rec()

    # We have to special-handle EPIPE IOErrors from write()s to stdout/err, but
    #   we want to preserve all the other exceptions.
    #
    # To avoid duplicating exception cleanup code, we use a nested try.
    #   This would be unnecessary if we could snag just EPIPE IOErrors with
    #   an except clause.
    try:
      try:
        write_a_rec()
      except IOError, x:
        if x.errno == errno.EPIPE:
          sys.stderr.write( "\nHandling EPIPE...exiting quietly.\n"
                            "  (Normally, nothing would be printed.)"
                            "\n\n" )
          sys.exit(1)
        else:
          raise
    except:
      # Do <NEEDED CLEANUP BLOCK A>
      raise
      

#
# MAIN
#
write_recs()


"""
------------------------------------------------------------------------------

Possible Solutions:

     1) Have Python not print a traceback and error if EPIPE IOErrors
        propagate up to the top stack level (what if this is a socket though;
        then you want the error).  Bad idea.
   
     2) Throw a separate exception for EPIPE (IOErrorEPIPE?) so you don't have
        to do double-try's to process them. (No, breaks existing code, and
        still demands EPIPE handling for all programs that write more than
        a few lines to stdout/stderr.

=>   3) Have a file descriptor attribute (such as "epipe_error") that
        selectively sets whether receiving EPIPE is turned into an IOError
        or a SystemExit if it propagates up to the top stack level.

        In other words, chose per file descriptor, whether EPIPE should
        be a hard error or not.  stdout/stderr's default could be "no",
        while for all other files, the default would be "yes".

        Alternatively, the flag could designate whether SystemExit or
        IOError is thrown to start with.

------------------------------------------------------------------------------