[Python-ideas] PEP 3151 - Reworking the OS and IO exception hierarchy (again)

[M.-A. Lemburg]

Antoine Pitrou wrote:
> On Thu, 11 Nov 2010 22:03:36 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> This issue is addressed directly in the PEP by the "Compatibility
>> Strategy" section. Code which checks errno will not suddenly start
>> catching additional exceptions due to the merger of IOError and
>> OSError.
>>
>> However, I agree with you that the PEP would benefit from keeping
>> socket.error, select.error and mmap.error as separate IOError
>> subclasses. Existing code may legitimately be catching those in try
>> blocks that include other unrelated operations that may raise IOError.
> 
> Most socket I/O errors use different errnos (socket-specific errnos such
> as ECONNRESET, etc.), so merging the exception classes wouldn't change
> anything for careful code which checks the errno.
> 
> As for mmap.error, I think it's a design mistake in the first place,
> since mmap'ed files are not consistently different from normal files.
> 
> As for select.error, the way the select module itself is generally
> inconsistent in which exceptions it raises also makes me think it is a
> design or implementation bug. The fact that it doesn't even inherit
> EnvinromentError reinforces that feeling.
> 
> More fundamentally, the effort to create a detailed IO exception
> hierarchy is much less useful if we keep islands such as socket.error,
> mmap.error, since those errors won't benefit. The PEP only achieves
> maximum usefulnesss if it can leverage a single IO exception base
> class.

Why ? You can use multiple inheritance to plug the existing
class structure into a new one.

> That said, it is obvious that this PEP can raise compatibility concerns
> for uncaring code (code which is already poorly written in the first
> place). 

There's nothing wrong with code such as this:

try:
  ...
except socket.error:
  print 'Network problem'
except IOError:
  print 'File I/O problem'
except OSError:
  print 'File system problem'

You probably have a different use of try-except in mind. Those
exception handling blocks are used at various levels in an
application and the code run between try-except may well be dealing
with sockets and file I/O, but require two different sets of
problem resolution or reporting implementations.

The PEP breaks such code.

It also doesn't make a difference whether you additionally
check errno or not in the except clauses, since Python will
never branch into later except clauses if the class matches
the first found exception class.

In the above example, the except socket.error clause would
receive all errors, but it's not prepared to deal with them,
since the code was written with a different hierarchy in mind.

And indeed, "careful" code using a more coarse class hierarchy
as basis is more likely to break because of such changes than
code which works based on classes of exceptions. Simply due
to the fact that the errno value loses the additional
context information of how it came to be under the PEP
proposal.

Since this is a known fact, PEP 348 used the 2.x-3.x transition
to implement those changes. The 3.x-4.x transition would be the
next chance to apply such structural changes.

> There's no way to reorganize the exception hierarchy without
> risking such issues. Actually, even without reorganizing the exception
> hierarchy, it often happens that we casually change the exception type
> raised by a given function, because we think the old behaviour was
> mistaken. Nobody apparently blames us for doing that (perhaps they
> are not reading python-checkins carefully enough :-)).

Such changes need to be carefully documented in the NEWS file.

They may well introduce DoS security problems in applications,
e.g. by causing an application to crash with a traceback due
to a try-except not triggering because of the change.

So again, +1 on adding the new exceptions and integrating
them with the existing IOError subclasses. -1 on removing those
subclasses.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/