[Python-3000] New section for PEP 3124

[Phillip J. Eby]

Taking the recent threads here, and Guido's comments off-list, I've 
attempted to put together a coherent response as a new section for 
the PEP, which I've checked in and included a copy of here.  If I 
have misrepresented anyone's argument, or if you spot something where 
you have a question or need a clarification, please let me know.  Thanks.


Overloading Usage Patterns
==========================

In discussion on the Python-3000 list, the proposed feature of allowing
arbitrary functions to be overloaded has been somewhat controversial,
with some people expressing concern that this would make programs more
difficult to understand.

The general thrust of this argument is that one cannot rely on what a
function does, if it can be changed from anywhere in the program at any
time.  Even though in principle this can already happen through
monkeypatching or code substitution, it is considered poor practice to
do so.

However, providing support for overloading any function (or so the
argument goes), is implicitly blessing such changes as being an
acceptable practice.

This argument appears to make sense in theory, but it is almost entirely
mooted in practice for two reasons.

First, people are generally not perverse, defining a function to do one
thing in one place, and then summarily defining it to do the opposite
somewhere else!  The principal reasons to extend the behavior of a
function that has *not* been specifically made generic are to:

* Add special cases not contemplated by the original function's author,
   such as support for additional types.

* Be notified of an action in order to cause some related operation to
   be performed, either before the original operation is performed,
   after it, or both.  This can include general-purpose operations like
   adding logging, timing, or tracing, as well as application-specific
   behavior.

None of these reasons for adding overloads imply any change to the
intended default or overall behavior of the existing function, however.
Just as a base class method may be overridden by a subclass for these
same two reasons, so too may a function be overloaded to provide for
such enhancements.

In other words, universal overloading does not equal *arbitrary*
overloading, in the sense that we need not expect people to randomly
redefine the behavior of existing functions in illogical or
unpredictable ways.  If they did so, it would be no less of a bad
practice than any other way of writing illogical or unpredictable code!

However, to distinguish bad practice from good, it is perhaps necessary
to clarify further what good practice for defining overloads *is*.  And
that brings us to the second reason why generic functions do not
necessarily make programs harder to understand: overloading patterns in
actual programs tend to follow very predictable patterns.  (Both in
Python and in languages that have no *non*-generic functions.)

If a module is defining a new generic operation, it will usually also
define any required overloads for existing types in the same place.
Likewise, if a module is defining a new type, then it will usually
define overloads there for any generic functions that it knows or cares
about.

As a result, the vast majority of overloads can be found adjacent to
either the function being overloaded, or to a newly-defined type for
which the overload is adding support.  Thus, overloads are highly-
discoverable in the common case, as you are either looking at the
function or the type, or both.

It is only in rather infrequent cases that one will have overloads in a
module that contains neither the function nor the type(s) for which the
overload is added.  This would be the case if, say, a third-party
created a bridge of support between one library's types and another
library's generic function(s).  In such a case, however, best practice
suggests prominently advertising this, especially by way of the module
name.

For example, PyProtocols defines such bridge support for working with
Zope interfaces and legacy Twisted interfaces, using modules called
``protocols.twisted_support`` and ``protocols.zope_support``.  (These
bridges are done with interface adapters, rather than generic functions,
but the basic principle is the same.)

In short, understanding programs in the presence of universal
overloading need not be any more difficult, given that the vast majority
of overloads will either be adjacent to a function, or the definition of
a type that is passed to that function.

And, in the absence of incompetence or deliberate intention to be
obscure, the few overloads that are not adjacent to the relevant type(s)
or function(s), will generally not need to be understood or known about
outside the scope where those overloads are defined.  (Except in the
"support modules" case, where best practice suggests naming them
accordingly.)