[Python-3000] PEP 3124 - Overloading, Generic Functions, Interfaces, etc.

[Guido van Rossum]

I refuse to continue this discussion until the PEP has been rewritten.
It's probably a much better use of your time to rewrite the PEP than
to argue with me in email too.

On 5/14/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:19 PM 5/14/2007 -0700, Guido van Rossum wrote:
> >On 5/14/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>However, with respect, I didn't go to all the trouble of implementing
> >>method combination in RuleDispatch just for the heck of it.  (And it
> >>was considerable trouble, doing it the way CLOS implements it, until
> >>I figured out an approach more suitable for Python and decorators.)
> >
> >So you owe us more motivating examples (in addition to the explanatory
> >examples), showing how you had a particular problem, and you couldn't
> >solve it cleanly using the usual suspects (subclassing, callbacks,
> >etc.), and how method combining came to the rescue. Perhaps writing it
> >up like a pattern description a la GoF might help.
>
> It's really not that complicated.  If you have only strict precedence
> (i.e., methods with the same signature are ambiguous), you wind up in
> practice needing a way to disambiguate methods when you don't really
> care what order they're executed in (because they're being registered
> independently).
>
> Before and After methods give you that escape, because they're
> assumed to be independent, and thus any number of libraries can thus
> register a before or after method for any given signature, without
> conflicting with each other.
>
> So the "particular problem" I had is simply that when you are using
> GF methods as "observer"-like hooks, you need a way to specify them
> that doesn't result in ambiguities between code that's watching the
> same thing (but is written by different people).  And, the nature of
> these observer-ish use cases is that you sometimes need
> pre-observers, and sometimes you need post-observers.
>
> (For example, a pre-observer like "block the sale if there's a hold
> on the item by a more valuable customer" or a post observer like,
> "send an email to the sales manager if this is an account we got from
> FooCorp.")
>
> Can these use cases be handled with callbacks of some other
> sort?  Sure!  But then, we can and do also get by with implementing
> ad-hoc generic functions using __special__ methods and copy_reg and
> so on.  The point of the PEP was to provide a standardized API for
> generic functions and method combination, so you don't need to
> reinvent or relearn new ways of doing it for every single Python
> library that uses something that follows these patterns.
>
> Indeed, having yet another implementation of generic functions was
> never the point of the PEP, as we already have several of them in the
> language and stdlib, plus several more third-party modules that implement them!
>
> The point, instead, was to standardize an *API* for generic
> functions, so that one need only learn that API once.  A default GF
> implementation is merely necessary for bootstrapping that API, and
> useful for "batteries included"-ness.
>
> So, if the bar is that a feature has to be unsolvable using ad hoc
> techniques, it seems the entire PEP would fail on those grounds.  We
> have plenty of ad hoc techniques for implementing GF's or quasi-GF's
> already, likewise for callbacks and the like.  The point was for you
> to Pronounce on One Obvious API (to Rule Them All).
>
>
> >>But let me try to get closer to the issue that I have.  I honestly
> >>don't see at this moment in time, how to split out most of the
> >>features you don't like (mainly before/after/around), in such a way
> >>that they can be put back in by a third-party module, without leading
> >>to other problems.  For example, I fear that certain of those
> >>features (especially before/after/around) require a single "blessed"
> >>implementation in order to have a sane/stable base for library
> >>inter-op, even if they *could* be separated out and put back
> >>in.  That is, even if it's possible to separate the "mechanism", I
> >>think that for "policy" reasons, they should have a canonical implementation.
> >
> >Please share more details, so your readers can understand this too.
> >Right now the whole discussion around this appears to be in your head
> >only, and what you write is the conclusion *you* have drawn.
>
> Actually, the discussion about method combination precedence has been
> ongoing in several threads here on Py3K, mostly with Greg Ewing and
> Jim Jewett.  These discussions illustrate why having some basic
> operators of known precedence gives the system more stability when
> multiple libraries start playing together.
>
>
> >But can you at least share enough of the problem so others can look at
> >it and either suggest a solution or agree with your conclusion?
>
> Sure.  Take a look at peak.rules.core (while keeping in mind all the
> bits that will be changed per your prior requests):
>
> http://svn.eby-sarna.com/PEAK-Rules/peak/rules/core.py?view=markup
>
> What you'll notice is that the method combination framework (Method,
> MethodList, combine_actions, always_overrides, and merge_by_default,
> if you don't count the places these things get called) is in fact
> most of the code, with relatively little of it being the actual
> implementation of Around, Before, or After (or even generic functions
> themselves!).
>
> In principle, I could pull that framework out and leave just a
> mechanism for adding it back in.  But in practice, that framework
> lays down the principles of "governance" for method combination, as
> far as how to decide what things have precedence over what.
>
> Thus, I'm skeptical of how useful it is in this area to provide
> mechanism but no policy.  It's always possible for someone to create
> their own independent policy within the mechanism -- even if there's
> a default policy.  But One Obvious Way suggests that there should be
> *some* sort of policy in place by default, just like we have a
> standard set of descriptors that implement the conventional forms of
> properties and methods.  You can subclass them or entirely replace
> them, but they cover all the typical use cases, and you can use them
> as examples to understand how to do more exotic things.
>
> Meanwhile, if we didn't have the examples of properties and methods,
> how would we know we were designing descriptor hooks correctly?  If
> we are positing that I know enough to design the hooks correctly, we
> are implicitly positing that I know what the hooks will be used and
> useful *for*.  :)  However, by making various use cases (before,
> after, around, and the custom example) explicit in the PEP, I was
> attempting to provide the motivation and rationale for the design of
> the hooks.  (Although in all fairness, the hooks are not actually
> documented in the PEP yet, aside from a listing of function names.)
>
>
> >I'm all for hooks. They can take the form of a particular factoring
> >into methods that make it easy to override some method; or using GF's
> >recursively for some of the implementation, etc.
>
> This is in fact how it works now; all the extension API functions in
> the PEP are either existing GF's in peak.rules.core, or proposed for addition.
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)