[Python-3000] iostack and Oh Oh

[Phillip J. Eby]

At 01:46 PM 12/1/2006 -0800, Guido van Rossum wrote:
>I'm not sure what you mean this. Are you proposing that a library
>function that might take a mapping would be rewritten as a generic
>library function with one implementation that takes a mapping? Or are
>you proposing that the library function, instead of documenting that
>it takes a mapping, documents which generic functions should be
>applied to it?
>
>Both sound like rather a big stretch from current practice.

Actually, to me the thing that's a stretch from current practice is the 
attempt to spell out in detail what a "mapping" is.

Note that in the simplest case, where you're only using getitem/setitem 
operations, you're already using generic functions, just ones that have 
syntax sugar (i.e. the [] operator).  So in that sense, you can say that 
"mapping" means "supported by operator.getitem and operator.setitem".

I'm not saying there is no such thing as "mapping", IOW, I am saying that 
"mapping" is an informal shorthand for a collection of operations.  If you 
want to be specific, refer to operations.  If you wish to be concise (but 
vague) then refer to "mapping".

To put it another way, I'm against halfway measures.  It bothers me that 
people are trying to introduce rigid interfaces, in order to address 
"quick-and-dirty" use cases.  These things are at opposite ends of the 
spectrum, IMO: if you want a quick-and-dirty type test, test on the bloody 
concrete type!  Testing some abstract interface type to soothe your OO 
conscience doesn't actually make the code any less rigid or dirty, it just 
hides the smell.  I'd rather that such code continued to obviously smell, 
rather than pretend it's as fresh as daisies because "interfaces are better".

Conversely, if you really want that bit of code to be reusable or 
extensible, then use a generic function, and the code will be in *fact* 
extensible, rather than simply pushing the compatibility problem to 
somebody else to figure out.

Example issue: library X demands a "mapping", but really only uses 
getitem.  Its code inspects interfaces to decide how to act on parameters, 
and behaves differently if it sees "a mapping".  I have an object that 
implements getitem, and I want the special behavior, so I declare that 
object "a mapping".

But now, library Y, that I also pass the same object to, suddenly starts 
behaving differently, because it sees, "ah, you're a mapping!  So I'll use 
setitem on you..."  And now I'm screwed.

Things like this used to happen to me all the time in Zope 2, which would 
introspect methods and attributes a lot so that Zope could "decide" what to 
do with an object based on what it was.  (And it was this experience that 
made me realize that inspect-and-decide is absolutely the wrong way to 
write composable code.)

Zope 3 at first replaced this attribute-inspection with interface 
inspection -- with no better result!  After all, using an interface as a 
flag is no different in essence than using a hasattr() check as a flag.  As 
I said, it only *looks* prettier.  It wasn't until interface adaptation 
arrived in Zope 3 that things actually improved, because (like generic 
functions) adaptation at least allows third-party registration.

However, for interface adaptation to work well, the interfaces need to be 
highly context-specific, or else we just end up back at the problem where 
"mapping" means different things in different contexts.  The PyProtocols 
approach to solving this was to say, "One use case = one interface", which 
eventually led me to realize that this generally amounts to having one or 
more generic functions specific to the thing you're actually trying to 
do.  It takes a lot less time to just *explicitly* add overloads to your 
code for the things you want to be able to do with an object, than to have 
to debug the stuff that just suddenly starts happening all over the place, 
as can happen with interface inspection.

Note that quick-and-dirty checks based on *concrete* types are actually 
*safer* than abstract checks or interface checks, as this generally 
prevents them from being used as mere behavior flags that can lead to 
conflicts of the type I've described.  In contrast, both duck typing (ie. 
hasattr) checks and interface checks have proven in practice (Zope 2 and 3 
respectively) to produce significant unwanted side-effects.

The only way to reduce these side effects is to allow persons other than 
the code author to decide what should happen in a specific 
context.  Adaptation and generic functions have this ability in common, but 
mere inspection (regardless of *what* is inspected) does not.


> > In other words, I just want to use my object with some operations that a
> > library provides (or requires).  An "interface" is excise: something I have
> > to mess with that doesn't directly relate to my goals for using the 
> library.
>
>I don't understand the sentence "An "interface" is excise". What does
>excise mean in this context?

It's an HCI buzzword meaning "work that doesn't obviously advance your 
goal, but that you have to do anyway because of the way the system was 
designed/implemented".


>Regardless, I find it quite a big change from various ways of saying
>"this object must have these methods (including perhaps some for which
>special syntax exists, like __getattr__)" to saying "this object must
>be supported by these generic functions".

Well, that's why I proposed last week that we allow you to say 
ISomething(foo).somemethod() to be able to use such things.  And, that you 
be allowed to define your arguments as being of type ISomething, in order 
to have the correct namespace automatically apply.

That proposal doesn't stop you from sticking with duck typing.  However, if 
you *want* to be explicit and extensible and pure, it allows you to do so.

Meanwhile, I argue that inspection is inherently quick-and-dirty, just like 
duck typing.  It doesn't actually improve anything, but instead makes you 
do more work for no new benefit, while keeping the same likelihood of 
unexpected behavior.

What's more inspection is no different from generic functions in terms of 
being able to produce "spooky action at a distance".  However, at least 
generic functions have tables whose contents can be inspected to find out 
all the behaviors that might result, whereas interface inspection can 
happen anywhere!  And generic functions are actually extensible by third 
parties, whereas inspection is not.  So:

Inspection:
* No way to find "spooky" actions
* No way to modify broken behaviors without changing the code or trying to 
"trick" the inspector

Generics:
* "Spooky" actions are all in a table that can be dumped out and read
* Broken or undesirable actions can be overridden


>A method is just an
>identifier in the object's attribute namespace. A generic function is
>an object that may hve to be imported from elsewhere.

But this is also true of interfaces, regardless of how they're 
defined.  You can't simultaneously have "safe" duck typing and avoid the 
use of imports, unless you use some kind of global naming scheme, as in Java.


>I object to the suggestion that seems to be implied here that in the
>future we'll all be writing ducklib.quack(ob) instead of the much
>simpler ob.quack().

No, I'm saying that in any case where current interface proposals would do 
this:

     if implements(ob, IDuck):
         ob.quack()

I would argue that you are better off with *either of*:

     ducklib.quack(ob)

OR:

     ob.quack()

And that most code that isn't trying to be explicitly reusable would in 
fact use the second, "normal" syntax.  (In all likelihood, the code using 
ducklib.quack would only be code *in* ducklib, actually.)  GF's really only 
come into play when you want to make a library extensible or generic, like 
for pickling, pretty-printing, AST-visiting, etc. etc.  Or, if you have 
some circumstance that requires you to add custom code (like the sendmail() 
overload example for 'str').


>I really like that when I have an object in my
>hands, I don't need to import anything else in order to manipulate it,
>as long as the manipulation can be done through methods. I also don't
>think that the homonym problem that you (and CLOS) are trying to solve
>here is all that important in practice.

Homonyms aren't the problem that's being solved, it's context-specific 
extensibility and library composability.  Those are much bigger problems 
than name collisions.  For example, a major source of Zope 2's 
architectural difficulties was the direct result of using the same type 
of  inspection that's being promoted here.  (Here being the Py3K list.)  I 
think it would be wise to learn from that experience.

Yes, Zope 2 used hasattr() checks, not interface checks, but the effect is 
the same: people fudging what they provide in order to trick Zope into 
doing the right thing(s), instead of being able to just directly define the 
desired behavior, as they can do with adapters or GF's.

Now, I suppose you could look at these things as being no worse than they 
are in various other languages, and that's probably true.  On the other 
hand, you could look at how much more composable libraries are in languages 
with generic functions, and observe that as a general rule, such languages 
do not have massive, all-inclusive branded frameworks like Zope, Twisted, 
PEAK, etc.  And the reason for that, is that in GF-based languages, 
*libraries are composable on a larger scale*.  So, there isn't a need for 
single integrators to pull together any huge "all-in-one" frameworks.

Of course, packaging is also a factor: Twisted, Zope, and PEAK are 
all-inclusive in part because the historical cost of depending on other 
Python packages is high.  I created setuptools specifically to address that 
problem.

But the other major factor is integration: all-inclusive frameworks work 
around the difficulties inherent in wrapping other packages.  Many was the 
time in developing PEAK that I wanted to use some existing library (or even 
stdlib module), but couldn't because there was no way to add the necessary 
"glue" without awkward monkeypatching.  I would thus end up rolling my own 
libraries more often than I wanted to.

GF-based frameworks, on the other hand, don't seem like frameworks at 
all.  They're just libraries that can be combined with other 
libraries.  Interfaces don't give you this ability on their own, and even 
adaptation only gets you part of the way (and requires writing more code to 
do the same things).

I'd like to see a Python where this type of composability falls out 
naturally as a side effect of using the language idiomatically.  Being able 
to add overloads to existing functions makes it straightforward to either 
override behaviors or add adapters as appropriate, when gluing disparate 
libraries together.