[Python-Dev] PEP 246, redux
Phillip J. Eby
pje at telecommunity.com
Mon Jan 10 18:43:44 CET 2005
At 03:42 PM 1/10/05 +0100, Alex Martelli wrote:
> The fourth case above is subtle. A break of substitutability can
> occur when a subclass changes a method's signature, or restricts
> the domains accepted for a method's argument ("co-variance" on
> arguments types), or extends the co-domain to include return
> values which the base class may never produce ("contra-variance"
> on return types). While compliance based on class inheritance
> _should_ be automatic, this proposal allows an object to signal
> that it is not compliant with a base class protocol.
-1 if this introduces a performance penalty to a wide range of adaptations
(i.e. those using abstract base classes), just to support people who want
to create deliberate Liskov violations. I personally don't think that we
should pander to Liskov violators, especially since Guido seems to be
saying that there will be some kind of interface objects available in
future Pythons.
> Just like any other special method in today's Python, __conform__
> is meant to be taken from the object's class, not from the object
> itself (for all objects, except instances of "classic classes" as
> long as we must still support the latter). This enables a
> possible 'tp_conform' slot to be added to Python's type objects in
> the future, if desired.
One note here: Zope and PEAK sometimes use interfaces that a function or
module may implement. PyProtocols' implementation does this by adding a
__conform__ object to the function's dictionary so that the function can
conform to a particular signature. If and when __conform__ becomes
tp_conform, this may not be necessary any more, at least for functions,
because there will probably be some way for an interface to tell if the
function at least conforms to the appropriate signature. But for modules
this will still be an issue.
I am not saying we shouldn't have a tp_conform; just suggesting that it may
be appropriate for functions and modules (as well as classic classes) to
have their tp_conform delegate back to self.__dict__['__conform__'] instead
of a null implementation.
> The object may return itself as the result of __conform__ to
> indicate compliance. Alternatively, the object also has the
> option of returning a wrapper object compliant with the protocol.
> If the object knows it is not compliant although it belongs to a
> type which is a subclass of the protocol, then __conform__ should
> raise a LiskovViolation exception (a subclass of AdaptationError).
> Finally, if the object cannot determine its compliance, it should
> return None to enable the remaining mechanisms. If __conform__
> raises any other exception, "adapt" just propagates it.
>
> To enable the third case, when the protocol knows about the
> object, the protocol must have an __adapt__() method. This
> optional method takes two arguments:
>
> - `self', the protocol requested
>
> - `obj', the object being adapted
>
> If the protocol finds the object to be compliant, it can return
> obj directly. Alternatively, the method may return a wrapper
> compliant with the protocol. If the protocol knows the object is
> not compliant although it belongs to a type which is a subclass of
> the protocol, then __adapt__ should raise a LiskovViolation
> exception (a subclass of AdaptationError). Finally, when
> compliance cannot be determined, this method should return None to
> enable the remaining mechanisms. If __adapt__ raises any other
> exception, "adapt" just propagates it.
> The fourth case, when the object's class is a sub-class of the
> protocol, is handled by the built-in adapt() function. Under
> normal circumstances, if "isinstance(object, protocol)" then
> adapt() returns the object directly. However, if the object is
> not substitutable, either the __conform__() or __adapt__()
> methods, as above mentioned, may raise an LiskovViolation (a
> subclass of AdaptationError) to prevent this default behavior.
I don't see the benefit of LiskovViolation, or of doing the exact type
check vs. the loose check. What is the use case for these? Is it to allow
subclasses to say, "Hey I'm not my superclass?" It's also a bit confusing
to say that if the routines "raise any other exceptions" they're
propagated. Are you saying that LiskovViolation is *not* propagated?
> If none of the first four mechanisms worked, as a last-ditch
> attempt, 'adapt' falls back to checking a registry of adapter
> factories, indexed by the protocol and the type of `obj', to meet
> the fifth case. Adapter factories may be dynamically registered
> and removed from that registry to provide "third party adaptation"
> of objects and protocols that have no knowledge of each other, in
> a way that is not invasive to either the object or the protocols.
This should either be fleshed out to a concrete proposal, or
dropped. There are many details that would need to be answered, such as
whether "type" includes subtypes and whether it really means type or
__class__. (Note that isinstance() now uses __class__, allowing proxy
objects to lie about their class; the adaptation system should support this
too, and both the Zope and PyProtocols interface systems and PyProtocols'
generic functions support it.)
One other issue: it's not possible to have standalone interoperable PEP 246
implementations using a registry, unless there's a standardized place to
put it, and a specification for how it gets there. Otherwise, if someone
is using both say Zope and PEAK in the same application, they would have to
take care to register adaptations in both places. This is actually a
pretty minor issue since in practice both frameworks' interfaces handle
adaptation, so there is no *need* for this extra registry in such cases.
> Adaptation is NOT "casting". When object X itself does not
> conform to protocol Y, adapting X to Y means using some kind of
> wrapper object Z, which holds a reference to X, and implements
> whatever operation Y requires, mostly by delegating to X in
> appropriate ways. For example, if X is a string and Y is 'file',
> the proper way to adapt X to Y is to make a StringIO(X), *NOT* to
> call file(X) [which would try to open a file named by X].
>
> Numeric types and protocols may need to be an exception to this
> "adaptation is not casting" mantra, however.
The issue isn't that adaptation isn't casting; why would casting a string
to a file mean that you should open that filename? I don't think that
"adaptation isn't casting" is enough to explain appropriate use of
adaptation. For example, I think it's quite valid to adapt a filename to a
*factory* for opening files, or a string to a "file designator". However,
it doesn't make any sense (to me at least) to adapt from a file designator
to a file, which IMO is the reason it's wrong to adapt from a string to a
file in the way you suggest. However, casting doesn't come into it
anywhere that I can see.
If I were going to say anything about that case, I'd say that adaptation
should not be "lossy"; adapting from a designator to a file loses
information like what mode the file should be opened in. (Similarly, I
don't see adapting from float to int; if you want a cast to int, cast
it.) Or to put it another way, adaptability should imply substitutability:
a string may be used as a filename, a filename may be used to designate a
file. But a filename cannot be used as a file; that makes no sense.
>Reference Implementation and Test Cases
>
> The following reference implementation does not deal with classic
> classes: it consider only new-style classes. If classic classes
> need to be supported, the additions should be pretty clear, though
> a bit messy (x.__class__ vs type(x), getting boundmethods directly
> from the object rather than from the type, and so on).
Please base a reference implementation off of either Zope or PyProtocols'
field-tested implementations which deal correctly with __class__ vs.
type(), and can detect whether they're calling a __conform__ or __adapt__
at the wrong metaclass level, etc. Then, if there is a reasonable use case
for LiskovViolation and the new type checking rules that justifies adding
them, let's do so.
> Transitivity of adaptation is in fact somewhat controversial, as
> is the relationship (if any) between adaptation and inheritance.
The issue is simply this: what is substitutability? If you say that
interface B is substitutable for A, and C is substitutable for B, then C
*must* be substitutable for A, or we have inadequately defined
"substitutability".
If adaptation is intended to denote substitutability, then there can be
absolutely no question that it is transitive, or else it is not possible to
have any meaning for interface inheritance!
Thus, the controversies are: 1) whether adaptation should be required to
indicate substitutability (and I think that your own presentation of the
string->file example supports this), and 2) whether the adaptation system
should automatically provide an A when provided with a C. Existing
implementations of interfaces for Python all do this where interface C is a
subclass of A. However, they differ as to whether *all* adaptation should
indicate substitutability. The Zope and Twisted designers believe that
adaptation should not be required to imply substitutability, and that only
interface and implementation inheritance imply
substitutability. (Although, as you point out, the latter is not always
the case.)
PyProtocols OTOH believes that *all* adaptation must imply
substitutability; non-substitutable adaptation or inheritance is a design
error: "adaptation abuse", if you will. So, in the PyProtocols view, it
would never make sense to define an adaptation from float or decimal to
integer that would permit loss of precision. If you did define such an
adaptation, it must refuse to adapt a float or decimal with a fractional
part, since the number would no longer be substitutable if data loss occurred.
Of course, this is a separate issue from automatic transitive adaptation,
in the sense that even if you agree that adaptation must imply
substitutability, you can still disagree as to whether automatically
locating a multi-step adaptation is desirable enough to be worth
implementing. However, if substitutability is guaranteed, then such
multi-step adaptation cannot result in anything "controversial" occurring.
> The latter would not be controversial if we knew that inheritance
> always implies Liskov substitutability, which, unfortunately we
> don't. If some special form, such as the interfaces proposed in
> [4], could indeed ensure Liskov substitutability, then for that
> kind of inheritance, only, we could perhaps assert that if X
> conforms to Y and Y inherits from Z then X conforms to Z... but
> only if substitutability was taken in a very strong sense to
> include semantics and pragmatics, which seems doubtful.
As a practical matter, all of the existing interface systems (Zope,
PyProtocols, and even the defunct Twisted implementation) treat interface
inheritance as guaranteeing substitutability for the base interface, and do
so transitively.
However, it seems to me to be a common programming error among people new
to interfaces to inherit from an interface when they intend to *require*
the base interface's functionality, rather than *offer* the base
interface's functionality. It may be worthwhile to address this issue in
the design of "standard" interfaces for Python.
This educational issue regarding substitutability is I believe inherent to
the concept of interfaces, however, and does not go away simply by making
non-inheritance adaptation non-transitive in the implementation. It may,
however, make it take longer for people to encounter the issue, thereby
slowing their learning process. ;)
>Backwards Compatibility
>
> There should be no problem with backwards compatibility unless
> someone had used the special names __conform__ or __adapt__ in
> other ways, but this seems unlikely, and, in any case, user code
> should never use special names for non-standard purposes.
Production implementations of the old version of PEP 246 exist, so the
changes in semantics you've proposed may introduce backward compatibility
issues. More specifically, some field code may not work correctly with
your proposed reference implementation, in the sense that code that worked
with Zope or PyProtocols before, may not work with the reference
implementation's adapt(), resulting in failure of adaptation where success
occurred before, or in exceptions raised where no exception was raised before.
More information about the Python-Dev
mailing list