isinstance() necessary and helpful, sometimes

[Alex Martelli]

"Raymond Hettinger" <othello at javanet.com> wrote in message
news:a2qqd1$7q6$1 at bob.news.rcn.net...
> Since I've made many uses of isinstance(), Kragen Sitaker's post inspired
me
> to scan my code and review each occurance.

Very interesting and instructive: thanks!  Sometimes one is prone to make
general observations just because they apply in (say) 99% of the cases, but
thorough analysis such as yours may reveal the wider purview.

For example, one might reasonably assert "don't smoke, it's always bad
for your health".  However, right after an earthquake in Central Italy
a few years ago, one of the art experts who were examining the damage
in an Assisi church took a cigarette break and stepped out of the
church, where her colleagues were still examining the damaged frescoes
while perched up on some rigged ladders etc.  She was the only one to
survive, as another earthquake hit right then and collapsed the ladders,
killing the other experts.  If you're about to be shot dead by a firing
squad in some foreign country, and you're offered a last cigarette, you
might reasonably take it -- maybe an order for a stay of execution will
arrive just in the minute or two you gain by accepting it, after all.

So, 'always' is a tad too strong here.  Similarly, 'never' is too
strong in the admonition 'never typetest in Python', and you do a
great job of ferreting out some cases where 'never' is too strong.
I don't entirely agree with all of your specific conclusions, and
I'm going to quibble about some, but, you DO get me to admit that
SOME uses of typetesting are preferable to some alternatives, which
IS quite a feat:-).


> I found three categories:
> 1) isinstance is necessary and helpful
> 2) isinstance needed to be replaced with hasattr()
> 3) isinstance() was the least worst alternative in the absence of PEP-246.
>
> Helpful and Necessary Cases:
> ---------------------------------
>
> 1.  Since Python does not directly support multiple-dispatch like CLOS and
> Dylan, isinstance() provides a way to handle multiple argument types.  For

Even as a sworn enemy of type-testing, I have to concede this one: when
I need to synthesize multi-dispatch, I do typetest (via isinstance or
issubclass, that's quite a minor issue).  Which is part of why I try to
avoid using multi-dispatch in languages that don't support it (C++'s
"Visitor" horrors, type-testing in Python, ...) -- but sometimes it's
still the most viable architecture despite the unsighly plumbing.

> 2.  Several pieces of code used the Composite pattern and needed a way to
> differentiate between complex and atomic arguments.  The most basic and
> common case occurs in code recursing through nested lists --
> isinstance(elem, list) provided the necessary test (try solving basic LISP
> problems without a test to discriminate between atoms and lists).   A more
> complex version of this case occurs when the container is a user defined
> class rather than built-in type (sort of a Russian dolls phenomenon).

You need a way to code isatom(x), yes.  But isinstance is not the
optimal way in my experience.  Potentially more helpful:

def issequencelike(x):
    try:
        for item in x: break
    except: return 0
    else: return 1

def isstringlike(x):
    try:
        x+''
    except: return 0
    else: return 1

def isatom(x):
    return isstringlike(x) or not issequencelike(x)

These can no doubt be improved, but isinstance would not substantially
improve them -- why not let clientcode use UserStrings in lieu of
strings, UserLists in lieu of lists, and so on?


> 3.  Adding functionality with a Decorator pattern, it's easy to end up
> re-wrapping an object more than once.  To avoid multiple wraps, it's
helpful
> to use isinstance() to see if the argument is already of the right type.
In
> my code, the Table class added elementwise operations to built-in lists.
> Everywhere I needed a Table, the user could have easily supplied either a
> Table or a List, so used something like:   arg = isinstance(arg,Table) and
> arg or Table(arg).

If you _add_ functionality, hasattr seems just about right.  If you
_modify_ functionality, so that the Decorated object has no extra
attributes wrt the unDecorated ones, then that doesn't help -- and
if you do all your Decoration with [subclasses of] the same Decorator
class, then typetesting for that is reasonable here.

Choosing typetesting in this case may impede framework augmentation
"from the outside".  If client-code needs to provide some kind of
SuperTable, it must subclass it from Table, which may mean carrying
around unwanted baggage.  One approach (to keep typetesting) is to
make Table into a no-extra-baggage abstract class -- either a pure
interface, or an interface plus some helpful Template methods, or
a full-fledged Haskell-like typeclass, but still without possibly
unwanted data attributes.  Another possibility might be to add to
Table an attribute (_yesiamalreadydecoratedthankyouverymuch would
seem to be the natural name for it) and replace the typechecking
with a check for that attribute: this still lets client-code choose
to subclass Table, but offers client-code the extra possibility of
reimplementing its own decorators from scratch -- it just has to
supply that 'flag' attribute as well to assert it knows what it's
doing.  Admittedly, such a measure of generality may be excessive
and unwarranted, but it's a possibility to keep in mind when a
framework of this kind is evolving -- even if you strictly relied
on typetesting in version N, you can provide backwards-compatible
greated flexibility in version N+1 by switching to using a flag
attribute instead.


> 4.  Some code needs to fork paths when the class indicates an intrinsic
> quality not revealed by the attributes.  Implementing matrix
exponentation,
> __pow__(self,exponent), efficiently requires testing
> isinstance(exponent,int) because the float and complex cases are to be
> computed differently.

Wouldn't we want in this case to treat X**1.0 the same as X**1 the
same as X**1L...?  A test such as exponent==int(exponent) would
then seem to be more general and useful than isinstance(exponent,int).


> In the same package, lower triangular matrices were a sub-class of
> generalized matrices.  Occasionally, it was necessary to assert that a
> matrix was of a certain type eventhough it supported exactly the same
> methods as general matrices (for instance the Mat.LU method carries an
> assertion, isinstance(L, LowerTri)).

Admittedly a harder case.  Optimization doth make typetesters of us
all.  If hasattr does not apply, and along a bottleneck of the code
a frequent and important special case may be shunted along a different
and faster computational track by typetesting, I'll accept typetesting
as one of the many complications, obfuscations and rigidifications one
may endure in optimization's name (I would expect the general case to
be handled along another slower path, but I'll accept you had good
reasons to raise an assertion error instead in your code).


> Cases Where hasattr() is Better:
> ------------------------------------
> 1.  Supplying a default result when the required methods are not available
> for all possible argument types.  For example, .conjugate() is only
> available for complex numbers but has meaning for other numeric types.
> Defining myConj(z) as:  hasattr(z,'conjugate') and z.conjugate() or z.
> Similar definitions can be supplied for .imag and .real so that any
numeric
> type will "do the right thing" when fed to the functions.  For example:
> map( myConj, [ 3+4j, 5, 6.0, 7L ] )

A very good point, and of wide applicability.  Many Python built-ins work
conceptually along this pattern.  For example, testing X for truth checks
X for a __nonzero__ method to call, and proceeds to other cases if no such
method is available.

As a quibble, this case is often better handled by try/except rather
than if/else -- and I don't think that squashing the test into one
expression is worth it.  Take exactly the example you suggest:

def myConj(z): return hasattr(z,'conjugate') and z.conjugate() or z

this only works because, if z.conjugate() is false, then z==z.conjugate().
When generalized, e.g., to .real, it breaks down:

>>> def myReal(z): return hasattr(z,'real') and z.real or z
...
>>> print myReal(0+4j)
4j

"Oops".  An if/else or try/except doesn't run such risks:

def myReal1(z):
    if hasattr(z, 'real'):
        return z.real
    else:
        return z

or

def myReal2(z):
    try:
        return z.real
    except AttributeError:
        return z

The try/except form avoids coding the name 'real' in two places -- maybe
that is the reason it appeals so much more to me (allergy against coding
the same thing twice being even stronger in me than dislike for
typetesting:-).

More generally I don't like the structure

    if <there is a solution>:
        return <find the solution>
    else:
        return <indicator of 'no solution exists'>

and similar ones.  Determining whether there is a solution at all
is often tantamoung to finding it, after all, so getting used to
coding this way implies that one will sometimes do double work
or end up with subtle, mysterious caching schemes whereby the
<there is a solution> function squirrels away the solution for
the benefit of the following <find the solution> call.  I think
it's more straightforward to have <find the solution>'s contract
be: return the solution, or, if there is none, then raise an
appropriate "there is no solution" exception; then, code
    try:
        return <find the solution>
    except <there is no solution>:
        return <indicator of 'no solution exists'>

i.e., "easier to get forgiveness than permission" rather than the
widespread "look before you leap" approach.


> 2.   In a similar vein, I needed to make sure that an iteration interface
> was supplied even when xrange or sequence types were supplied as
arguments:
>        gens = [ type(s)==types.GeneratorType and s or iter(s) for s in
> sequences]
> Was better replaced by:
>        gens = [ hasattr(s,'next') and s or iter(s) for s in sequences]

Functions that are idempotent when applied to already-OK arguments
are preferable -- just as I wouldn't code:

        x = isinstance(x, tuple) and x or tuple(x)

even apart from issues of and/or, and typechecking, but just

        x = tuple(x)

knowing that this does return x if x is already a tuple.

Similarly, in your case,
        gens = map(iter, sequences)
should work just fine!  We don't need to worry about whether s
can be false in the and/or construct, etc, etc.

"Potentially-idempotent adapter functions are a honking great
idea -- let's do more of those", to paraphrase the timbot.  PEP 246
can be seen in that light, too: function adapt(object, protocol)
returns object if object ALREADY satisfies the protocol.


> Case Where isinstance() Was the Least Worst Alternative

Actually, I would tend to classify typetesting as a "least of evils" even
in the cases above where I've agreed that other alternatives are worse:-).


> In the following code, the goal is to accept a regular expression pattern
in
> the form of a string or in the form of an already compiled expression.

But we'd also like to accept a UserString or other string-workalike
object, without forcing the user to call us with str(pattern) in
those cases, no?  And Unicode objects?  And...?

> isinstance() gives a direct way of checking to see if a string was
supplied.

But the above-coded function isstringlike would do the job better.

However, there IS an approach that is even better, and we have
just finished talking about it: *potentially idempotent adapting
functions*!  re.compile just happens to be one of those, thanks be:

>>> a=re.compile(r'\d+')
>>> b=re.compile(a)
>>> a is b
1

So just code
    pattern = re.compile(pattern)
and live happily ever after.

PEP 246 would give a systematic approach to this, but until and unless
it's passed, we can still use the specific cases where this works.


I think this discussion is highlighting TWO key points:

a) there ARE cases where typetesting is the least of evils in Python today
    (rare and marginal ones, but, they DO exist);

b) if you think you have found one such case, look again, and reflect a lot
    about it, because it may well LOOK to you like you've found one, when in
    fact you're still better off without typetesting, through any of the
    several alternatives discussed here;

c) PEP 246 would help a lot [well, OK, THREE key points...].


Alex