[Python-ideas] Runtime types vs static types

[Lucas Wiman]

>
> For some background on the removal of __instancecheck__, check the linked
> issues here:
>

Thanks for the reference (the most relevant discussion starts here
<https://github.com/python/typing/issues/136#issuecomment-104698674>).
That said, I think I totally disagree with the underlying philosophy of
throwing away a useful and intuitive feature (having `is_instance(foo,
Union[Bar, Baz])` just work as you'd naively expect) in the name of making
sure that people *understand* there's a distinction between types and
classes.

This seems opposed to the "zen" of python that there should be exactly one
obvious way to do it, since (1) there isn't a way to do it without a third
party library, and (2) the obvious way to do it is with `isinstance` and
`issubclass`. Indeed, the current implementation makes it somewhat
nonobvious even how to implement this functionality yourself in a
third-party library (see this gist
<https://gist.github.com/lucaswiman/21373bea33ccd2c5e868ec52b6eff412>).

One of the first things I did when playing around with the `typing` module
was to fire up the REPL, and try runtime typechecks:

>>> from typing import *
>>> isinstance(0, Union[int, float])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py",
line 767, in __instancecheck__
    raise TypeError("Unions cannot be used with isinstance().")
TypeError: Unions cannot be used with isinstance().

I think the natural reaction of naive users of the library is "That's
annoying. Why? What is this library good for?", not "Ah, I've sagely
learned a valuable lesson about the subtle-and-important-though-unmentioned
distinction between types and classes!" The restriction against runtime
type checking makes `typing` pretty much *only* useful when used with the
external library `mypy` (or when writing a library with the same purpose as
`mypy`), which is a pretty unusual situation for a standard library module.

Mark Shannon's example also specifically does not apply to the types I'm
thinking of for the reasons I mentioned:

> For example,
> List[int] and List[str] and mutually incompatible types, yet
> isinstance([], List[int]) and isinstance([], List[str))
> both return true.
>
> There is no corresponding objection for `Union`; I can't think of any*
inconsistencies or runtime type changes that would result from defining
`_Union.__instancecheck__` as `any(isinstance(obj, t) for t in
self.__args__`. For `Tuple`, it's true that `()` would be an instance of
`Tuple[X, ...]` for all types X. However, the objection for the `List` case
(IIUC; extrapolating slightly) is that the type of the object could change
depending on what's added to it. That's not true for tuples since they're
immutable, so it's not *inconsistent* to say that `()` is an instance of
`Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible
definition to the base case of an empty tuple.

That said, it sounds like the decision has already been made, and this
would be quite useful functionality to have in *some* form. What do people
think about implementing `__contains__` (for `__instancecheck__`) and
`__lt__` (for `__subclasscheck__`) for these cases? Then there would still
be a convenient syntax for doing runtime type checking/analysis, but
wouldn't violate Mark Shannon's objections.

Best,
Lucas

* Counterexamples welcomed, of course!
<http://www.dictionary.com/browse/more-things-in-heaven-and-earth--horatio>


On Sun, Jun 25, 2017 at 6:21 AM, rymg19 at gmail.com <rymg19 at gmail.com> wrote:

> For some background on the removal of __instancecheck__, check the linked
> issues here:
>
>
> https://github.com/python/typing/issues/135
>
>
> --
> Ryan (ライアン)
> Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttp://refi64.com
>
> On Jun 25, 2017 at 8:11 AM, <Koos Zevenhoven <k7hoven at gmail.com>> wrote:
>
> On Sat, Jun 24, 2017 at 11:30 PM, Lucas Wiman <lucas.wiman at gmail.com>
> wrote:
>
>> ​
>> On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven <k7hoven at gmail.com>
>>  wrote:
>>
>>> There has been some discussion here and there concerning the differences
>>> between runtime types and static types (mypy etc.). What I write below is
>>> not really an idea or proposal---just a perspective, or a topic that people
>>> may want to discuss. Since the discussion on this is currently very fuzzy
>>> and scattered and not really happening either AFAICT (I've probably missed
>>> many discussions, though). Anyway, I thought I'd give it a shot:
>>>
>>>
> ​[...]​
>
>
>
>> Regarding runtime types and isinstance, let's look at the Iterable[int]
>>> example. For this case, there are a few options:
>>>
>>> 1) Don't implement isinstance
>>>
>>> This is problematic for runtime uses of annotations.
>>>
>>> 2) isinstance([1, '2', 'three'], Iterable[int]) returns True
>>>
>>> This is in fact now the case. This is ok for many runtime situations,
>>> but lacks precision compared to the static version. One may want to
>>> distinguish between Iterable[int] and Iterable[str] at runtime (e.g. the
>>> multidispatch example above).
>>>
>>> 3) Check as much as you can at runtime
>>>
>>> There could be something like Reiterable, which means the object is not
>>> consumed by iterating over it, so one could actually check if all elements
>>> are instances of int. This would be useful in some situations, but not
>>> available for every object. Furthermore, the check could take an arbitrary
>>> amount of time so it is not really suitable for things like multidispatch
>>> or some matching constructs etc., where the performance overhead of the
>>> type check is really important.
>>>
>>> 4) Do a deeper check than in (2) but trust the annotations
>>>
>>> For example, an instance of a class that has a method like
>>>
>>> def __iter__(self) -> Iterator[int]:
>>>     some code
>>>
>>> could be identified as Iterable[int] at runtime, even if it is not
>>> guaranteed that all elements are really integers.
>>>
>>> On the other hand, an object returned by
>>>
>>> def get_ints() -> Iterable[int]:
>>>     some code
>>>
>>> does not know its own annotations, so the check is difficult to do at
>>> runtime. And of course, there may not be annotations available.
>>>
>>> 5) Something else?
>>>
>>>
>>> And what about PEP544 (protocols), which is being drafted? The PEP seems
>>> to aim for having type objects that represent duck-typing
>>> protocols/interfaces. Checking whether a protocol is implemented by an
>>> object or type is clearly a useful thing to do at runtime, but it is not
>>> really clear if isinstance would be a guaranteed feature for PEP544
>>> Protocols.
>>>
>>> So one question is, is it possible to draw the lines between what works
>>> with isinstance and what doesn't, and between what details are checked by
>>> isinstance and what aren't? -- Or should insinstance be reserved for a more
>>> limited purpose, and add another check function, say `implements(...)`,
>>> which would perhaps guarantee some answer for all combinations of object
>>> and type?
>>>
>>
>>>
>> I'm guessing to implement PEP 544, many of the `__instancecheck__` and
>> `__subclasscheck__` methods in `typing.py` would need to be updated to
>> check the `__annotations__` of the class of the object it's passed against
>> its own definition, (covered in this section
>> <https://www.python.org/dev/peps/pep-0544/#runtime-decorator-and-narrowing-types-by-isinstance>
>> of the PEP).
>>
>>
> ​I may have missed something, but I believe PEP544 is ​not suggesting that
> annotations would have any effect on isinstance. Instead, isinstance would
> by default not work.
>
>
>
>> I've been somewhat surprised that many of the `__instancecheck__`
>> implementations do not work at runtime, even when the implementation would
>> be trivial (e.g. for `Union`), or would not have subtle edge cases due to
>> immutability (e.g. for `Tuple`, which cannot be used for checking
>> parameterized instances). This seems like counterintuitive behavior that
>> would be straightforward to fix, unless there are subtleties & edge cases
>> I'm missing.
>>
>>
> ​Tuple is an interesting case, because for small tuples (say 2- or
> 3-tuples), it makes perfect sense to check the types of all elements for
> some runtime purposes.​ Regarding Union, I believe the current situation
> has a lot to do with the fact that the relation between type annotations
> and runtime behavior hasn't really settled yet.
>
>
> If people are amenable to updating those cases, I'd be interested in
>> submitting a patch to that effect.
>>
>>
> ​Thanks for letting us know. (There may not be an instant decision on this
> particular case, though, but who knows :)
>
> -- Koos
>
>
> --
> + Koos Zevenhoven + http://twitter.com/k7hoven +
> _______________________________________________ Python-ideas mailing list
> Python-ideas at python.org https://mail.python.org/
> mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/
> codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170625/b853c9d6/attachment-0001.html>