[Python-3000] Third-party annotation libraries vs the stdlib

[Collin Winter]

In working on the annotations PEP, I've run into more issues
concerning the balance of responsibility between third-party libraries
and the stdlib.

So far, the trend has been to push responsibility for manipulating and
interpreting annotations into libraries, keeping core Python free from
any built-in semantics for the annotation expressions. However, nearly
all the issues that have been discussed on this list go against the
flow: the proposed Function() and Generator() classes, used for
expressing higher-order functions and generator functions,
respectively; type operations, like "T1 & T2" or "T1 | T2"; and the
type parameterisation mechanism.

Shipping any of these things with Python raises a number of other
issues/questions that would need to be dealt with:

1. If Function() and Generator() ship in the stdlib, where do they go?
In types? In a new module?

Also, if Function() and Generator() come with Python, how do we make
sure that third-party libraries can use them with minimal extra
overhead (e.g., wrapping layers to make the shipped Function() and
Generator() objects compatible with the library's internal
architecture)?

2. If "T1 & T2" is possible with core Python (ie, no external
libraries), what does "type(T1 & T2)" return? Is "type(T1 & T2)" the
same as "type(T1 | T2)"?

What can you do with these objects in core Python? Can you subclass
from "T1 & T2"? Does "issubclass(T1, T1 | T2)" return True? What about
"isinstance(5, int | dict)"?

Are "T1 & T2" and "T1 | T2" the only defined operations? What about xor or not?

3. Similar questions are raised by having the "T1[x, y, z]"
parameterisation method present in core Python: what is the type of
"tuple[int, int]"? What can you do with it? Does "isinstance((5, 6,
7), tuple[int, int, int])" return True? Do they have the same & and |
operations as other built-in types? What happens when you mix
parameterised types and non-parameterised types, e.g., "tuple[int,
(int, int)]"?


Based on the complexity involved in specifying all of these issues, I
say we punt: let the third-party libraries handle this. Addressing the
above issues from this perspective:

1. Shipping Function() and Generator() objects is a (relative) piece of cake.

2. In my own experience with this kind of stuff, there's very little
need to express and-ing and or-ing of type expressions. Third-party
libraries can provide this on their own via And() and Or()
classes/functions/whatevers.

If some particular library absolutely insists on using the & and |
operators, there might be some metaclass wizardry that could
accomplish this, but I'm not saying I know what it is.

3. The questions raised by the special type parameterisation mechanism
can be removed by simply omitting the mechanism. In particular, using
regular tuples/lists/dicts/etc instead of the tuple[]/list[]/dict[]
spelling completely removes the issue of mixing parameterised and
non-parameterised expressions.

To sum up: I propose that -- to combat these issues -- I limit the PEP
to discussing how to supply annotations (the annotation syntax and C
API) and how to read them back later (via __signature__).

Collin Winter