[Python-ideas] Conventions for function annotations

[Guido van Rossum]

On Tue, Dec 4, 2012 at 10:12 AM, Masklinn <masklinn at masklinn.net> wrote:
> On 2012-12-03, at 18:27 , Guido van Rossum wrote:
>>
>> Obviously this would require inventing and standardizing notations for
>> things like "list of X", "tuple with items X, Y, Z", "either X or Y",
>> and so on, as well as a standard way of combining annotations intended
>> for different tools.
>
> I've always felt that __getitem__ and __or__/__ror__ on type 1. looked
> rather good and 2. looked similar to informal type specs and type specs
> of other languages. Although that's the issue with annotations being
> Python syntax: it requires changing stuff fairly deep into Python to
> be able to experiment.

So, instead of using

def foo(a: int, b: str) -> float:
  <blah>

you use

from experimental_type_annotations import Int, Str, Float

def foo(a: Int, b: Str) -> Float:
  <blah>

And now we're ready for experimentation.

[Warning: none of this is particularly new; I've had these things in
my brain for years, as the referenced Artima blog post made clear.]

> The most bothersome part is that I "feel" "either X or Y" (aka `X | Y`)
> should be a set of type (and thus the same as {X, Y}[0]) but that doesn't
> work with `isinstance` or `issubclass`. Likewise, `(a, b, c)` in an
> annotation feels like it should mean the same as `tuple[a, b, c]` ("a
> tuple with 3 items of types resp. a, b and c") but that's at odds with
> the same type-checking functions.

Note that in Python 3 you can override isinstance, by defining
__instancecheck__ in the class:
http://docs.python.org/3/reference/datamodel.html?highlight=__instancecheck__#class.__instancecheck__

So it shouldn't be a problem to make isinstance(42, Int) work.

We can also make things like List[Int] and Dict[Str, Float] work, and
even rig it so that

  isinstance([1, 2, 3], List[Int]) == True

while

  isinstance([1, 2, 'booh'], List[Int]) == False

Of course there are many bikeshedding topics like whether we should
ever write List -- maybe we should write Iterable or Sequence instead,
and maybe we have to be able to express mutability, and so on. The
numeric tower (PEP 3141) is also good to keep in mind. I think that's
all solvable once we start experimenting a bit.

Some important issues to bikeshed over:

- Tuples. Sometimes you want to say e.g. "a tuple of integers, don't
mind the length"; other times you want to say e.g. "a tuple of fixed
length containing an int and two strs". Perhaps the former should be
expressed using ImmutableSequence[Int] and the second as Tuple[Int,
Str, Str].

- Unions. We need a way to say "either X or Y". Given that we're
defining our own objects we may actually be able to get away with
writing e.g. "Int | Str" or "Str | List[Str]", and isinstance() would
still work. It would also be useful to have a shorthand for "either T
or None", written as Optional[T] or Optional(T).

- Whether to design notations to express other constraints. E.g.
"integer in range(10, 100)", or "one of the strings 'r', 'w' or 'a'",
etc. You can go crazy on this.

- Composability (Nick's pet peeve, in that he is against it). I
propose that we reserve plain tuples for this. If an annotation has
the form "x: (P, Q)" then that ought to mean that x must conform to
both P and Q. Even though Nick doesn't like this, I don't think we
should do everything with decorators. Surly, the decorators approach
is good for certain use cases, and should take precedence if it is
used. But e.g. IDEs that use annotations for suggestions and
refactoring should not require everything to be decorated -- that
would just make the code too busy.

- Runtime enforcement. What should we use type annotations for? IDEs,
static checkers (linters) and refactoring tools only need the
annotations when they are parsing the code. While it is tempting to
invent some kind of runtime checking that automatically checks the
actual types against the annotations whenever a function is called, I
think this is rarely useful, and often prohibitively slow. So I'd say
don't focus on this. Instead, explicit type assertions like "assert
isinstance(x, List[Int])" might be used, sparingly, for those cases
where we'd otherwise write a manual assertion with the same meaning
(which is also sparingly!). A decorator to do this might be useful
(especially if there's a separate mechanism for turning actual
checking on or off through some configuration mechanism).

> The first could be fixable by relaxing slightly the constraints of
> isinstance and issubclass, but not so for the second.
>
> [0] which works rather neatly for anonymous unions as `|` is the union
>     of two sets, so the arithmetic would be `type | type -> typeset`,
>     `type | typeset -> typeset` and `typeset | typeset -> typeset`,
>     libraries could offer opaque types/typesets which would be composable
>     without their users having to know whether they're type atoms or
>     typesets

I like this for declaring union types. I don't like it for composing
constraints that are intended for different tools.

-- 
--Guido van Rossum (python.org/~guido)