[Python-Dev] Type hints -- a mediocre programmer's reaction

[Cory Benfield]

On 21 April 2015 at 01:45, Chris Angelico <rosuav at gmail.com> wrote:
> When you're writing a library, it can be a great help to provide type
> annotations, because every application that uses your library can
> benefit.

It can be a great help to whom? Not to me (the library author),
because I can't use them in my library code, because I have to support
2.7. That's by no means a bad thing (after all, most libraries are
written to help others), but I found it really unclear who was being
advantaged here.

To show the downside, I decided to annotate requests' API a little
bit. Then I stopped, because I really couldn't work out how to do it.

For example, requests.get() takes a URL parameter. This can be an
instance of basestring, or anything that provides a __str__ or
__unicode__ method that provides us a string that looks like a URL
(such as a URL abstraction class). We don't know ahead of time what
those objects may be, so there's no type signature I can provide here
that is of any use beyond 'Any'.

We also provide headers/params options, whose type signature would be
(as far as I can tell) Union[Iterable[Tuple[basestring, basestring]],
Mapping[basestring, basestring]], which is a fairly unpleasant type
(and also limiting, as we totally accept two-element lists here as
well as two-tuples. That's got *nothing* on the type of the `files`
argument, which is the most incredibly polymorphic argument I've ever
seen: the best I can work out it would be:

Optional[
    Union[
        Mapping[
            basestring,
            Union[
                Tuple[basestring, Optional[Union[basestring, file]]],
                Tuple[basestring, Optional[Union[basestring, file]],
Optional[basestring]],
                Tuple[basestring, Optional[Union[basestring, file]],
Optional[basestring], Optional[Headers]]
            ]
        ],
        Iterable[
            Tuple[
                basestring,
                Union[
                    Tuple[basestring, Optional[Union[basestring, file]]],
                    Tuple[basestring, Optional[Union[basestring,
file]], Optional[basestring]],
                    Tuple[basestring, Optional[Union[basestring,
file]], Optional[basestring], Optional[Headers]]
            ]
        ]
    ]
]

Headers = Union[
    Mapping[basestring, basestring],
    Iterable[Tuple[basestring, basestring]],
]

This includes me factoring out the Headers type for my own sanity, and
does not include the fact that in practice almost all instances of
'basestring' above actually mean 'anything that can be safely cast to
basestring', and 'file' means 'any file-like object including BytesIO
and StringIO' (speaking of which, the PEP doesn't even begin to go
into how to handle that kind of requirement: do we have some kind of
Readable interface that can be implemented? Or do I have to manually
union together all types someone might want to use?).

Now, this can be somewhat improved: the three types of tuple (again,
not just tuples, often lists!) can be factored out, as can the
Union[basestring, file]. However, there's still really no way around
the fact that this is a seriously complex type signature!

Python has had years of people writing APIs that feel natural, even if
that means weird contortions around 'types'. Writing an argument as
'clever' as the 'files' argument in requests in a statically typed
language is an absolute nightmare, but Python makes it an easy and
natural thing to do.

Further, Python's type system is not sufficiently flexible to allow
library authors to adequately specify the types their code actually
works on. I need to be able to talk about interfaces, because
interfaces are the contract around which APIs are build in Python, but
I cannot do that with this system in a way that makes any sense at
all. To even begin to be useful for library authors this PEP would
need to allow some kind of type hierarchy that is *not* defined by
inheritance, but instead by interfaces. We've been discouraging use of
'type' and 'isinstance' for years because they break duck typing, but
that has *nothing* on what this PEP appears to do to duck typing.

I suppose the TLDR of this email is that I think that libraries with
'pythonic' APIs are the least likely to take up this typing system
because it will provide the least value to them. Maintaining
signatures will be a pain (stub files are necessary), writing clear
signatures will be a pain (see above), and writing signatures that are
both sufficiently open to be back-compatible and sufficiently
restrictive to be able to catch bugs seems like it might be very
nearly impossible.

I'd love for someone to tell me I'm wrong, though. I want to like this
PEP, I really do.