[Python-ideas] Proposal: Use mypy syntax for function annotations

[Guido van Rossum]

On Wed, Aug 13, 2014 at 6:00 PM, Łukasz Langa <lukasz at langa.pl> wrote:

> It’s great to see this finally happening!
>

Yes. :-)


> I did some research on existing optional-typing approaches [1]. What I
> learned in the process was that linting is the most important use case for
> optional typing; runtime checks is too little, too late.
>
> That being said, having optional runtime checks available *is* also
> important. Used in staging environments and during unit testing, this case
> is able to cover cases obscured by meta-programming. Implementations like
> “obiwan” and “pytypedecl” show that providing a runtime type checker is
> absolutely feasible.
>

Yes. And the proposal here might well enable such applications (by
providing a standard way to spell complex types). But I think it's going to
be less important than good support for linting, so that's what I want to
focus on first.


> The function annotation syntax currently supported in Python 3.4 is not
> well-suited for typing. This is because users expect to be able to operate
> on the types they know. This is currently not feasible because:
> 1. forward references are impossible
>

(Mypy's hack for this is that a string literal can be used as a forward
reference.)

2. generics are impossible without custom syntax (which is the reason
> Mypy’s Dict exists)
> 3. optional types are clumsy to express (Optional[int] is very verbose for
> a use case this common)
>

So define an alias 'oint'. :-)


> 4. union types are clumsy to express
>

Aliasing can help.


> All those problems are elegantly solved by Google’s pytypedecl via moving
> type information to a separate file.
>

Mypy supports this too using stub files, but I think it is actually a
strength that it doesn't require new syntax (although if the idea becomes
popular we could certainly add syntax to support those things where mypy
currently requires magic comments).

Honestly I'm not sure what to do about mypy vs. pytypedecl. Should they
compete, collaborate, converge? Do we need a bake-off or a joint hackathon?
Food for thought.


> Because for our use case that would not be an acceptable approach, my
> intuition would be to:
>
> 1. Provide support for generics (understood as an answer to the question:
> “what does this collection contain?”) in Abstract Base Classes. That would
> be a PEP in itself.
>
2. Change the function annotation syntax so that it’s not executed at
> import time but rather treated as strings. This solves forward references
> and enables us to…
>
3. Extend the function annotation syntax with first-class generics support
> (most languages like "list<str>”)
>
4. Extend the function annotation syntax with first-class union type
> support. pytypedecl simply uses “int or None”, which I find very elegant.
>
5. Speaking of None, possibly further extend the function annotation syntax
> with first-class optionality support. In the Facebook codebase in Hack we
> have tens of thousands of optional ints (nevermind other optional types!),
> this is a case that’s going to be used all the time. Hack uses ?int, that’s
> the most succinct style you can get. Yes, it’s special but None is a
> special type, too.
>

Hm. I think that selling such (IMO) substantial changes to Python's syntax
is going to be much harder than just the idea of a standard typing syntax
implemented as a new stdlib module. While mypy's syntax is perhaps not as
concise or elegant as would be possible if we were to design the syntax
from the ground up, it's actually pretty darn readable, and it is
compatible with Python 3.2. It has decent ways to spell generics, forward
references, unions and optional types already. And while I want to
eventually phase out other uses of function annotations, your change #2
would break all existing packages that use them for other purposes (like
Ethan Furman's scription).


> All in all, I believe Mypy has the highest chance of becoming our typing
> linter, which is great! I just hope we can improve on the syntax, which is
> currently lacking. Also, reusing our existing ABCs where applicable would
> be nice. With Mypy’s typing module I feel like we’re going to get a new,
> orthogonal set of ABCs, which will confuse users to no end. Finally, the
> runtime type checker would make the ecosystem complete.
>

We can discuss these things separately. Language evolution is an exercise
in compromise. We may be able to reuse the existing ABCs, and mypy could
still support Python 3.2 (or, with the codeck hack, 2.7) by having the
typing module export aliases to those ABCs. I won't stop you from
implementing a runtime type checker, but I think it should be a separate
project.


> This is just the beginning of the open issues I was juggling with and the
> reason my own try at the PEP was coming up slower than I’d like.
>

Hopefully I've motivated you to speed up!


> [1] You can find a summary of examples I looked at here:
> http://lukasz.langa.pl/typehinting/
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140813/4e4336a0/attachment.html>