[Python-ideas] Optional static typing -- the crossroads

[Steven D'Aprano]

On Sat, Aug 16, 2014 at 10:03:48PM -0700, Guido van Rossum wrote:
> I'd like to summarize the main issues that have come up. As an experiment,
> I'm not changing the subject, but I am still not quoting anything in
> particular. Only two issues (or issue clusters) really seem contentious:
> 
> (1) Should the function annotation syntax (eventually) be reserved for type
> annotations in a standard syntax? 

Reserved, as in alternatives are prohibited? No.

But assumed to be types by default? I think so.


> Or can multiple different uses of
> annotations coexist? And if they can, how should a specific use be
> indicated? (Also, some questions about compile-time vs. run-time use.)

Determining whether annotations should be interpreted as type 
annotations or not needs to be possible both at compile-time and 
run-time. I suggest a special decorator, imported from the typing 
module:


from typing import skip_types  # or some better name?

@skip_types(marker)  # see below for the purpose of marker
@another_decorator  # the order of decorators doesn't matter
def function(x:"this is not a type")->[int]:
    ...


At compile-time, static typing tools should treat any function decorated 
by skip_types as if it were dynamic.

The skip_types decorator also writes marker to the __annotations__ dict, 
using a key which cannot be used as an identifier. Say, "+mark+". The 
purpose of the marker is so that other tools can determine whether the 
annotations are aimed at them.

def handle_annotations(func):
    if func.__annotations__.get("+mark+", None) is MyMarker:
        do_stuff(func.__annotations__)
    else:
        pass

Any time a function is not decorated by skip_types (or whatever name it 
has), or doesn't have that "+mark+" key in the __annotations__ dict, 
static type checking tools are entitled to assume the annotations are 
used for typing.


> (2) For type annotations, should we adopt (roughly) the mypy syntax or the
> alternative proposed by Dave Halter? This uses built-in container notations
> as a shorthand, e.g. {str: int} instead of Dict[str, int]. This also
> touches on the issue of abstract vs. concrete types (e.g. iterable vs.
> list).

I prefer the mypy syntax.

 
> Regarding (1), I continue to believe that we should eventually reserve
> annotations for types, to avoid confusing both humans and tools, but I
> think there's nothing we have to do in 3.5 -- 3.5 must preserve backward
> compatibility, and we're not proposing to give annotations any new
> semantics anyway -- the actual changes to CPython are limited to a new
> stdlib module (typing) and some documentation.
> 
> Perhaps a thornier issue is how mypy should handle decorators that
> manipulate the signature or annotations of the function they wrap. But I
> think the only reasonable answer here can be that mypy must understand what
> decorators do if it wants to have any chance at type-checking decorated
> functions. 

Hmmm. Anything the decorators do to the annotations will be at runtime, 
so is it reasonable to say that the static typing tool will only operate 
on the annotations available at compile time?

That is, given:

@mangle_annotations
def spam(x:int)->List[str]: ...

the type checker is expected to use the annotations seen at 
compile-time, no matter what the mangle_annotations decorator happens to 
do at run-time.

Otherwise, the type-checker needs to be a full Python interpreter, in 
order to see what mangle_annotations does. And that could be an
intractible problem:

def mangle_annotations(func):
    if random.random() < 0.5:
        func.__annotations__['x'] = List[str]
    else:
        func.__annotations__['x'] = float


I don't see how any type-checker is supposed to take that into account.

(1) Compile-time type checkers should ignore the decorator and just use 
the annotations available at compile-time;

(2) Run-time analysis tools should use the annotations available at 
run-time;

(3) If they happen to be different, oh well, consenting adults.



-- 
Steven