[Python-ideas] PEP 484 (Type Hints) -- first draft round

[Guido van Rossum]

On Sun, Jan 18, 2015 at 12:36 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Guido van Rossum schrieb am 18.01.2015 um 05:11:
> > On Sat, Jan 17, 2015 at 1:43 AM, Stefan Behnel wrote:
> >> Guido van Rossum schrieb am 16.01.2015 um 21:08:
> >>> I don't know what to answer for Cython -- doesn't it have its own
> syntax
> >>> for declaring C types? Or is it now also using annotations?
> >>
> >> It can, but it's optional. When I added support for it, I took care not
> to
> >> enable it by default, so that it wouldn't accidentally interfere with
> other
> >> usages of annotations.
> >>
> >> The current proposal seems less generous in this regard, and the way I
> see
> >> it, the only gain from Cython's point of view is its very limited
> support
> >> for container item type declarations and function signatures of callable
> >> objects. Both are restricted to Python object types and even imply
> slightly
> >> different semantics than Cython (I didn't see a notion of "exact types"
> as
> >> opposed to "type or subtype").
> >
> > Actually, while the PEP is still lacking in clarity and we're still
> having
> > some discussions, type variables will most likely be "invariant" by
> > default, which I think is the "exact types" notion you are after. There's
> > lots of discussion in https://github.com/ambv/typehinting/issues/2 (you
> > might want to read this from the end :-).
>
> No, AFAICT, it's actually not what I meant. The discussion above seems to
> refer only to the case where a container like "List[X]" may or may not
> accept subtypes of *X*. For Cython, what matters is whether "list" (or
> "List[X]") refers to exactly a "list" or allows subtypes of the "list"
> container type. If the type annotation allows subtypes of builtins, it's
> useless for Cython, as it does not allow it to generate better code than it
> does anyway. If it means "exactly list and no subtypes of list", Cython can
> use it to avoid generating generic fallback code.


Can you explain how this works? How does Cython manage to use the knowledge
that the argument's __class__ is equal to builtins.list to generate more
efficient code?

Also, don't you have to insert dynamic checks to ensure the caller passes
exactly a list anyway?

Very few people subclass builtins.list (usually it's much simpler to
subclass collections.abc.MutableSequence). So perhaps Cython could just
assume that typing.List means builtins.list and (dynamically) reject calls
that pass a subclass of builtins.list?


> That's why Cython's
> current type system enforces exact types when builtin Python types are
> declared. (You may call it use case optimised: declare only types that
> help.) And it's one of the reasons why the proposed type system
> representation is of so little value for Cython. It's simply not intended
> for anything but type checking.
>
> Note that I'm not proposing Cython's semantics for a type system
> representation that is designed only for type checking, but I guess it
> would be nice to at least be able to explicitly express somehow that
> subtypes should be disallowed.
>

Can you come up with a specific proposal? Maybe we're not as far apart as
we think. :-)


> >> If there was at least a notion of "unknown types" that tools like static
> >> type checkers MUST ignore (couldn't find that in the PEP either), then
> >> Cython users could mix this with Cython's Python level type mapping
> through
> >> the "cython" magic module, e.g. something like "cython.struct(x=
> cython.int
> >> , y=cython.p_char)".
> >
> > Perhaps you could supply a stub for the cython module that declares
> > everything in it as type 'Any'? That should shut the type checker up.
>
> Yes, I guess that would allow Cython's extensions to Python's type system
> to be ignored. We could even try to be a little smarter and map some C
> parts of Cython's type system down to more generic types in Python's type
> system (e.g. int32->int). That would at least give us a minimal baseline
> level of "interoperability" (as in "it doesn't break and isn't completely
> useless").
>

Right. I still don't understand enough about Cython's type system to be
able to make useful suggestions myself -- I am really hoping that you'll
take the time to understand PEP 484 well enough so you can help guide it
towards something that would benefit Cython without giving up the existing
goals (which are similar to what is done e.g. in Hack and Typescript -- see
PEP 482).


> Regarding non-typish declarations (like the contrived doc() example in my
> pull request), will there be a way (maybe in that "stub" mechanism) to tell
> type analysis tools that whatever they are looking for, they are not going
> to find it in this specific kind of annotation, as it's "not a type"? That
> seems sufficiently different (semantically) from "this represents any
> type". If it's really left to users to do that at a per-annotation basis,
> it sounds cumbersome enough to make the real intention appear as saying
> "annotations are for types only".
>

We'll start out with a way using magic comments (maybe # type: OFF|ON) to
disable the interpretation of annotations as types by the type checker
during specific sections of a module. The checker will then assume no
annotations, and in that case it will assume every argument and return
value has the special type 'Any'. (Please do read up in PEP 483 about the
important difference between Any and object.) We'll also have a decorator
to disable type checking per function or class.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150118/91842fbc/attachment.html>