[Python-ideas] Proposal to extend PEP 484 (gradual typing) to support Python 2.7

[Brett Cannon]

On Sat, 9 Jan 2016 at 11:31 Guido van Rossum <guido at python.org> wrote:

> On Sat, Jan 9, 2016 at 1:54 AM, Pavol Lisy <pavol.lisy at gmail.com> wrote:
>
>> Could not something like this ->
>>
>>     def embezzle(self, account, funds=1000000, *fake_receipts):
>>         # def embezzle(self, account: str, funds: int = 1000000,
>> *fake_receipts: str) -> None:
>>         """Embezzle funds from account using fake receipts."""
>>         <code goes here>
>>
>> make
>> 1. transition from python2 to python3 more simple?
>> 2. python3 checkers more easily changeable to understand new python2
>> standard?
>> 3. simpler impact to documentation (means also simpler knowledbase to
>> be learn) about annotations?
>>
>
> There would still have to be some marker like "# type:" for the type
> checker to recognize -- I'm sure that plain comments with alternate 'def'
> statements are pretty common and we really don't want the type checker to
> be confused by those.
>
> I don't like that the form you propose has so much repetition -- the
> design of Python 3 annotations intentionally is the least redundant
> possible, and my (really Jukka's) proposal tries to keep that property.
>
> Modifying type checkers to support this syntax is easy (Jukka already did
> it for mypy).
>
> Note that type checkers already have to parse the source code without the
> help of Python's ast module, because there are other things in comments:
> PEP 484 specifies variable annotations and a few forms of `# type: ignore`
> comments.
>
> Regarding the idea of a decorator, this was discussed and rejected for the
> original PEP 484 proposal as well. The problem is similar to that with your
> 'def' proposal: too verbose. Also a decorator is more expensive (we're
> envisioning adding many thousands of decorators, and it would weigh down
> program startup). We don't envision needing to introspect __annotations__
> at run time. (Also, we already use decorators quite heavily -- introducing
> a @typehint decorator would make the code less readable due to excessive
> stacking of decorators.)
>
> Our needs at Dropbox are several: first, we want to add annotations to the
> code so that new engineers can learn their way around the code quicker and
> refactoring will be easier; second, we want to automatically check
> conformance to the annotations as part of our code review and continuous
> integration processes (this is where mypy comes in); third, once we have
> annotated enough of the code we want to start converting it to Python 3
> with as much automation is feasible. The latter part is as yet unproven,
> but there's got to be a better way than manually checking the output of
> 2to3 (whose main weakness is that it does not know the types of variables).
> We see many benefits of annotations and automatically checking them using
> mypy -- but we don't want the them to affect the runtime at all.
>

To help answer the question about whether this could help with porting code
to Python 3, the answer is "yes"; it's not essential but definitely would
be helpful.

Between Modernize, pylint, `python2.7 -3`, and `python3 -bb` you cover
almost all of the issues that can arise in moving to Python 3. But notice
that half of those tools are running your code under an interpreter with a
certain flag flipped, which means run-time checks that require excellent
test coverage. With type annotations you can do offline, static checking
which is less reliant on your tests covering all corner cases. Depending on
how the tools choose to handle representing str/unicode in Python 2/3 code
(i.e., say that if you specify the type as 'str' it's an error and anything
that is 'unicode' is considered the 'str' type in Python 3?), I don't see
why mypy can't have a 2/3 compatibility mode that warns against uses of,
e.g. the bytes type that don't directly translate between Python 2 and 3
like indexing. That kind of static warning would definitely be beneficial
to anyone moving their code over as they wouldn't need to rely on e.g.,
`python3 -bb ` and their tests  to catch that common issue with bytes and
indexing.

There is also the benefit of gradual porting with this kind of offline
checking. Since you can slowly add more type information, you can slowly
catch more issues in your code. Relying on `python3 -bb`, though, requires
you have ported all of your code over first before running it under Python
3 to catch some issues.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160109/54707726/attachment-0001.html>