Python is DOOMED! Again!

[Steven D'Aprano]

Mario Figueiredo wrote:

> In article <54c0a571$0$13002$c3e8da3$5496439d at news.astraweb.com>,
> steve+comp.lang.python at pearwood.info says...
>> 
>> The point isn't that there are no other alternative interpretations
>> possible, or that annotations are the only syntax imaginable, but that
>> they're not hard to guess what they mean, and if you can't guess, they're
>> not hard to learn and remember.
> 
> Possibly one common use case will be Unions. And that factory syntax is
> really awful and long when you look at a function definition with as
> little as 3 arguments. The one below has only 2 arguments.
> 
> def handle_employees(emp: Union[Employee, Sequence[Employee]], raise:
> Union[float, Sequence[float]]) -> Union[Employee, Sequence[Employee],
> None]:

You can't use "raise" as a parameter name, since that's a keyword. Using
floats for money is Just Wrong and anyone who does so should have their
licence to program taken away. And I really don't understand what this
function is supposed to do, that it returns None, a single Employee, or a
sequence of Employees. (If it's hard to declare what the return type is,
perhaps your function does too much or the wrong thing.)

But putting those aside, let's re-write that with something a little closer
to PEP-8 formatting:

def handle_employees(
            emp: Union[Employee, Sequence[Employee]],
            pay_raise: Union[int, Sequence[int]]
            ) -> Union[Employee, Sequence[Employee], None]:
    pass


That's quite nice and easy to follow. I can think of three improvements:

(1) Allow the return annotation to be on a line on its own rather than force
it to follow the closing bracket;

(2) Support | to make Unions of types;

(3) Have a shorter way to declare "Spam or Sequence (tuple?) of Spam". 


def handle_employees(
            emp: OneOrMore[Employee],
            pay_raise: OneOrMore[int])
            -> OneOrMore[Employee] | None:
    pass


(2) has been rejected by Guido, but he may change his mind. The name I've
chosen for (3) is just the first thing I've thought of.



> Meanwhile there's quite a few more generics like the Sequence one above
> you may want to take a look at and try and remember. And that's just one
> factory (the generics support factory). You may also want to take a look
> at TypeVar and Callable for more syntactic hell.

Exaggerate, much?

  
> Meanwhile, there's the strange decision to implement type hints for
> local variables # comment lines. I have an hard time wrapping my head
> around this one. Really, comments!?

Yes, really. There is plenty of prior art for machine-meaningful comments:

- mypy uses it, and it works fine
- Pascal uses {$ ...} compiler directives
- Unix uses a special hash-bang #! comment in the first line to 
  specify the executable that runs the script
- Python supports a special encoding declaration using #
- doctest uses comments for directives
- HTML puts code (Javascript usually) inside of comments
- JMSAssert for Java uses comments for design-by-contract assertions


But note that the type declarations have *no runtime effect*. They truly are
comments, aimed at the human reader and any compliant type-checker.


Guido has said that he isn't ruling out an explicit syntactic support for
type declarations in the future, but right now there are various
constraints:

- it must be backwards compatible, i.e. something that Python 3.4 and older
  will just ignore
- it must be available by lexical analysis, that is, from reading the 
  source code, which rules out anything that happens at runtime
- it must be human-readable
- and easily parsed by even simple tools



> Finally, remember all this is being added to your code just to
> facilitate static analysis. 

You say "just" to facilitate static analysis, but let me put it this way. It
is "just" to facilitate:

- improved correctness of programs
- to assist in finding bugs as early as possible
- reduced need to write tedious, boring unit tests to check things 
  that an automated type-checker can deal with instead
- to reduce the need for runtime type-checks and isinstance() calls
- to aid in producing documentation
- to give hints to IDEs, editors, linters, code browsers and 
  similar tools.



> Strangely enough though I was taught from 
> the early beginning that once I start to care about types in Python, I
> strayed from the pythonic way. I'm confused now... What is it then?

That's actually a really good question.

Good practice in Python will not change. If anything, explicitly checking
types with isinstance will become even less common: if you can use lexical
analysis to prove that the argument passed to a function is always a float,
there is no need to perform a runtime check that it is a float.

Notice that nearly all the examples in the PEP use abstract classes like
Sequence instead of concrete classes like list? This is a good thing, and
will encourage more flexible code. You don't need a specific type of
sequence, any type of sequence will do -- that's pretty much the definition
of duck-typing.

I've watched the evolution of typing in Python over the 15+ years. Back in
the old days of Python 1.4 and 1.5, the only way to type-check was to
compare two types for equality:

if type(x) == type(1.0):
    print "x is a float"


That was very restrictive. You couldn't check for subclasses. You couldn't
subclass built-in types like floats, but you could use delegation, but
there was no way to tell Python your float-proxy should be treated as if it
were a float. Being so restrictive, it was best avoided, hence the very
strong emphasis on duck-typing.

Python 2.2 introduced "new style classes", which meant that you could
subclass built-ins. Functions like str, float, int and list became type
objects (classes) instead of functions, so now you could write:

if type(x) == float: print "x is a float"

Python 2.2 also introduced issubclass and isinstance, which allowed you to
check for a specific type or sub-type. That's less restrictive than exact
type equality: if you want something that quacks like a float, chances are
that it probably is a float or a sub-class of a float.

Python 2.5 (I think?) introduced Abstract Base Classes, which takes it to
the next level. Now, if you want something that quacks like a float, you
don't even have to inherit from float! You can create your own classes
without inheriting from float at all, and then register it as a float, and
Python will treat it as if it were a float. Or you can inherit from
*abstract* classes which provide a lot of the basic functionality needed so
you're not forced to re-invent the wheel.

Duck-typing in the Python 1.5 sense has a number of issues that make it less
than a panacea:

- Sometimes you really must have an duck, not just something duck-like. 
  If you're trying to find a mate for Donald Duck, you want a duck, 
  not a goose.

- More practically, if you're interfacing with C or Java or Fortran code,
  you don't have the luxury of passing any old "duck like" object. If
  the C code requires a float, you have to be sure you give it a float.

- It is good programming practice to raise errors as close as possible 
  to the source as you can. If your function requires a float, it is 
  often better to raise an exception *immediately* if it receives a
  non-float, not deep inside the function's body.

- Duck-typing has its share of problems too. Consider a function that
  expects an Artist object, and calls the draw() method, but instead it
  received a Gunslinger object. Instead of getting a nice AttributeError,
  your code will now do the wrong thing. How do you prevent errors like
  this, if not by type-checking?


Since the "language wars" of the 1990s, dynamic languages have won. Even
static languages like Java contain dynamic features. But the victory isn't
all one way: dynamic languages are gaining static features too. Static
typing can improve performance and correctness, and it is silly to reject
that out of some misplaced idealism that there is One True Way to design a
language.




-- 
Steven