Notice: While Javascript is not essential for this website, your interaction with the content will be limited. Please turn Javascript on for the full experience.

PEP 604 -- Complementary syntax for Union[]

PEP:604
Title:Complementary syntax for Union[]
Author:Philippe PRADOS <python at prados.fr>
Sponsor:Chris Angelico <rosuav at gmail.com>
BDFL-Delegate:Ivan Levkivskyi <levkivskyi at gmail.com>
Status:Draft
Type:Standards Track
Created:28-Aug-2019
Python-Version:3.9

Introduction

This PEP describes an extension to Python language, which aims to add a complementary syntax to write Union[X,Y] easier.

Motivation

The PEP484 [1] and PEP526 [2] propose a generic syntax to add typing to variables, parameters and function returns. The PEP585 [3] proposes to expose parameters to generics at runtime. MyPy [4] accepts a syntax which looks like something like this:

annotation: name_type
name_type: NAME (args)?
args: '[' paramslist ']'
paramslist: annotation (',' annotation)* [',']
  • To describe a disjunction, the user must use Union[X,Y].

The verbosity of this syntax does not help the adoption.

Proposal

Inspired by Scala language [5], this proposal adds operator __or__() in the root type. With this new operator, it is possible to write int | str in place of Union[int,str]. This proposition uses the standard meaning of the | operator. Then, it is possible to extend isinstance() and issubclass() to accept this new syntax:

isinstance(int, int | str)

Examples

Here are some examples of what we can do with this feature.

# in place of
# def f(list: List[Union[int, str]], param: Optional[int]) -> Union[float, str]
def f(list: List[int | str], param: int | None) -> float | str:
    pass

f([1,"abc"],None)

assert str | int == Union[str,int]
assert str | int | float == Union[str, int, float]

assert isinstance("", int | str)
assert issubclass(int, int | str)

Once the Python language is extended, MyPy [3] must be updated to accept this new syntax.

Incompatible changes

In some situations, some exceptions will not be raised as expected.

If some metaclass overload the __or__ operator, the user must resolve the ambiguities with Union.

>>> class M(type):
...         def __or__(self,other): return "Hello"
...
>>> class C(metaclass=M):pass
...
>>> C | int
'Hello'
>>> int | C
typing.Union[int, __main__.C]
>>> Union[C,int]
typing.Union[__main__.C, int]

Dissenting Opinion

1. Add a new operator for Union[type1|type2]?

  • CONS: This is not a new proposal. If I recall correctly, it was proposed way back at the very beginning of the type-hinting discussion, and there has been at least one closed feature request for it: Issue 387
    • It is maybe too late to change this, many people are already get used to current notation.
      • This PEP propose to add a new notation, not to replace the notation
    • This syntax is difficult to google, if someone encounters it in code
    • It is still not possible to use | for unions because of built-in types. (This would require a corresponding slot in type which is a non-starter)
      • The proposed implementation do it
    • There are currently no volunteer to implement this in mypy
      • An implementation is proposed now (One patch for CPython and one for MyPy).
    • "but as @ilevkivskyi pointed out, that is not an option (at least until Python 4)."
      • Is it time now ?
  • PRO: It's similar of Scala
  • PRO: Seems like foo | None is just as readable
  • PRO: Which means you couldn't use this feature in Python 3.7, much less 2.7. I'm not sure it maintaining backward compatibility in typing and in mypy is still as important today as it was 5 years ago, but I'm pretty sure it hasn't been abandoned entirely.
  • CONS: add operator introducing a dependency to typing in builtins
  • CONS: supporting this would likely break compatibility with existing code that overloads | for class objects using a metaclass. We could perhaps work around this by making | inside an annotation context different from the regular | operator.
    • A work around is to use Union[type1,type2] in this case
  • CONS: You need __ror__ as well as __or__ - No, in this situation, Python auto invoke ``__or__`` in case of ``__ror__``.
  • CONS: as breaking the backport (in that typing.py can easily be backported but core types can't)
    • There are several things in the typing syntax that require a certain minimum version. E.g. type annotations require Python 3 (whereas type comments work in Python 2 too), type annotations on variables (PEP 526) require 3.6+, from __future__ import annotations (PEP 563) requires 3.7+.
  • PRO: I mean that at run-time, int|str might return a very simple object in 3.9, rather than everything that you'd need to grab from importing typing. Wondering if doing so would close off the possibility of, in 3.12 or something, making it a more directly usable "type union" that has other value.
  • CONS: if Python itself doesn't have to be changed, we'd still need to implement it in mypy, Pyre, PyCharm, Pytype, and who knows what else.
    • A proposed patch of mypy is just 20 lines of codes
  • If yes,

Change only the PEP484 (Type hints) to accept the syntax type1 | type2 ?

  • PRO: The PEP563 [6] (Postponed Evaluation of Annotations) is enough to accept this proposition
  • CONS: The Resolving type hints at runtime says: “For code which uses annotations for other purposes, a regular eval(ann, globals, locals) call is enough to resolve the annotation.". Without add a new operator __or__ in type type, it's not possible to resolve type hints at runtime.
  >>> from __future__ import annotations
  >>> def foo() -> int | str: pass
  ...
  >>> eval(foo.__annotations__['return'])
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<string>", line 1, in <module>
  TypeError: unsupported operand type(s) for |: 'type' and 'type'

- CONS: Without operator, it's not possible to write

>>> u = int | str
>>> u
typing.Union[int, str]

Use (int, str) in place of Union[int,str] ?

  • PRO: This doesn't have compatibility issues and it's similar to isinstance(foo, (int, str))
  • PRO: Either better than breaking backward compatibility by adding new operator methods to the type type.
  • CONS: In most languages with similar-ish type syntax, (int, str) means Tuple[int, str], not Union[int, str].

Use {int, str} in place of Union[int,str] ?

  • PRO: big advantage of {int, str} over int|str. It doesn't require adding anything to type, and we don't need to introduce a new lightweight builtin union type.

2. Extend isinstance() and issubclass() to accept Union ?

isinstance(x, str | int) ==> "is x an instance of str or int"
  • PRO: if they were permitted, then instance checks could use an extremely clean-looking notation for "any of these":
  • PRO: The implementation can use the tuple present in Union parameter, without create a new instance.

Reference Implementation

A proposed implementation for cpython is here. A proposed implementation for mypy is here.

Source: https://github.com/python/peps/blob/master/pep-0604.rst