[issue27213] Rework CALL_FUNCTION* opcodes

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Currently there are 4 opcodes (CALL_FUNCTION, CALL_FUNCTION_VAR, CALL_FUNCTION_KW, CALL_FUNCTION_VAR_KW) for calling a function depending of presenting the var-positional and var-keyword arguments:

    func(arg1, ..., argN, name1=kwarg1, ..., nameM=kwargM)
    func(arg1, ..., argN, *args, name1=kwarg1, ..., nameM=kwargM)
    func(arg1, ..., argN, name1=kwarg1, ..., nameM=kwargM, **kwargs)
    func(arg1, ..., argN, *args, name1=kwarg1, ..., nameM=kwargM, **kwargs)

The number of positional and keyword arguments are packed in oparg, and both numbers are limited by 255. Thus the single keyword argument makes oparg not fitting in 8 bit and requires EXTENDED_ARG. The stack contains first positional arguments, then optional args tuple, then keyword arguments, then optional kwargs dict. For every keyword argument the two values are pushed on the stack: argument name (always constant string) and argument value.

I collected a statistic about opcodes in compiled code during running Python tests [1] (maybe it is biased, but this is large and multifarious assembly, and I hope it takes some representation about average Python code). According to it about 90% of compiled function calls are calls with the fixed number of positional arguments (CALL_FUNCTION with oparg < 256), the rest 10% are calls with the fixed number of positional and keyword arguments (CALL_FUNCTION with oparg >= 256), and only about 0.5% are calls with the var-positional or var-keyword arguments.

I propose to use the different sets of opcodes that corresponds to these cases:

    func(arg1, ..., argN)
    func(arg1, ..., argN, name1=kwarg1, ..., nameM=kwargM)
    func(*args, **kwargs)

1. CALL_FUNCTION takes the fixed number of positional arguments. oparg is the number of arguments. The stack contains positional arguments.

2. CALL_FUNCTION_KW takes the fixed number of positional and keyword arguments. oparg is the number of all arguments. The stack contains values of arguments (first positional, then keyword), then a tuple of keyword names (as in proposed new opcode BUILD_CONST_KEY_MAP in issue27140).

3. CALL_FUNCTION_EX takes the variable number of positional and keyword arguments. oparg is 0. The stack contains a tuple of positional arguments and a dict of keyword arguments (they are build in the bytecode with using BUILD_TUPLE, BUILD_CONST_KEY_MAP, BUILD_LIST_UNPACK and BUILD_MAP_UNPACK_WITH_CALL). This is the most general variant, others exist just for the optimization of common cases.

Benefits:

1. Calling a function with keyword arguments uses less stack and less LOAD_CONST instructions.
2. Calling a function with keyword arguments no longer needs EXTENDED_ARG.
3. The number of positional and keyword arguments is no longer limited by 255 (at least not in the bytecode).
4. The bytecode looks simpler, oparg always is just the number of arguments taken from the stack.

This proposition was discussed on Python-Ideas [2].

[1] http://permalink.gmane.org/gmane.comp.python.ideas/39993
[2] http://comments.gmane.org/gmane.comp.python.ideas/39961

----------
components: Interpreter Core
messages: 267241
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Rework CALL_FUNCTION* opcodes
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27213>
_______________________________________