[New-bugs-announce] [issue26828] Implement __length_hint__() on map() and filter() to optimize list(map) and list(filter)

[STINNER Victor]

New submission from STINNER Victor:

When I compared the performance of filter() and map() between Python 2.7 and 3.4, I noticed a huge performance drop in Python 3!
http://bugs.python.org/issue26814#msg264003

I didn't analyze yet exactly why Python 3 is so much slower (almost 100% slower for the case of fitler!). Maybe it's because filter() returns a list on Python 2, whereas filter() returns an iterator on Python 3.

In Python 2, filter() and map() use _PyObject_LengthHint(seq, 8) to create the result list. Why don't we do the same in Python 3?

filter.__length_hint__() and map.__length_hint__() would return seq.__length_hint__() of the input sequence, or return 8. It's a weak estimation, but it can help a lot of reduce the number of realloc() when the list is slowly growing.

See also the PEP 424 -- A method for exposing a length hint.

Note: the issue #26814 (fastcall) does make filter() and map() faster on Python 3.6 compared to Python 2.7, but it's not directly related to this issue. IMHO using length hint would make list(filter) and list(map) even faster.

----------
messages: 264007
nosy: alex, haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Implement __length_hint__() on map() and filter() to optimize list(map) and list(filter)
type: performance
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26828>
_______________________________________