[Python-Dev] int/float freelists vs pymalloc
Christian Heimes
lists at cheimes.de
Wed Feb 13 17:02:28 CET 2008
M.-A. Lemburg wrote:
> +1 on removing the freelists from ints and floats, but not the
> small int sharing optimization
>
> +1 on focusing on improving pymalloc to handle int and float
> object allocations even better
>
> -1 on changing the freelist implementations to use pymalloc for
> allocation of the freelist members instead of malloc, since
> this would potentially lead to pools (and arenas) being held alive
> by just a few objects - in the worst case a whole arena (256kB)
> for just one int object (14 bytes on 32bit platforms).
>
> Eventually, all freelists should be removed, unless there's a
> significant performance loss.
The small ints (and small longs in py3k) should definitely stay. They
make a huge speed impact and they reduce the memory usage a tiny bit, too.
I like to keep the free lists for basic types, too. They bring a
measurable speedup. My profiling has shown that the small free lists
safe about 70% malloc and PyObject_INIT() calls for lists. In Python 3.0
the long free list increases the general speed of int operations by 10%+.
+1 on replacing int's and float's block allocation schema
with pymalloc
+1 on keeping small int optimization
+1 on focusing on improving pymalloc to handle int and float
object allocations even better
+1 on adding a small int, float and long (py3k only) free
list with about 80 to 160 objects each
+1 for MvL's idea to compact all free lists during a full
gc sweep. Every type with a free list gets a new function
PySpam_ClearFreeList() which frees all items in the type's
free list.
The latter counteracts the potential issue with arenas.
By the way objects are always aligned at 8 byte address boundaries. A 12
or 4 bytes object occupies 16 bytes.
Christian
More information about the Python-Dev
mailing list