[New-bugs-announce] [issue37211] obmalloc: eliminate limit on pool size

[Tim Peters]

New submission from Tim Peters <tim at python.org>:

On 64-bit Python, many object sizes essentially doubled over 32-bit Python, because Python objects are so heavy with pointers.  More recently, forcing alignment to 16 bytes on 64-bit boxes boosted the memory requirements more modestly.

But obmalloc's 256 KiB arenas and 4 KiB pools haven't changed since obmalloc was first written, and its `address_in_range()` machinery cannot deal with pools bigger than that (they're segfault factories, because the machinery relies on that a pool is no larger than a system page).

obmalloc's fastest paths are those that stay within a pool.  Whenever a pool boundary is hit, it necessarily gets slower, then slower still if an arena boundary is hit.

So I propose to:

- Remove the 4 KiB pool limit, by making `address_in_range()` page-based rather than pool-based.  Pools should be able to span any power-of-2 number of pages.  Then a pool for a given size class will be able to hold that many more times as many objects too, and so stay in the fastest paths more often.

- On 64-bit boxes, increase both POOL_SIZE and ARENA_SIZE.

----------
components: Interpreter Core
messages: 345097
nosy: tim.peters
priority: normal
severity: normal
status: open
title: obmalloc:  eliminate limit on pool size
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37211>
_______________________________________