[Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

[Victor Stinner]

Thanks for your feedback, you are asking good questions :-)

2016-02-04 13:54 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
>> There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
>> and PyMem_Free().
>>
>> I would prefer to modify a single place having to replace 536 calls :-/
>
> You have a point there, but I don't think it'll work out
> that easily, since we are using such calls to e.g. pass
> dynamically allocated buffers to code in extensions (which then
> have to free the buffers again).

Ah, interesting. But I'm not sure that we delegate the responsability
of freeing the memory to external libraries. Usually, it's more the
opposite: a library gives us an allocated memory block, and we have to
free it. No?

I checked if we call directly malloc() to pass the buffer to a
library, but I failed to find such case.

Again, in debug mode, calling free() on a memory block allocated by
PyMem_Malloc() will likely crash. Since we run the Python test suite
with a Python compiled in debug mode, we would already have detected
such bug, no?

See also my old issue http://bugs.python.org/issue18203 which replaced
almost all direct calls to malloc() with PyMem_Malloc() or
PyMem_RawMalloc().


> Good question. I guess developers simply thought of PyObject_Malloc()
> being for PyObjects,

Yeah, I also understood that, but in practice, it looks like
PyMem_Malloc() is slower than so using it makes the code less
efficient than it can be.

Instead of teaching developers that well, in fact, PyObject_Malloc()
is unrelated to object programming, I think that it's simpler to
modify PyMem_Malloc() to reuse pymalloc ;-)

Victor