[Numpy-discussion] Memory allocation of arrays and tracemalloc

[Nathaniel Smith]

On Thu, Jan 15, 2015 at 5:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> I see that the PyDataMem_* APIs call malloc()/free()/etc. directly,
> instead of going through PyMem_Malloc, etc. This means the memory
> allocated by those APIs won't be seen by tracemalloc. Is it deliberate?
> Am I missing something?

There are two reasons:

1) We need PyMem_Calloc, which doesn't exist in any released version
of Python. It *will* exist in 3.5, though (thanks to Victor Stinner
for adding it for us!).

2) We *might* in the future need further API extensions (e.g. for
aligned memory, or who knows what), and this makes us nervous. If we
start supporting tracemalloc, then that becomes a user-facing feature
that we'll be committed to supporting indefinitely, which means that
we'll be locked into using Python's allocation API forever. (See
https://github.com/numpy/numpy/issues/4663.) So we're reluctant to
accept that lock-in without having some sort of escape hatch, e.g. the
one described at the end of this message:
   http://bugs.python.org/issue18835#msg232221

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org