Python C API String Memory Consumption

[John Machin]

On Apr 7, 9:19 pm, MRAB <goo... at mrabarnett.plus.com> wrote:
> k3xji wrote:
> > Interestaing I changed malloc()/free() usage with PyMem_xx APIs and
> > the problem resolved. However, I really cannot understand why the
> > first version does not work. Here is the latest code that has no
> > problems at all:
>
> > static PyObject *
> > penc(PyObject *self, PyObject *args)
> > {
> >    PyObject * result = NULL;
> >    unsigned char *s= NULL;
> >    unsigned char *buf = NULL;
> >    unsigned int v,len,i = 0;
>
> >    if (!PyArg_ParseTuple(args, "s#", &s, &len))
> >         return NULL;
>
> >    buf = (unsigned char *) PyMem_Malloc(len);
> >    if (buf == NULL) {
> >            PyErr_NoMemory();
> >            return NULL;
> >    }
>
> >         /* string manipulation. */
>
> >    result = PyString_FromStringAndSize((char *)buf, len);
> >    PyMem_Free(buf);
> >    return result;
> > }
>
> In general I'd say don't mix your memory allocators. I don't know
> whether CPython implements PyMem_Malloc using malloc,

The fantastic manual (http://docs.python.org/c-api/
memory.html#overview) says: """the C allocator and the Python memory
manager ... implement different algorithms and operate on different
heaps""".

> but it's better to
> stick with CPython's memory allocators when writing for CPython.

for the reasons given in the last paragraph of the above reference.

HTH,
John