[Numpy-discussion] subclassing array in c

[Christoph Gohle]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:

> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>> thanks for testing. I have now tried on different platforms. I get
>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>> leak...
>> 
>> Any hints would be welcome.
> 
> The type object inherits `tp_alloc` from Numpy. This routine always
> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
> PyArrayObject. Therefore, the write to new->unit in your
> __array_finalize__ goes to unallocated memory.
> 
> This is probably a bug in Numpy --- arrayobject.c:array_alloc should
> respect the size specified by the subtype.
> 
> A workaround is probably to specify a suitable tp_alloc routine yourself:
> 
>   PyType_GenericAlloc,        /* tp_alloc */
>    unitArray_new,              /* tp_new */
>    _PyObject_Del               /* tp_free */
Now I have been playing around with this configuration and can't seem to get rid of the memory leak I was talking about earlier. Reference counting seems to be fine as far as I can tell. I am using the following tp_dealloc:

static void unitArray_dealloc(UnitArrayObject *obj) {
	Py_XDECREF(obj->unit);
	obj->unit = NULL;
	if (obj->base.ob_type->tp_base != NULL) {
		//call base dealloc
		obj->base.ob_type->tp_base->tp_dealloc((PyObject*)obj);
	}
}

I was also trying to provide my own tp_alloc like below and using the inherited tp_free. This also crashes with a segmentation fault upon freeing saying that the memory block was not allocated. Does this make sense?

static PyObject *
unitArray_alloc(PyTypeObject *type, Py_ssize_t NPY_UNUSED(nitems))
{
	PyObject *obj;
	/* nitems will always be 0 */
	obj = (PyObject *)PyObject_Malloc(type->tp_basicsize);
	PyObject_Init(obj, type);
	return obj;
}

Cheers,
Christoph

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iEYEARECAAYFAk90FvkACgkQLYu25rCEIzvd+gCeNRgsv44g8kFJut5OQNXvK9zv
XckAoKWEjj3A34i4H+POOU/EIzzSU1EX
=kJPT
-----END PGP SIGNATURE-----