[Python-Dev] Malloc interfaced

[Vladimir Marangozov]

Greg Stein wrote:
> 
> > ...
> > 
> > #define PyMem_DEL(p)	PyMem_FREE(p)
> > #define PyMem_XDEL(p)	PyMem_FREE(p)  /* obsolete */
> 
> Both of these would be obsolete.

This makes a lot of sense to me.

(Oh boy, I've spent big time this weekend trying to clean up the distrib,
discovering subtle errors, etc, and now that I think I have something
ready, you suggest me rethink the interface and rename DEL to FREE in the
whole distribution? :-)

> 
> Note that the PyMem_NEW/RESIZE/FREE macros are intended for internal use
> by Python only. (move to a private header?)
> 
> Py_Malloc and friends, and PyMem_Malloc and friends are to be used by C
> extensions (or embeddors).

I couldn't agree more.

(modulo that hiding the macros in a pvt header may cause gobs of feedback
like "Why PyMem_DEL isn't available anymore? I use it to release my objects
allocated with PyObject_NEW"...).

Let me give you a status:

Currently, malloc/realloc/free/PyMem_NEW/RESIZE/DEL & PyObject_NEW are
used in the core and in the extensions without respecting any particular
rules, because such rules didn't exist.

Everything relies on the assumption that these primitives boil down
to malloc/realloc/free. And this is what prevents Python from being
user-malloc friendly (although "hostile" would be a better word ;)

After extensive malloc figthing, I have a working patch which corrects
this aspect of the code (core + extensions in Modules/*), but preserves
the PyMem_MALLOC/REALLOC/FREE and PyMem_NEW/RESIZE/DEL families entirely,
assuming they're public macros.

With Fredrik's and Greg's remarks included, mymalloc.h would look like:

-----------
/* Python's raw memory interface.

   To make the interpreter user-malloc friendly, all other memory or object
   APIs are implemented on top of this one.

   These can be changed to make the interpreter use another allocator. */
...
#define PyMem_MALLOC(n)		malloc(n)
#define PyMem_REALLOC(p, n)	realloc((ANY *)(p), (n))
#define PyMem_FREE(p)		free((ANY *)(p))

/* Type-oriented memory interface. */

#define PyMem_NEW(type, n) \
	( (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)) )
#define PyMem_RESIZE(p, type, n) \
	if ((p) == NULL) \
	    (p) = (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)); \
	else \
	    (p) = (type *) PyMem_REALLOC((p), \
				       _PyMem_EXTRA + (n) * sizeof(type))

#define PyMem_DEL(p)  PyMem_FREE(p)  /* obsolete */
#define PyMem_XDEL(p) PyMem_FREE(p)  /* obsolete */
----------

Indeed, I'm inclined to zap all PyMem_DEL and rename them to PyMem_FREE
in the core, thus leaving PyMem_NEW & PyMem_RESIZE as handy shotcuts.
I'm okay to put all these macros in a private header too, but...

Problem:

There's a lot of code (core + Modules/* + perhaps 3rd party) which says:

	o = PyObject_NEW(<some_struct>, &<some_type>);

then:

	Mem_DEL(o);

This is messy. This is the wrong NEW/DEL pair for extension modules.

It's acceptable to use this for the core, but for C extensions, the right
pair is PyObject_NEW()/PyMem_Free().

PyObject_NEW is a public C interface for creating objects
(defined as _PyObject_New, which calls PyMem_MALLOC).

The public destructor function should be named PyObject_DEL, which should
be defined as PyMem_Free (which in turn calls PyMem_FREE).


And this is where we are at the moment. No PyObject_DEL, spaghetti code.

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252