[issue3526] Customized malloc implementation on SunOS and AIX

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> Fortunately, it is easy to solve by defining the following in
> dlmalloc:
> #define HAVE_MORECORE 0

I was expecting this answer ;-)
Here's a quick demo, on a Linux box:

cf at neobox:~/cpython$ ./python Tools/pybench/pybench.py -n 1
-------------------------------------------------------------------------------
Totals:                          19787ms  19787ms

cf at neobox:~/cpython$ MALLOC_MMAP_THRESHOLD_=0 ./python Tools/pybench/pybench.py -n 1
[...]
-------------------------------------------------------------------------------
Totals:                          33375ms  33375ms

That's a mere 70% slowdown, and without pymalloc, it would be much worse. malloc with mmap() is way slower than with sbrk() (see http://sources.redhat.com/ml/libc-alpha/2006-03/msg00033.html for more details). Since your benchmarks don't show this type of regression it probably means that AIX's malloc implementation is really broken (there's also the fact that part of the allocations are still routed to the libc's malloc, or maybe your workload is too specific to demonstrate this behavior).

> sbrk is generally considered quite archaic.

I wouldn't say that; see the above link on malloc's dynamic mmap() threshold.

> I also don't expect this patch to be integrated spontaneously without
> someone interested in AIX pushing for it.

Indeed.
As far as I'm concerned, there are two "showstoppers":
- shipping an implementation of dlmalloc with Python
- mixing dlmalloc with the host's malloc implementation

But I think the main problem with this patch is that AIX represents such a tiny fraction of the user base. This might change in the future, especially if IBM is successfull in its effort of pushing AIX (I hope they'll finally fix AIX's malloc by then...).

> I have been quite busy recently with the birth of my second daughter,
> a new job, a new home town and soon a new home.

Congratulations, and good luck!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3526>
_______________________________________