[Python-Dev] The untuned tunable parameter ARENA_SIZE

[Antoine Pitrou]

On Thu, 1 Jun 2017 01:41:15 -0700
Larry Hastings <larry at hastings.org> wrote:
> On 06/01/2017 01:19 AM, Antoine Pitrou wrote:
> > If you'd like to go that way anyway, I would suggest 1MB as a starting
> > point in 3.7.  
> 
> I understand the desire for caution.  But I was hoping maybe we could 
> experiment with 4mb in trunk for a while?  We could change it to 1mb--or 
> even 256k--before beta 1 if we get anxious.

Almost nobody tests "trunk" (or "master" :-)) on production systems.  At
best a couple rare open source projects will run their test suite on
the pre-release betas, but that's all.  So we are unlikely to spot
memory usage ballooning problems before the final release.

> >>    * Many programs would be slightly faster now and then, simply because
> >>      we call malloc() 1/16 as often.  
> > malloc() you said?  Arenas are allocated using mmap() nowadays, right?  
> 
> malloc() and free().  See _PyObject_ArenaMalloc (etc) in Objects/obmalloc.c.

_PyObject_ArenaMalloc should only be used if the OS doesn't support
mmap() or MAP_ANONYMOUS (see ARENAS_USE_MMAP).  Otherwise
_PyObject_ArenaMmap is used.

Apparently OS X doesn't have MAP_ANONYMOUS but it has the synonymous
MAP_ANON:
https://github.com/HaxeFoundation/hashlink/pull/12

Regards

Antoine.