[Python-Dev] Using PEP384 Stable ABI for the lzma extension module

[Charles-François Natali]

>> > > > That's not a given. Depending on the memory allocator, a copy can be
>> > > > avoided. That's why the "str += str" hack is much more efficient under
>> > > > Linux than Windows, AFAIK.
>> > >
>> > > Even Linux will have to copy a block on realloc in certain cases, no?
>> >
>> > Probably so. How often is totally unknown to me :)
>> >
>> http://www.gnu.org/software/libc/manual/html_node/Changing-Block-Size.html
>>
>> It depends on whether there's enough free memory after the buffer you
>> currently have allocated.  I suppose that this becomes a question of what
>> people consider "the general case" :-)
>
> But under certain circumstances (if a large block is requested), the
> allocator uses mmap(), no?

That's right, if the block requested is bigger than mmap_threshold
(256K by default with glibc, forgetting the sliding window algorithm):
I'm not sure of what percentage of strings/buffers are concerned in a
"typical" program.

> In which case mremap() should allow to resize without copying anything.

Yes, there's no copying. Note however that it doesn't come for free,
the kernel will still zero-fill the pages before handling them to
user-space. It is still way faster than on, let's say, Solaris.

cf