[issue36611] Debug memory allocators: remove useless "serialno" field to reduce memory footprint

STINNER Victor report at bugs.python.org
Fri Apr 12 07:01:10 EDT 2019


STINNER Victor <vstinner at redhat.com> added the comment:

The serialno has been added at the same time than the whole debug hooks on Python memory allocated by Tim Peters in 2002, 17 years ago:

commit ddea208be9e2a8fa281e25ebbc890378dd2aa286
Author: Tim Peters <tim.peters at gmail.com>
Date:   Sat Mar 23 10:03:50 2002 +0000

    Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
    
    When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
    allocator.  This can be done independent of build type (release or debug).
    A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
    enabled.  It's a detected error to define PYMALLOC_DEBUG when pymalloc
    isn't enabled.
    
    Two debugging entry points defined only under PYMALLOC_DEBUG:
    
    + _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
      to sanity-check a memory block obtained from pymalloc.  It sprays
      info to stderr (see next) and dies via Py_FatalError if the block is
      detectably damaged.
    
    + _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
      about a debug memory block to stderr.
    
    A tiny start at implementing "API family" checks isn't good for
    anything yet.
    
    _PyMalloc_DebugRealloc() has been optimized to do little when the new
    size is <= old size.  However, if the new size is larger, it really
    can't call the underlying realloc() routine without either violating its
    contract, or knowing something non-trivial about how the underlying
    realloc() works.  A memcpy is always done in this case.
    
    This was a disaster for (and only) one of the std tests:  test_bufio
    creates single text file lines up to a million characters long.  On
    Windows, fileobject.c's get_line() uses the horridly funky
    getline_via_fgets(), which keeps growing and growing a string object
    hoping to find a newline.  It grew the string object 1000 bytes each
    time, so for a million-character string it took approximately forever
    (I gave up after a few minutes).
    
    So, also:
    
    fileobject.c, getline_via_fgets():  When a single line is outrageously
    long, grow the string object at a mildly exponential rate, instead of
    just 1000 bytes at a time.
    
    That's enough so that a debug-build test_bufio finishes in about 5 seconds
    on my Win98SE box.  I'm curious to try this on Win2K, because it has very
    different memory behavior than Win9X, and test_bufio always took a factor
    of 10 longer to complete on Win2K.  It *could* be that the endless
    reallocs were simply killing it on Win2K even in the release build.

----------
nosy: +tim.peters

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36611>
_______________________________________


More information about the Python-bugs-list mailing list