[Csv] bug fixed

Skip Montanaro skip at pobox.com
Tue Feb 4 23:56:58 CET 2003


    Andrew> If I remember correctly, Python 2.3 and Python 2.2 are very
    Andrew> different with regard to memory allocation - most of our testing
    Andrew> has been done with 2.2, PyMem_Malloc is a thin layer on top of
    Andrew> the system malloc, is it not?

No, it's the portal into PyMalloc, an efficient, generic small block
allocator.

    Andrew> Funnily enough, I had a weird heap corruption problem in a
    Andrew> python application that used csv a while back - because csv was
    Andrew> the only extension module I was using, I immediatly assumed csv
    Andrew> was the source, and spent many hours trying to find a
    Andrew> miss-handled memory allocation. I eventually decided the problem
    Andrew> was elsewhere (can't remember details).  Looks like it might
    Andrew> have been csv after all.

And the behavior will be different with different mallocs.  free() on Mac OS
X is apparently smart enough to realize it was being handed bad memory and
refused to really free() it.  Other free()'s might blindly charge ahead,
corrupting PyMalloc's memory.  In my case, I think it mostly just caused
memory leaks because those chunks were never freed as far as PyMalloc was
concerned.

Looking at .../include/python2.N/pymem.h, it looks like PyMem_Malloc and
MyMem_Free have needed to be paired up for awhile whenever PyMalloc was
enabled.  From 2.1/2.2:

    extern DL_IMPORT(void *) PyMem_Malloc(size_t);
    extern DL_IMPORT(void *) PyMem_Realloc(void *, size_t);
    extern DL_IMPORT(void) PyMem_Free(void *);

>From 2.3:

    PyAPI_FUNC(void *) PyMem_Malloc(size_t);
    PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t);
    PyAPI_FUNC(void) PyMem_Free(void *);

The difference between 2.1/2.2 and 2.3 is that PyMalloc is enabled by
default in 2.3.

Skip



More information about the Csv mailing list