This program makes Python segfault - no other does

[Michael Hudson]

Juho Saarikko <sorry at but.no.spam> writes:

> On Mon, 17 May 2004 17:54:32 +0000, Michael Hudson wrote:
> 
> > Juho Saarikko <sorry at but.no.spam> writes:
> >> #0  0x400c4c1b in free () from /lib/libc.so.6
> >> #1  0x400c4aa3 in free () from /lib/libc.so.6
> >> #2  0x0807ff2e in PyObject_Free (p=0x81d7240) at Objects/obmalloc.c:774
> >> #3  0x0807f5a6 in PyMem_Free (p=0x81d7240) at Objects/object.c:2111
> >> #4  0x4023a2d4 in unQuoteBytea (sin=0x81dd2ec ">nemo wrote:") at libpqmodule.c:417
> > 
> > Oh look, this is clearly inside the libpq extension module!  What
> > evidence do you have for a bug in Python itself?
> 
> The function unQuoteBytea allocates memory with PyMem_Malloc, and frees it
> with PyMem_Free. The segfault happens at freeing the memory (as the
> backtrace shows). It seems to me that if Python's memory management
> routines fail to free an object they've allocated, it must be a bug in
> Python. 

Um.  Almost certainly PyMem_Malloc winds up just calling malloc(), and
you can see above that PyMem_Free is winding up calling free().

Is this a debug build of Python?  You might want to try one of them.

> That or some other bug corrupts memory structures, in which case
> it's almost impossible to track down. At this point I'm considering
> either switching to a different database plugin, or to Java.
> 
> I tried the new Python version (3.3.4c1) and got the exact same behaviour.
> Aarrgghh.
> 
> Here, I'll attach the unQuoteBytea function, it's a short one. Maybe you
> can find some problem in it I couldn't:
> 
> 
> PyObject *unQuoteBytea(char *sin)
> {
>     int i, j, slen, byte;
>     char *sout;
>     PyObject *result;
>                                                                                                                                              
>     slen = strlen(sin);
>     sout = (char *)PyMem_Malloc(slen);
>     if (sout == (char *)NULL)
>         return PyErr_NoMemory();
>                                                                                                                                              
>     for (i = j = 0; i < slen;)
>     {
>         switch (sin[i])
>         {
>             case '\\':
>                 i++;
>                 if (sin[i] == '\\')
>                     sout[j++] = sin[i++];
>                 else
>                 {
>                     if ((!isdigit(sin[i]))   ||
>                         (!isdigit(sin[i+1])) ||
>                         (!isdigit(sin[i+2])))
>                         goto unquote_error;
>                                                                                                                                              
>                     byte = VAL(sin[i++]);
>                     byte = (byte << 3) + VAL(sin[i++]);
>                     sout[j++] = (byte << 3) + VAL(sin[i++]);
>                 }
>                 break;
>                                                                                                                                              
>             default:
>                 sout[j++] = sin[i++];
>         }
>     }
> 
>     sout[j] = (char)0;

I think j can equal slen at this point?

Truth in advertising: I googled for libpqmodule.c, got (as I hoped)
the CVS logs from SF, noticed that the most recent log entry said:

09NOV2003 bga	Fixed a buffer overrun error in libPQquoteBytea based on a fix
		by James Matthew Farrow. [Bug #838317].

, that the date of the this log was after the most recent release of
pypgsql and then looked at the diff.

Maybe you should try building pypgsql from CVS...

Cheers,
mwh

-- 
  If you're talking "useful", I'm not your bot.
                                            -- Tim Peters, 08 Nov 2001