From noreply at sourceforge.net  Thu Sep 10 01:09:04 2009
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 09 Sep 2009 23:09:04 +0000
Subject: [Expat-bugs] [ expat-Bugs-2855609 ] dangling positionPtr after error
Message-ID: <E1MlWHg-0005zT-7Z@g45xhf1.ch3.sourceforge.com>

Bugs item #2855609, was opened at 2009-09-09 19:09
Message generated for change (Tracker Item Submitted) made by dvitek
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Dave Vitek (dvitek)
Assigned to: Nobody/Anonymous (nobody)
Summary: dangling positionPtr after error

Initial Comment:
I am getting a segv when asking pyexpat to parse some bad xml.  I'm not sure whether this is a problem with pyexpat or with expat, but I'm leaning towards expat after finding a (partial?) fix.

The stack of the segv is:
(gdb) bt
#0  0x010245f8 in normal_updatePosition (enc=0x1035fc0, ptr=0x73b000 <Address 0x73b000 out of bounds>, end=0x186cc50 "</<die", 'x' <repeats 76 times>, "</<die", 'x' <repeats 76 times>, "</<die", 'x' \
<repeats 30 times>..., pos=0x1317668) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmltok_impl.c:1748
#1  0x01011545 in XML_GetCurrentLineNumber (parser=0x13174d0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmlparse.c:1793
#2  0x010093ca in set_error (self=0x725540, code=XML_ERROR_INVALID_TOKEN) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:126
#3  0x0100c633 in get_parse_result (self=0x725540, rv=0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:918
#4  0x0100c6e0 in xmlparse_Parse (self=0x725540, args=0x7701e8) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:940
#5  0x00068067 in PyCFunction_Call (func=0x770a30, arg=0x7701e8, kw=0x0) at Objects/methodobject.c:81
...

What appears to be going on is something like this:

There is an existing parser object whose m_positionPtr and m_eventPtr properly point somewhere inside m_buffer.  Everything is fine at this point.

Someone calls XML_Parse on this parser.  It proceeds to call XML_GetBuffer.  XML_GetBuffer allocates a new buffer, but doesn't touch m_positionPtr or m_eventPtr.  So these two pointers now point to freed memory (the old buffer).

After XML_GetBuffer returns, there is a call to XML_ParseBuffer.  This function has a failure case just before it sets m_positionPtr and m_eventPtr to point somewhere better.  It returns XML_STATUS_ERROR before touching m_positionPtr.

pyexpat sees this error code and proceeds to call XML_GetErrorLineNumber.  XML_GetLineNumber dereferences m_positionPtr in the stack we see above, causing the crash.

I'm using the pyexpat (part of Python 2.6) interface to expat.  Python 2.6 embeds a copy of expat's lib subdirectory at Modules/expat.  To confirm that things were not better in 2.0.1 I just copied the 2.0.1 sources over the old ones.  I was pleasantly surprised when it built 2.0.1 fine without any tinkering.  Anyway, the problem still happened with 2.0.1.

My fix was to add these three lines near the end of XML_GetBuffer, under the assumption that nothing good can come of the dangling pointers:

     }
+     positionPtr = NULL;
+     eventPtr = NULL;
+     eventEndPtr = NULL;
   }
   return bufferEnd;
 }


My test case was:

from xml.parsers.expat import ParserCreate

for abc in range(1,1000):
    print abc
    p = ParserCreate()
    for i in xrange(100):
        try:
            p.Parse('<die%s</' % ('x' * abc), False)
        except:
            pass


It only caused a crash on MacOSX; it worked OK on Linux.  That's probably just because glibc's free isn't giving back pages to the OS as aggressively as the BSD one though.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127

From noreply at sourceforge.net  Thu Sep 10 01:20:36 2009
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 09 Sep 2009 23:20:36 +0000
Subject: [Expat-bugs] [ expat-Bugs-2855609 ] dangling positionPtr after error
Message-ID: <E1MlWSq-0002Sk-6H@665xhf1.ch3.sourceforge.com>

Bugs item #2855609, was opened at 2009-09-09 19:09
Message generated for change (Comment added) made by dvitek
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Dave Vitek (dvitek)
Assigned to: Nobody/Anonymous (nobody)
Summary: dangling positionPtr after error

Initial Comment:
I am getting a segv when asking pyexpat to parse some bad xml.  I'm not sure whether this is a problem with pyexpat or with expat, but I'm leaning towards expat after finding a (partial?) fix.

The stack of the segv is:
(gdb) bt
#0  0x010245f8 in normal_updatePosition (enc=0x1035fc0, ptr=0x73b000 <Address 0x73b000 out of bounds>, end=0x186cc50 "</<die", 'x' <repeats 76 times>, "</<die", 'x' <repeats 76 times>, "</<die", 'x' \
<repeats 30 times>..., pos=0x1317668) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmltok_impl.c:1748
#1  0x01011545 in XML_GetCurrentLineNumber (parser=0x13174d0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmlparse.c:1793
#2  0x010093ca in set_error (self=0x725540, code=XML_ERROR_INVALID_TOKEN) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:126
#3  0x0100c633 in get_parse_result (self=0x725540, rv=0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:918
#4  0x0100c6e0 in xmlparse_Parse (self=0x725540, args=0x7701e8) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:940
#5  0x00068067 in PyCFunction_Call (func=0x770a30, arg=0x7701e8, kw=0x0) at Objects/methodobject.c:81
...

What appears to be going on is something like this:

There is an existing parser object whose m_positionPtr and m_eventPtr properly point somewhere inside m_buffer.  Everything is fine at this point.

Someone calls XML_Parse on this parser.  It proceeds to call XML_GetBuffer.  XML_GetBuffer allocates a new buffer, but doesn't touch m_positionPtr or m_eventPtr.  So these two pointers now point to freed memory (the old buffer).

After XML_GetBuffer returns, there is a call to XML_ParseBuffer.  This function has a failure case just before it sets m_positionPtr and m_eventPtr to point somewhere better.  It returns XML_STATUS_ERROR before touching m_positionPtr.

pyexpat sees this error code and proceeds to call XML_GetErrorLineNumber.  XML_GetLineNumber dereferences m_positionPtr in the stack we see above, causing the crash.

I'm using the pyexpat (part of Python 2.6) interface to expat.  Python 2.6 embeds a copy of expat's lib subdirectory at Modules/expat.  To confirm that things were not better in 2.0.1 I just copied the 2.0.1 sources over the old ones.  I was pleasantly surprised when it built 2.0.1 fine without any tinkering.  Anyway, the problem still happened with 2.0.1.

My fix was to add these three lines near the end of XML_GetBuffer, under the assumption that nothing good can come of the dangling pointers:

     }
+     positionPtr = NULL;
+     eventPtr = NULL;
+     eventEndPtr = NULL;
   }
   return bufferEnd;
 }


My test case was:

from xml.parsers.expat import ParserCreate

for abc in range(1,1000):
    print abc
    p = ParserCreate()
    for i in xrange(100):
        try:
            p.Parse('<die%s</' % ('x' * abc), False)
        except:
            pass


It only caused a crash on MacOSX; it worked OK on Linux.  That's probably just because glibc's free isn't giving back pages to the OS as aggressively as the BSD one though.

----------------------------------------------------------------------

>Comment By: Dave Vitek (dvitek)
Date: 2009-09-09 19:20

Message:
Just noticed this is a duplicate of 2518079.  However that one was
evidently closed because it "stopped happening."

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127

From feldmann_markus at gmx.de  Fri Sep 11 12:48:48 2009
From: feldmann_markus at gmx.de (markus)
Date: Fri, 11 Sep 2009 10:48:48 +0000 (UTC)
Subject: [Expat-bugs] gnuradio gets segfault in libexpat
Message-ID: <loom.20090911T124127-731@post.gmane.org>

Hi All,

i am using Gnuradio, which uses python and python uses libexpat. :-) My Gnuradio
programm has some graphical output using wx libraries, for example to show a
fft-plot or a scope plot. I have 2 PC's one works and the other get the
segfault. Both have the same OS(openSUSE 11.1). The segfault is:

Sep 11 12:08:26 linux-bqhl kernel: python[15201]: segfault at 8009 ip b45b9678
sp bfba7860 error 4 in libexpat.so.1.5.2[b45b6000+25000]

I do not know how to debug this further and get more information. Any idea ?

Maybe i am missing some libraries or some is broken.

Regards Markus


From noreply at sourceforge.net  Tue Sep 29 04:51:52 2009
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 29 Sep 2009 02:51:52 +0000
Subject: [Expat-bugs] [ expat-Bugs-2855609 ] dangling positionPtr after error
Message-ID: <E1MsSoi-0006t2-EW@665xhf1.ch3.sourceforge.com>

Bugs item #2855609, was opened at 2009-09-09 19:09
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Test Required
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Dave Vitek (dvitek)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: dangling positionPtr after error

Initial Comment:
I am getting a segv when asking pyexpat to parse some bad xml.  I'm not sure whether this is a problem with pyexpat or with expat, but I'm leaning towards expat after finding a (partial?) fix.

The stack of the segv is:
(gdb) bt
#0  0x010245f8 in normal_updatePosition (enc=0x1035fc0, ptr=0x73b000 <Address 0x73b000 out of bounds>, end=0x186cc50 "</<die", 'x' <repeats 76 times>, "</<die", 'x' <repeats 76 times>, "</<die", 'x' \
<repeats 30 times>..., pos=0x1317668) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmltok_impl.c:1748
#1  0x01011545 in XML_GetCurrentLineNumber (parser=0x13174d0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/expat/xmlparse.c:1793
#2  0x010093ca in set_error (self=0x725540, code=XML_ERROR_INVALID_TOKEN) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:126
#3  0x0100c633 in get_parse_result (self=0x725540, rv=0) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:918
#4  0x0100c6e0 in xmlparse_Parse (self=0x725540, args=0x7701e8) at /Volumes/extra/dvitek/trunk/third-party/python/Modules/pyexpat.c:940
#5  0x00068067 in PyCFunction_Call (func=0x770a30, arg=0x7701e8, kw=0x0) at Objects/methodobject.c:81
...

What appears to be going on is something like this:

There is an existing parser object whose m_positionPtr and m_eventPtr properly point somewhere inside m_buffer.  Everything is fine at this point.

Someone calls XML_Parse on this parser.  It proceeds to call XML_GetBuffer.  XML_GetBuffer allocates a new buffer, but doesn't touch m_positionPtr or m_eventPtr.  So these two pointers now point to freed memory (the old buffer).

After XML_GetBuffer returns, there is a call to XML_ParseBuffer.  This function has a failure case just before it sets m_positionPtr and m_eventPtr to point somewhere better.  It returns XML_STATUS_ERROR before touching m_positionPtr.

pyexpat sees this error code and proceeds to call XML_GetErrorLineNumber.  XML_GetLineNumber dereferences m_positionPtr in the stack we see above, causing the crash.

I'm using the pyexpat (part of Python 2.6) interface to expat.  Python 2.6 embeds a copy of expat's lib subdirectory at Modules/expat.  To confirm that things were not better in 2.0.1 I just copied the 2.0.1 sources over the old ones.  I was pleasantly surprised when it built 2.0.1 fine without any tinkering.  Anyway, the problem still happened with 2.0.1.

My fix was to add these three lines near the end of XML_GetBuffer, under the assumption that nothing good can come of the dangling pointers:

     }
+     positionPtr = NULL;
+     eventPtr = NULL;
+     eventEndPtr = NULL;
   }
   return bufferEnd;
 }


My test case was:

from xml.parsers.expat import ParserCreate

for abc in range(1,1000):
    print abc
    p = ParserCreate()
    for i in xrange(100):
        try:
            p.Parse('<die%s</' % ('x' * abc), False)
        except:
            pass


It only caused a crash on MacOSX; it worked OK on Linux.  That's probably just because glibc's free isn't giving back pages to the OS as aggressively as the BSD one though.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2009-09-28 22:51

Message:
Fixed in xmlparse.c rev. 1.164.
Please report back if there are still issues.

----------------------------------------------------------------------

Comment By: Dave Vitek (dvitek)
Date: 2009-09-09 19:20

Message:
Just noticed this is a duplicate of 2518079.  However that one was
evidently closed because it "stopped happening."

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2855609&group_id=10127