[ python-Bugs-1075984 ] Memory fault pyexpat.so on SGI

SourceForge.net noreply at sourceforge.net
Fri Sep 30 08:01:16 CEST 2005


Bugs item #1075984, was opened at 2004-11-30 05:13
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075984&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: XML
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Maik Hertha (mhertha)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: Memory fault pyexpat.so on SGI

Initial Comment:
I build the latest RC1 of python 2.4.
System SGI Fuel IP35, Irix 6.5.26m

cc -version
MIPSpro Compilers: Version 7.3.1.3m

- make tests passes test_pyexpat without error
- running this code leads to a core dump

-- code ---
import xml.dom.minidom
doc = xml.dom.minidom.parseString(fstream)

<<< core dump >>>
--- runing python -v test.py
#
/opt/python24c1/lib/python2.4/xml/dom/expatbuilder.pyc
matches
/opt/python24c1/lib/python2.4/xml/dom/expatbuilder.py
import xml.dom.expatbuilder # precompiled from
/opt/python24c1/lib/python2.4/xml/dom/expatbuilder.pyc
import xml.parsers # directory
/opt/python24c1/lib/python2.4/xml/parsers
#
/opt/python24c1/lib/python2.4/xml/parsers/__init__.pyc
matches
/opt/python24c1/lib/python2.4/xml/parsers/__init__.py
import xml.parsers # precompiled from
/opt/python24c1/lib/python2.4/xml/parsers/__init__.pyc
# /opt/python24c1/lib/python2.4/xml/parsers/expat.pyc
matches /opt/python24c1/lib/python2.4/xml/parsers/expat.py
import xml.parsers.expat # precompiled from
/opt/python24c1/lib/python2.4/xml/parsers/expat.pyc
dlopen("/opt/python24c1/lib/python2.4/lib-dynload/pyexpat.so",
2);
Memory fault(coredump)

- running this code from an interactive session leads
not to a coredump

I need some assistance howto provide further debug
information.

--maik./

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2005-09-29 23:01

Message:
Logged In: YES 
user_id=33168

The problem seems to be related (same?) as bug #1295808. 
Does the patch there fix these problems?

----------------------------------------------------------------------

Comment By: Steve Juranich (sjuranic)
Date: 2005-07-19 09:22

Message:
Logged In: YES 
user_id=1315245

FWIW, this same problem crops up when using Python & VTK
together (which also ships with its own version of expat). 
bos's workaround will make things work, though.

----------------------------------------------------------------------

Comment By: Bernhard Herzog (bernhard)
Date: 2005-03-29 10:11

Message:
Logged In: YES 
user_id=2369

I ran into this problem as well on a debian GNU/Linux system
on x86 hardware.  It occurs both with pygtk 2.4 and wxPython
2.5 both built against gtk 2.4.

bos' patch at least solves the immediate problem of the
segmentation fault.  It may be a good idea to print a
warning message if some of the error codes cannot be
translated to strings, though.


----------------------------------------------------------------------

Comment By: Bryan O'Sullivan (bos)
Date: 2005-02-04 15:16

Message:
Logged In: YES 
user_id=28380

With the GNU linker, you can pass in -Bstatic to force it to resolve the symbols in the local shared library, instead of globally. This also works on Irix. I don't know about other Unixes.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-02-02 02:35

Message:
Logged In: YES 
user_id=6656

It's a nasty one, I'll give you that.

Fred, what do you think of bos's patch?  It solves the immediate issue, 
but I'm not sure it's a complete fix.

It seems to me that it would be better to resolve the expat symbols at 
builf time, but I don't know how to arrange for that.

----------------------------------------------------------------------

Comment By: Bryan O'Sullivan (bos)
Date: 2005-02-01 22:09

Message:
Logged In: YES 
user_id=28380

By the way, this is not an SGI-specific bug. This will bite any Unix system with a modern sysv-style dynamic linker. The priority of this bug should be higher, IMO.

----------------------------------------------------------------------

Comment By: Bryan O'Sullivan (bos)
Date: 2005-01-31 23:53

Message:
Logged In: YES 
user_id=28380

I've been bitten by the same bug. In my case, the problem was with Qt, not GTK.

It's not actually necessary to check the expat version; just see if XML_ErrorString actually returns anything.

Here's the patch I use for this problem:

--- python/Modules/pyexpat.c        2005-01-06 16:26:13 -08:00
+++ python/Modules/pyexpat.c  2005-01-31 23:46:36 -08:00
@@ -1936,9 +1936,12 @@
     }
 #endif

-#define MYCONST(name) -    PyModule_AddStringConstant(errors_module, #name, -                               (char*)XML_ErrorString(name))
+#define MYCONST(name)                           +    { +        char *_v = (char*)XML_ErrorString(name); +        if (_v) +            PyModule_AddStringConstant(errors_module, #name, _v); +    }

     MYCONST(XML_ERROR_NO_MEMORY);
     MYCONST(XML_ERROR_SYNTAX);


----------------------------------------------------------------------

Comment By: Stephen Watson (kerofin)
Date: 2004-12-13 05:46

Message:
Logged In: YES 
user_id=471223

pyexpat initializes using its internal copy of expat. 
libexpat.so is still mapped in (after pyexpat has
initilized), but gdb finds the python copy of
XML_ErrorString and other functions.

I suspect that this will be a problem if the system version
of expat is newer than Python's.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-12-13 05:01

Message:
Logged In: YES 
user_id=6656

1) Good sleuthing.
2) Argh!
3) What happens if you import pyexpat before pygtk?

----------------------------------------------------------------------

Comment By: Stephen Watson (kerofin)
Date: 2004-12-13 01:29

Message:
Logged In: YES 
user_id=471223

I've looked at the program that was dumping core and the
sequence is this:

1) Program imports pygtk, which links in the GTK libraries
2) Program loads SVG image which links in librsvg.so which
in turn links in /usr/local/lib/libexpat.so
3) Program imports pyexpat.
4) pyexpat calls XML_ErrorString, but as ld.so has already
linked in XML_ErrorString from /usr/local/lib/libexpat.so it
calls that version, not the one in pyexpat.so.  
5) pyexpat looks up an error defined by the later version of
expat it is expecting and gets a NULL pointer from the
earlier version it has.  It attempts to use it without
checking (strlen) and dumps core.

----------------------------------------------------------------------

Comment By: Stephen Watson (kerofin)
Date: 2004-12-10 09:01

Message:
Logged In: YES 
user_id=471223

Maybe it compiled against its own copy of expat, but pulled
in the system's copy when run?  

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-12-10 08:52

Message:
Logged In: YES 
user_id=6656

Uh, Python includes its own copy of expat, and I really
thought we were supposed to prefer our own version over
anything found on the system...

----------------------------------------------------------------------

Comment By: Stephen Watson (kerofin)
Date: 2004-12-10 08:46

Message:
Logged In: YES 
user_id=471223

I also got a core dump importing pyexpat on Solaris (SPARC)
and somebody else reported it on a *BSD system.

It appears to be a problem with older versions of expat not
being tested for.  I used gdb to trace it down to line 1972
in pyexpat.c where it attempts to define the first constant
from 1.95.8.  I had expat 1.95.7.  After upgrading to 1.95.8
it worked fine, as did the *BSD system. 

I think Python needs to check the expat version, as it does
elsewhere in the file, before defining those constants. 

I am still puzzelled over how it managed to compile
pyexpat.c in the first place...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075984&group_id=5470


More information about the Python-bugs-list mailing list