From noreply at sourceforge.net  Sat Mar  3 20:01:32 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:01:32 -0800
Subject: [Expat-bugs] [ expat-Bugs-3496608 ] CVE-2012-0876 - Hash DOS attack
Message-ID: <3V0cDZ56QHzNtr@mail.python.org>

Bugs item #3496608, was opened at 2012-03-03 11:01
Message generated for change (Tracker Item Submitted) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Karl Waclawek (kwaclaw)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CVE-2012-0876 - Hash DOS attack

Initial Comment:
The hash table implementation in Expat can be attacked by a carefully crafted input document where all identifiers hash to the same value.
This leads to a denial of service scenario by forcing hash table lookups to do linear searching.
CVE-2012-0876 (see http://http://www.cve.mitre.org) has been reserved for this issue.
Also discussed on bugs.python.org/issue13703#msg151870 .

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:14:28 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:14:28 -0800
Subject: [Expat-bugs] [ expat-Patches-3446384 ] Report byte offsets for attr
	names and values
Message-ID: <3V0cWV2f8JzNx5@mail.python.org>

Patches item #3446384, was opened at 2011-11-30 12:20
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3446384&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Moses Hall (ikavairai)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: Report byte offsets for attr names and values

Initial Comment:
The attached patch exposes a new API for querying the byte offsets of attribute names and values. The new API is XML_GetAttributeInfo(), which returns a structure with four byte offsets: attribute name start & end, and attribute value start & end.

We use this new functionality in our indexing software at the University of Michigan Libraries; we needed to be able to index all parts of the source XML file, including attributes.

Patch is against the latest CVS sources. Thank you for your attention.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:14

Message:
This patch was applied in expat.h rev 1.81 and xmlparse.c rev 1.168.
It was implemented as a new conditional compile feature
XML_FEATURE_ATTR_INFO.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3446384&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:16:04 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:16:04 -0800
Subject: [Expat-bugs] [ expat-Bugs-3206497 ] Unitialized memory returned
	from XML_Parse
Message-ID: <3V0cYL4KJ9zNyd@mail.python.org>

Bugs item #3206497, was opened at 2011-03-11 06:52
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3206497&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Peter Van der Beken (peter_vdb)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: Unitialized memory returned from XML_Parse

Initial Comment:
XML_Parse ends with |return result;|, but there's a case where result is left unitialized, namely if the call to processor on line 1483 returns XML_ERROR_NONE and ps_parsing has been set to XML_FINISHED. I'm attaching a patch that makes sure result is always initialized.

(This was reported in Mozilla as https://bugzilla.mozilla.org/show_bug.cgi?id=607721)

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:16

Message:
Patch applied in xmlparse.c rev 1.168.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3206497&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:20:16 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:20:16 -0800
Subject: [Expat-bugs] [ expat-Bugs-3496608 ] CVE-2012-0876 - Hash DOS attack
Message-ID: <3V0cfC0JhrzP1Y@mail.python.org>

Bugs item #3496608, was opened at 2012-03-03 11:01
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Test Required
Status: Open
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Karl Waclawek (kwaclaw)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CVE-2012-0876 - Hash DOS attack

Initial Comment:
The hash table implementation in Expat can be attacked by a carefully crafted input document where all identifiers hash to the same value.
This leads to a denial of service scenario by forcing hash table lookups to do linear searching.
CVE-2012-0876 (see http://http://www.cve.mitre.org) has been reserved for this issue.
Also discussed on bugs.python.org/issue13703#msg151870 .

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:20

Message:
Fixed in expat.h rev 1.81 and xmlparse.c rev 1.168.
Thanks to David Malcolm (RedHat) for providing me with the initial version
of the patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:22:02 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:22:02 -0800
Subject: [Expat-bugs] [ expat-Bugs-2895533 ] found a resource leak
Message-ID: <3V0chD6s8QzP1s@mail.python.org>

Bugs item #2895533, was opened at 2009-11-10 14:38
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2895533&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: martin ettl (ettlmartin)
Assigned to: Nobody/Anonymous (nobody)
Summary: found a resource leak 

Initial Comment:
during a check with the static code analysis tool cppcheck, i found a resource leak, I reported it to the wxWidgets developers. The told me to contact you:
http://trac.wxwidgets.org/ticket/11432
http://trac.wxwidgets.org/ticket/11194

Best regards

Orbitcowboy



----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-12 08:53

Message:
Attached patch was not done against current CVS. Applied modified patch -
see readfilemap.c rev. 1.15.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2895533&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:22:50 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:22:50 -0800
Subject: [Expat-bugs] [ expat-Bugs-3000537 ] expat 2.0.1 unpacking fails on
	ubuntu 10.04, fedora 12/13
Message-ID: <3V0cj83NmNzP6T@mail.python.org>

Bugs item #3000537, was opened at 2010-05-12 09:03
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3000537&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: None
Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: playya (playya)
Assigned to: Nobody/Anonymous (nobody)
Summary: expat 2.0.1 unpacking fails on ubuntu 10.04, fedora 12/13

Initial Comment:
if you try to unpack the tar.gz you get the following error:

gzip: expat-2.0.1.tar.gz: invalid compressed data--crc error

Anyone knows the tool/version which was used to generate the archive to isolate the problem?
Would it be possible to repack the archive?

Additional information:
* This bug only affects 32 bit systems
* launchpad bug: http://bugs.launchpad.net/fedora/+source/gzip/+bug/524366


----------------------------------------------------------------------

Comment By: playya (playya)
Date: 2010-05-17 16:07

Message:
Another malformed archive has been detected.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2010-05-13 13:47

Message:
Following the link in the original post this does not seem an issue with
the archive, but rather with a specific build of gzip.

----------------------------------------------------------------------

Comment By: Christian Anke (carcinoma)
Date: 2010-05-13 12:37

Message:
have the same problem

redownload it, use other mirror, same thing

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2010-05-12 10:55

Message:
Have you tried re-downloading the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3000537&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:24:05 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:24:05 -0800
Subject: [Expat-bugs] [ expat-Bugs-2020141 ] Characters lost during parsing
Message-ID: <3V0ckd2Db7zP6T@mail.python.org>

Bugs item #2020141, was opened at 2008-07-16 18:22
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2020141&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Closed
>Resolution: Rejected
Priority: 5
Private: No
Submitted By: Andrew D. Arenson (arenson9)
Assigned to: Sebastian Pipping (hartwork)
Summary: Characters lost during parsing

Initial Comment:
Characters can be lost during parsing.

I'm going to attach a file. My example file is too big to be included through this interface, so I've made it available at:

http://miniscd.uits.iupui.edu/aarenson/example6.xml

I put the Perl program I used to demonstrate the error as an attachment on this submission, but here it is as well:

#!/usr/bin/perl -w
use XML::Parser;
my $XmlFile = shift @ARGV;
my $xp = new XML::Parser(Handlers => {Start => \&start,
                                      End   => \&end,
                                      Char  => \&cdata});
$xp->parsefile($XmlFile);

sub start { $curTag = lc($_[1]); }

sub end { $curTag = ""; }

sub cdata {
    my ($xp,$data) = @_;

    if ($curTag eq "globalid") { $ID = $data; }

    if ($data eq ".5") {
        print "ID: $ID; TAG: $curTag\n";
    }
}


When I use the above program on the example XML file, the last value in the XML file, '52.5', gets parsed as just '.5'.


I wonder if this is related to something that was reported twelve months ago on the XML::Parser bug list at:

   http://rt.cpan.org/Public/Bug/Display.html?id=28585

That bug report on the XML::Parser bug list has not been opened. It is still listed as New.

I'm sorry I don't know the version number of Expat that I'm using or how to determine it.


----------------------------------------------------------------------

Comment By: Andrew D. Arenson (arenson9)
Date: 2008-08-04 14:11

Message:
Logged In: YES 
user_id=466852
Originator: YES

Thanks! Problem solved.

With your modified show_err2.pl file, I  was still able to demonstrate the
error, but your pointing out the problem (character data handler can
receive the content of an element split among several handler calls) was
what I needed to devise a solution. I've uploaded show_err3.pl in case
anyone reading this is interested in the solution.

For completeness, here's the output when running show_err2.pl:

bash> ./show_err2.pl example6.xml
ID: LRB0008; TAG: ofc; DATA: 52.5
ID: LRB0014; TAG: ofc; DATA: 52
ID: LRB0049; TAG: ofc; DATA: 52.5
ID: LRB0061; TAG: ofc; DATA: 52
ID: LRB0079; TAG: ofc; DATA: 52.5
ID: PMP008; TAG: ofc; DATA: 52.5
ID: PMP017; TAG: ofc; DATA: 52.5
ID: PMP043; TAG: ofc; DATA: 52
ID: PMP043; TAG: ofc; DATA: .5

Here's the output when running show_err3.pl:

bash> ./show_err3.pl example6.xml
ID: LRB0008; TAG: ofc; DATA: 52.5; CC: 0
ID: LRB0014; TAG: ofc; DATA: 52; CC: 0
ID: LRB0049; TAG: ofc; DATA: 52.5; CC: 0
ID: LRB0061; TAG: ofc; DATA: 52; CC: 0
ID: LRB0079; TAG: ofc; DATA: 52.5; CC: 0
ID: PMP008; TAG: ofc; DATA: 52.5; CC: 0
ID: PMP017; TAG: ofc; DATA: 52.5; CC: 0
ID: PMP043; TAG: ofc; DATA: 52; CC: 0
ID: PMP043; TAG: ofc; DATA: 52.5; CC: 1

Basically, my solution was to keep track of how many times the character
data handler has been called since the start handler has been called. If
it's more than one, then I append the most recent character data to the
previously returned character data.

So, why does eXpat work this way? Why not only call the character data
handler once? Is it some sort of efficiency thing when working with
streams? Is it part of wanting to be able to handle values of any size?
File Added: show_err3.pl

----------------------------------------------------------------------

Comment By: Sebastian Pipping (hartwork)
Date: 2008-08-03 07:47

Message:
Logged In: YES 
user_id=1022691
Originator: NO

Hello Andrew. Are you aware that character data handlers can receive
the content of an element split among several handler calls?
For instance your "52.5" text here could come in as two calls serving
"52" first and then ".5".

I modified the script you attached to help tracing this. Please let us
know if this is happening on your machine. (On mine "52.5" is served
as a single unit from the XML file you provided.)

Best regards, Sebastian
File Added: show_err_2.pl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2020141&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:25:07 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:25:07 -0800
Subject: [Expat-bugs] [ expat-Bugs-1837011 ] Unable to find library
	'libexpat.so.6'
Message-ID: <3V0cln5mLTzP0P@mail.python.org>

Bugs item #1837011, was opened at 2007-11-23 03:53
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1837011&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: XML::Parser (inactive)
Group: None
Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: Unable to find library 'libexpat.so.6'

Initial Comment:
email address annemarie.nolan at meteor.ie

ldd -s Expat.so

  find library=libexpat.so.6; required by ./Expat.so
    search path=:/isv/isv_app/expat/usr/local/lib:/isv/isv_app/expat/usr/local/include:/isv/isv_app/expat/usr/local/bin:/opt/hp-gcc-
4.2.1/lib/hpux64:/usr/lib/hpux64  (LD_LIBRARY_PATH)
    trying path=/isv/isv_app/expat/usr/local/lib/libexpat.so.6
    trying path=/isv/isv_app/expat/usr/local/include/libexpat.so.6
    trying path=/isv/isv_app/expat/usr/local/bin/libexpat.so.6
    trying path=/opt/hp-gcc-4.2.1/lib/hpux64/libexpat.so.6
    trying path=/usr/lib/hpux64/libexpat.so.6
    search path=/isv/isv_app/tuxedo/8.1.273/lib:/isv/isv_app/sqr/rel/bin/SQR/Server/Oracle/bin:/isv/isvsbx1b/rel/server/lib:/isv/isv
_app/oracle/product/10.2.0.3/lib::/isv/isvsbx1b/imp/loc/server/lib:/isv/isvsbx1b/imp/rel/server/lib:/isv/isvsbx1b/rel/server/lib32:/
isv/isv_app/oracle/oracle/rel/lib32:/isv/isv_app/tools/bin/../lib:/isv/isv_app/src:/isv/isv_app/expat/usr/local/lib:/isv/isv_app/exp
at/usr/local/include:/isv/isv_app/expat/usr/local/bin:/opt/hp-gcc-4.2.1/lib/hpux64:/usr/lib/hpux64:/opt/hp-gcc-4.2.1/lib  (SHLIB_PAT
H)
    trying path=/isv/isv_app/tuxedo/8.1.273/lib/libexpat.so.6
    trying path=/isv/isv_app/sqr/rel/bin/SQR/Server/Oracle/bin/libexpat.so.6
    trying path=/isv/isvsbx1b/rel/server/lib/libexpat.so.6
    trying path=/isv/isv_app/oracle/product/10.2.0.3/lib/libexpat.so.6
    trying path=/isv/isvsbx1b/imp/loc/server/lib/libexpat.so.6
    trying path=/isv/isvsbx1b/imp/rel/server/lib/libexpat.so.6
    trying path=/isv/isvsbx1b/rel/server/lib32/libexpat.so.6
    trying path=/isv/isv_app/oracle/oracle/rel/lib32/libexpat.so.6
    trying path=/isv/isv_app/tools/bin/../lib/libexpat.so.6
    trying path=/isv/isv_app/src/libexpat.so.6
    trying path=/isv/isv_app/expat/usr/local/lib/libexpat.so.6
    trying path=/isv/isv_app/expat/usr/local/include/libexpat.so.6
    trying path=/isv/isv_app/expat/usr/local/bin/libexpat.so.6
    trying path=/opt/hp-gcc-4.2.1/lib/hpux64/libexpat.so.6
    trying path=/usr/lib/hpux64/libexpat.so.6
    trying path=/opt/hp-gcc-4.2.1/lib/libexpat.so.6
    search path=/usr/local/lib:/usr/lib/hpux64:/sv/app/expat/product/2.0.1_64/lib  (RUNPATH)
    trying path=/usr/local/lib/libexpat.so.6
    trying path=/usr/lib/hpux64/libexpat.so.6
    trying path=/sv/app/expat/product/2.0.1_64/lib/libexpat.so.6
    search path=/usr/lib/hpux64  (default)
    trying path=/usr/lib/hpux64/libexpat.so.6
        Unable to find library 'libexpat.so.6'.

Expat was installed to the /isv/isv_app/expat directory.

Below, you can see that libexpat.so. exists.
$ ll
total 2180
drwxr-xr-x   2 isv_app    isv           1024 Nov 22 23:05 ./
drwxr-xr-x   6 isv_app    isv             96 Nov 22 23:05 ../
-rw-r--r--   1 isv_app    isv         579680 Nov 22 23:05 libexpat.a
-rwxr-xr-x   1 isv_app    isv            797 Nov 22 23:05 libexpat.la*
lrwxr-xr-x   1 root       sys             15 Nov 22 23:05 libexpat.so@ -> libexpat.so.6.2
lrwxr-xr-x   1 root       sys             15 Nov 22 23:05 libexpat.so.6@ -> libexpat.so.6.2
-r-xr-xr-x   1 isv_app    isv         533240 Nov 22 23:05 libexpat.so.6.2*

The SHLIB_PATH variable has been configured to include the path containing the above file.
Why would the library file not be detected?




----------------------------------------------------------------------

Comment By: SourceForge Robot (sf-robot)
Date: 2008-08-23 19:20

Message:
Logged In: YES 
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 20 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: Sebastian Pipping (hartwork)
Date: 2008-08-03 08:11

Message:
Logged In: YES 
user_id=1022691
Originator: NO

In the output your provived I see the text
":/isv/isv_app/expat/usr/local/lib:".
To me this clearly looks like there is a colon missingin between -
shouldn't it read
":/isv/isv_app/expat:/usr/local/lib:" instead?

Please let us know if that helps.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1837011&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:27:04 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:27:04 -0800
Subject: [Expat-bugs] [ expat-Bugs-1836891 ] compile problem on hpux11.23.
	Urgent!
Message-ID: <3V0cp40npgzP9D@mail.python.org>

Bugs item #1836891, was opened at 2007-11-22 23:44
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1836891&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: www.libexpat.org
Group: Platform Specific
Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Arthur Zhu (arthurzhu)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: compile problem on hpux11.23. Urgent!

Initial Comment:
My server is: HP Integrity rx2620 hpux11.23.

I want to compile one source which needs to link libexpat.a on hpux. The message always as following:
ld: Can't find library or mismatched ABI for -lexpat
Fatal error.
*** Error exit code 1

Stop.
*** Error exit code 1

Stop.
*** Error exit code 1

I downloaded a version from http://hpux.cs.utah.edu/ which was special for hpux11.23. After installing expat, I copied the libexpated to my source's lib folder, compiled again, also couldn't find expat library.

Then, I downloaded expat source. First, did ./configure, then did make. When making, message was:
ld: Unknown input file type: "./.libs/libexpat.so"
Fatal error.
*** Error exit code 1
Under .libs folder, libexpat.a and libexpat.so had been created.
I copied this licexpat.a to my lib folder, also couldn't use.

The same source, I compiled on AIX5.3, Linux, Windows.... no problem.

Anybody could explain? Kindly thanks!

Kind Regards,
Arthur Zhu

----------------------------------------------------------------------

Comment By: Arthur Zhu (arthurzhu)
Date: 2007-11-26 19:37

Message:
Logged In: YES 
user_id=1943705
Originator: YES

Hi Mauad,

I'm so glad to hear from you.

I also used sqlite. But I'm not familiar with it.
My projects are usually under Windows Platform. But the project needs to
run under Linux, AS400, Solaris, hpux, AIX, Windows. I'm not famaliar with
Unix and Linux.
My msn: dongpaidoudou at msn.com
 Email: arthurzhuwei at gmail.com or gongxifacai at yahoo.cn

Could you tell me your msn or mail address? If I have some problem, I think
I could get help from you.
Thanks!

Kind Regards,
Arthur Zhu

----------------------------------------------------------------------

Comment By: mauad (mauad)
Date: 2007-11-26 17:57

Message:
Logged In: YES 
user_id=1516525
Originator: NO

Hi Arthur!

Thanks! 


I'm sorry, but I had not figured out that your CC was cc. Mine was gcc ;)
I had a similar problem with sqlite some time ago. Subtle gotchas.


Best regards.
Mauad, from Porto Alegre, Brazil

----------------------------------------------------------------------

Comment By: Arthur Zhu (arthurzhu)
Date: 2007-11-26 17:07

Message:
Logged In: YES 
user_id=1943705
Originator: YES

Hi, mauad,

I'm so glad to get response from you.
I found the problem.
First, the depot is ok.
But when I compiled source(Version2.0.1), error occured.
My step was:
./configure
make // when making, can't find libexpat.a.
I checked the "Makefile". Changed the line 106: from "CC = cc" to "CC = cc
+DSitanium2 + DD64 +w1". Then make again, also had some problem. But the
libexpat.a could be used.
I thought that's the problem. If I didn't change "Makefile", compiled
libexpat.a is 32bit, after changed, compiled libexpat.a is 64bit.

Kind Regards,
Arthur Zhu in Shanghai

----------------------------------------------------------------------

Comment By: mauad (mauad)
Date: 2007-11-26 14:51

Message:
Logged In: YES 
user_id=1516525
Originator: NO

This looks weird.

After downloading sources, I'd built expat on an Itanium server using the
following setup:

* gcc -v:
Using built-in specs.
Target: ia64-hp-hpux11.23
Configured with: ../gcc/configure
Thread model: posix
gcc version 4.1.1

GNU Make 3.81

After building the vanilla version (running ./configure)
with:
* make 
* make install

the files under examples/ were built, and the uutput was the expected upon
some XML files.

Note that the propper .so was built: 
* ldd /usr/local/lib/libexpat.so

/usr/local/lib/libexpat.so:     ELF-32 shared object file - IA64


It seems that your HPUX depot is influencing the linker.

Try out this steps:

1. Check which depot was downloaded (look for the IA64 one! there is a PA
and a IA64 version for 11.23) 
   with file $PATH_TO_EXPAT_LIB/libexpat.so

2(a). If the depot is wrong, you don't have a bug
2(b). If the depot is ok, uninstall it before rebuilding
3. Rebuild expat from scratch with ./configure, make make install
5. Change to examples/ and execute gcc -o elements elements.c -lexpat
6. Try it out with elements < file.xml

Mauad

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1836891&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:36:06 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:36:06 -0800
Subject: [Expat-bugs] [ expat-Bugs-1647805 ] Expat 2.0 does not build on Mac
	OS X 10.4 / intel
Message-ID: <3V0d0W2pYGzP7Q@mail.python.org>

Bugs item #1647805, was opened at 2007-01-30 03:25
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1647805&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Test Required
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Matthias Wiesmann (wiesmann)
Assigned to: Greg Stein (gstein)
Summary: Expat 2.0 does not build on Mac OS X 10.4 / intel

Initial Comment:
Installing expat 2.0.0 on Mac OS X (i386-apple-darwin8.8.2) fails with the following error:

/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H   -o lib/xmlparse.lo -c lib/xmlparse.c
lib/xmlparse.c:77:2: error: #error memmove does not exist on this platform, nor is a substitute available
make: *** [lib/xmlparse.lo] Error 1

Output of configure script:

checking build system type... i686-apple-darwin8.8.2
checking host system type... i686-apple-darwin8.8.2
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for a sed that does not truncate output... /usr/bin/sed
checking for egrep... grep -E
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... no
checking for /usr/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -p
checking whether ln -s works... yes
checking how to recognise dependent libraries... pass_all
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking for g77... no
checking for f77... no
checking for xlf... no
checking for frt... no
checking for pgf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for f90... no
checking for xlf90... no
checking for pgf90... no
checking for epcf90... no
checking for f95... no
checking for fort... no
checking for xlf95... no
checking for ifc... no
checking for efc... no
checking for pgf95... no
checking for lf95... no
checking for gfortran... no
checking whether we are using the GNU Fortran 77 compiler... no
checking whether  accepts -g... no
checking the maximum length of command line arguments... 196608
checking command to parse /usr/bin/nm -p output from gcc object... ok
checking for objdir... .libs
checking for ar... ar
checking for ranlib... ranlib
checking for strip... strip
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fno-common
checking if gcc PIC flag -fno-common works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking whether the gcc linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin8.8.2 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
configure: creating libtool
appending configuration tag "CXX" to libtool
checking for ld used by g++... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... no
checking whether the g++ linker (/usr/bin/ld) supports shared libraries... yes
checking for g++ option to produce PIC... -fno-common
checking if g++ PIC flag -fno-common works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking whether the g++ linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin8.8.2 dyld
checking how to hardcode library paths into programs... immediate
appending configuration tag "F77" to libtool
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ANSI C... (cached) none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether gcc accepts -fexceptions... yes
checking for ANSI C header files... (cached) yes
checking whether byte ordering is bigendian... no
checking for an ANSI C-conforming const... yes
checking for size_t... yes
checking for memmove... no
checking for bcopy... no
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for unistd.h... (cached) yes
checking for off_t... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for getpagesize... no
checking for working mmap... no
checking for an ANSI C99-conforming __func__... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating expat_config.h
config.status: expat_config.h is unchanged


----------------------------------------------------------------------

Comment By: Sebastian Pipping (hartwork)
Date: 2008-08-03 09:03

Message:
Logged In: YES 
user_id=1022691
Originator: NO

Matthias, is this issue still valid?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-05-09 09:58

Message:
Logged In: YES 
user_id=290026
Originator: NO

Matthias,

have you tried the latest configure.in from CVS?
Just to make sure this didn't fix your problem.

----------------------------------------------------------------------

Comment By: Matthias Wiesmann (wiesmann)
Date: 2007-05-09 09:23

Message:
Logged In: YES 
user_id=117374
Originator: YES

I dont' think the problem is related to the build environnement. I have
basically the same environnement (MacPorts 1.440) on a PPC PowerBook and an
Intel Macbook. Expat builds fine on the PPC and fails on the Intel. The
fact that the same problem arises with both ICC and GCC would indicate to
me that configure has trouble with Darwin/Intel.

I have attached the config.log file to this report.
File Added: config.log

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-05-09 06:28

Message:
Logged In: YES 
user_id=290026
Originator: NO

Thanks again for you exemplary feedback!

> This code checks if GCC is present (using $GCC which is set by
> macro AC_PROG_CC which tests for the presence of __GNUC__) and
> adds some compilation options including -fexceptions
> (BTW: it would be ideal to know why this option is given as\
> someone has already said that it is not such a good idea). 

I think the reason it to better interoperate with call-backs
that throw exceptions, so that the exceptions can pass though
to the original caller of XML_Parse(). Expat is called from other
languages as well!

> The latter brings up the question: Why is the CFLAGS variable
> being modified in configure.in? Was whoever added it expecting
> that the compilation options would be used to compile the sources
> or did he/she know that these compilation options would affect
> the compilation of the sources as well as programs to test for
> C features (configure)?

The CFLags can be modified as arguments to configure in order to
build the UTF-16 version of Expat (libexpatw.so/dll). (see README).

We are not using automake, because our main build expert (Greg Stein)
has too many bad experiences with it, and for Expat he considers
it overkill.

I prefer your first fix option, and have applied it to
configure.in rev. 1.45.

Needs testing on other platforms.


----------------------------------------------------------------------

Comment By: Hazael (hmaldonado)
Date: 2007-05-09 06:00

Message:
Logged In: YES 
user_id=1788089
Originator: NO

Hi there

I am not a UNIX build expert either but I think we can come up with some
solutions. Let me explain:

Firstly, I have already filed a report on the Intel Support Site and as of
today they have updated the report with the following text:

"Thank you for the detail report. I was able to reproduce the problem, and
the issue have been escalated. I will update this issue after our
investigation.

Intel Customer Support"

As you can see they have not given away so much information other that the
report has been escalated (whatever that means in their system).

Secondly, the main thing is that Intel wants its compiler to be 100%
compatible with gcc so anyone can easily migrate from gcc to icc. For this
particular option -fexceptions icc is failing to do so. The way icc
interoperate with gcc is by sort of posing as a gcc compiler. icc
predefined some macros that are predefined by gcc (__GNUC__,
__GNUC_MINOR__, __GNUC_PATCHLEVEL__) so icc can be seen as gcc.

After looking at expat's configure.in I found the following code:

if test "$GCC" = yes ; then
    dnl
    dnl Be careful about adding the -fexceptions option; some versions of
    dnl GCC don't support it and it causes extra warnings that are only
    dnl distracting; avoid.
    dnl
    OLDCFLAGS="$CFLAGS -Wall -Wmissing-prototypes -Wstrict-prototypes"
    CFLAGS="$OLDCFLAGS -fexceptions"
    AC_MSG_CHECKING(whether gcc accepts -fexceptions)
    AC_TRY_COMPILE(,(void)1,
                   AC_MSG_RESULT(yes),
                   AC_MSG_RESULT(no); CFLAGS="$OLDCFLAGS")
fi

This code checks if GCC is present (using $GCC which is set by macro
AC_PROG_CC which tests for the presence of __GNUC__) and adds some
compilation options including -fexceptions (BTW: it would be ideal to know
why this option is given as someone has already said that it is not such a
good idea). The Intel Manual advices to set CC to icc and when autoconf is
used to generate the configure script AC_PROG_CC will set GCC=yes (because
icc predefines __GNUC__) and also set leaves CC untouched (CC=icc) (as
indicated in the autoconf 2.61 manual).

The code above will execute as GCC=yes is true and because icc does accept
the option -fexceptions (although it does not handle it as expected)
therefore the -fexceptions option is added to CFLAGS which, according with
the autoconf manual, is used when compiling or linking programs to test for
C features.

The latter brings up the question: Why is the CFLAGS variable being
modified in configure.in? Was whoever added it expecting that the
compilation options would be used to compile the sources or did he/she know
that these compilation options would affect the compilation of the sources
as well as programs to test for C features (configure)?

To the best of my understanding if one wants to compile the sources (e.g.
lib/xmlparse.c) with some specific options then those options are given in
the Makefile.am (if automake is being used) located in the directory where
the sources are. The presence of CFLAGS in the configure.in may be because
expat is not using automake (as far as I can see). However, I believe
CFLAGS can still be given somewhere else.

Then, our first problem is that the programs to test for C features are
failing (memmove, bcopy, etc.) when compiling with icc because CFLAGS
contains the -fexceptions option. There are different ways to solve this:

a. Replace the following code at configure.in:
if test "$GCC" = yes ; then
    dnl
    dnl Be careful about adding the -fexceptions option; some versions of
    dnl GCC don't support it and it causes extra warnings that are only
    dnl distracting; avoid.
    dnl
    OLDCFLAGS="$CFLAGS -Wall -Wmissing-prototypes -Wstrict-prototypes"
    CFLAGS="$OLDCFLAGS -fexceptions"
    AC_MSG_CHECKING(whether gcc accepts -fexceptions)
    AC_TRY_COMPILE(,(void)1,
                   AC_MSG_RESULT(yes),
                   AC_MSG_RESULT(no); CFLAGS="$OLDCFLAGS")
fi

>> WITH <<

if test "$GCC" = yes ; then
    dnl
    dnl Be careful about adding the -fexceptions option; some versions of
    dnl GCC don't support it and it causes extra warnings that are only
    dnl distracting; avoid.
    dnl
    OLDCFLAGS="$CFLAGS -Wall -Wmissing-prototypes -Wstrict-prototypes"
    CFLAGS="$OLDCFLAGS -fexceptions"
    AC_MSG_CHECKING(whether $CC accepts -fexceptions)
    AC_TRY_LINK( , ,
                   AC_MSG_RESULT(yes),
                   AC_MSG_RESULT(no); CFLAGS="$OLDCFLAGS")
fi

The only two lines that change are AC_MSG_CHECKING... and AC_TRY_COMPILE...
which now is using the macro AC_TRY_LINK. The reason for this is that
-fexceptions only makes the Intel compiler fails when linking and not when
compiling. This is the reason why the -fexceptions options is added when
icc is used. By changing AC_TRY_COMPILE to AC_TRY_LINK (which implies a
AC_TRY_COMPILE) the test will fail when using -fexceptions and this will
prevent the -fexceptions option to be added. Also this solution has
backwards compatibility with other GCC versions and when Intel fixes icc
the -fexceptions option will be added as it would be supported.

b. This solution involves to make a proper distinction between icc and gcc
regarding the effords of icc to be seen as gcc. The line:
if test "$GCC" = yes ; then

can be changed for something that makes sure that the compiler is truly the
gcc and not something else. One way would be to check is the
__INTEL_COMPILER__ macro is defined. Then if this macro is defined and GCC
= yes then we know that it is not the gcc compiler but something that is
compatible.

You folks are the right people to decide which way to go as you know the
reasons why expat does not use automake, why -fexceptions is needed, etc.

I have tried solution (a) and it worked fine on my system (of course I had
to run autoconf after changing configure.in so that configure was updated)
without any further changes. The configure script detects icc as my default
compiler, identifies the -fexceptions option as NOT supported by icc and,
finally, finds memmove and bcopy.

Regards

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-05-08 16:12

Message:
Logged In: YES 
user_id=290026
Originator: NO

Thanks for the nice analysis!

Now, the question is (not being a Unix build expert), what can be done
at the configuration level to deal with that in Expat?

----------------------------------------------------------------------

Comment By: Hazael (hmaldonado)
Date: 2007-05-08 03:38

Message:
Logged In: YES 
user_id=1788089
Originator: NO

I have the same problem on a combination of Mac OS X 10.4.9 + expat
(Macport version) + Intel C/C++ Compiler fo Mac.

I really don't know who this error concerns to but it seems it is related
with the Intecl C/C++ compilers more than anything else. The problem arises
because the configure script can not find memmove and bcopy. I have
extracted the code generated by autoconf to test for the presence of
memmove and tried to compile it with gcc and icc. GCC has not problems
compiling it which as expected means that expat can be compiled with gcc
without any trouble.

However, ICC (Intel compiler) can not compile the code which means that the
configure test will fail and memmove is considered as not present in the
system which in turn triggers compilation errors.

autoconf uses the following compiler options to compile the test code:

icc -o conftest -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions
ac_check_memmove.c

And according to my research -fexceptions is the option making icc to fail.
According to gcc documentation the -fexceptions option can make gcc to link
the code to libstdc++ even when it is not a C++ program. However, it seems
that icc does not do the same and outputs the following messages:
ac_check_memmove.c(49): warning #310: old-style parameter list
(anachronism)
  char memmove ();
       ^

ac_check_memmove.c(49): warning #1419: external declaration in primary
source file
  char memmove ();
       ^

ac_check_memmove.c(67): remark #111: statement is unreachable
    return 0;
    ^

ld: Undefined symbols:
___gxx_personality_v0


The undefined symbol ___gxx_personality_v0 is, not surprisely, found at the
libstdc++ library which is automatically included by gcc (trigger by the
-fexceptions option) but not included by icc. If the option -lstdc++ is
given manually to icc then the compilation succeeds.

I think this explains the error and a quick solution for compiling expat
will be adding -lstdc++ to the LDFLAGS environment variable at
configuration time.

LDFLAGS="-lstdc++" ./configure --prefix=/path/to/


----------------------------------------------------------------------

Comment By: Sebastian Pipping (hartwork)
Date: 2007-05-02 11:55

Message:
Logged In: YES 
user_id=1022691
Originator: NO

 
 
wiesmann, please let us know if it was your
build environment's fault so we know if we
can close the bug. Thank you!



Sebastian
 
 

----------------------------------------------------------------------

Comment By: Elias Pipping (pipping)
Date: 2007-05-02 06:20

Message:
Logged In: YES 
user_id=1697847
Originator: NO

> I am using Mac ports (MacPorts 1.320). I know that expat builds
correctly
> on PPC machines, as I use it on my G4 Powerbook.

I'm on an intel mac, too (see above).

----------------------------------------------------------------------

Comment By: Elias Pipping (pipping)
Date: 2007-05-02 06:17

Message:
Logged In: YES 
user_id=1697847
Originator: NO

You might want to consider running a selfupdate via 'sudo port selfupdate'
to get MacPorts 1.4.3. The portfile itself basically hasn't changed for
nearly a year, but the base-code has (significantly).

Also, make sure you have the latest version of the Developer Tools
installed, (on an intel mac the output of 'gcc --version' should be:
i686-apple-darwin8-gcc-4.0.1 ... build 5367)


----------------------------------------------------------------------

Comment By: Matthias Wiesmann (wiesmann)
Date: 2007-05-02 06:08

Message:
Logged In: YES 
user_id=117374
Originator: YES

I am using Mac ports (MacPorts 1.320). I know that expat builds correctly
on PPC machines, as I use it on my G4 Powerbook.

----------------------------------------------------------------------

Comment By: Elias Pipping (pipping)
Date: 2007-05-02 05:55

Message:
Logged In: YES 
user_id=1697847
Originator: NO

works fine on my mac (Mac OS X 10.4.9, i386-apple-darwin8.9.1, latest
version from cvs, developer tools 2.4.1)

/bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2
-Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DHAVE_EXPAT_CONFIG_H -o lib/xmlparse.lo -c lib/xmlparse.c
/bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2
-Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DHAVE_EXPAT_CONFIG_H -o lib/xmltok.lo -c lib/xmltok.c
...



----------------------------------------------------------------------

Comment By: Sebastian Pipping (hartwork)
Date: 2007-04-27 02:56

Message:
Logged In: YES 
user_id=1022691
Originator: NO

Have you tried the MacPorts port for Expat?
http://trac.macports.org/projects/macports/browser/trunk/dports/textproc/expat/Portfile



Sebastian


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1647805&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:36:51 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:36:51 -0800
Subject: [Expat-bugs] [ expat-Bugs-1515266 ] missing check of stopped parser
	in doContent() 'for' loop
Message-ID: <3V0d1L324WzPBl@mail.python.org>

Bugs item #1515266, was opened at 2006-06-30 11:04
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1515266&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Brett Cannon (bcannon)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: missing check of stopped parser in doContent() 'for' loop

Initial Comment:
In Expat 2.0.0, in expat.c:doConvert() there is a 'for'
loop for the XML_TOK_DATA_CHARS case.  There is
unfortunately no check in that loop whether the parser
was stopped during that call because of an error.

This was discovered in Python
(Lib/test/crashers/xml_parsers.py) because pyexpat,
upon error where there is no error return code like
with characterDataHandlers, sets all handlers to 0,
sets parsingStatus to XML_FINISHED, and sets errorCode.
 This leads to a segfault if the 'for' loop goes around
again because parser->m_characterDataHandler is set to 0.

A simple check if the parser is stopped fixes the
problem.  I have attached a simple patch that just
breaks out of the loop and lets execution fall through
to the bottom of the 'switch' statement.  I don't know
if returning errorCode directly would be better or if
checking for XML_SUSPENDED is also desirable.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-17 08:09

Message:
Comment for Fred:
So, do we need to address this last comment - i.e. that Python is
supposedly broken with release 2.0.1?

Karl

----------------------------------------------------------------------

Comment By: David Leverton (dleverton)
Date: 2008-05-24 16:50

Message:
Logged In: YES 
user_id=2011415
Originator: NO

> Python compatibility still needs testing.

This does indeed break Python: when it clears the handler, it sets the
variable containing the Python-side handler to NULL, and then assumes that
said variable is non-null when expat calls the old handler.

(The vanilla Python distribution uses a bundled copy of expat from before
this change was made, which is probably why no-one's complained until now,
but some Linux distros like to use the system copy instead of the bundled
one.)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-11-25 09:41

Message:
Logged In: YES 
user_id=290026
Originator: NO

Is anyone testing CVS for this solution?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-10 12:02

Message:
Logged In: YES 
user_id=290026

Applied the patch preserving default handler failover.
See xmlparse.c rev. 1.158. Docs updated as well.
Python compatibility still needs testing.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-06 10:19

Message:
Logged In: YES 
user_id=290026

I am attaching a patch to current CVS that preserves the
default handler failover logic by saving the character data
handler to a local variable instead of moving the NULL check
into the inner for loop (file "localCharDataHandlerPatch.diff").

The drawback: Even if the handler is cleared, it will be
called back on as long as the inner for loop is active.

Could be a problem for Python, if it cannot deal with a few
more call-backs despite clearing the handlers.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-07-06 10:03

Message:
Logged In: YES 
user_id=357491

Yes, I'm listening, Fred.  =)

If you look at PEP 356
(http://www.python.org/dev/peps/pep-0356/) it seems like b2
is due on July 12 and rc1 August 1.  So there is still time
to get whatever change/fix needed to Python's wrapper before
we hit final release.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-06 05:56

Message:
Logged In: YES 
user_id=290026

One way to preserver the old default handler logic would be
this:

Revert back to the original code, but save the character
data handler into a local variable for the duration of the
inner for loop. This would prevent the segfault, but would
enforce the call-backs in the loop to go on until the loop
terminates, even if the character data handler was cleared.

I personally like this solution, but the question is how
Python could handle it if there were more call-backs even
after the handlers were cleared.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-07-05 22:15

Message:
Logged In: YES 
user_id=3066

Python (on the trunk) is no longer quite as sensitive to the
Expat implementation for this, so that's not a source of
time pressure to come up with the final fix for this.

Reducing priority back to "Medium"

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-07-05 20:23

Message:
Logged In: YES 
user_id=3066

The tests now pass, but agree that the lack of falling back
to the default handler is undesirable.  As noted, I'm not
sure how much we want to worry about this in code, though,
rather than through documentation.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-05 19:55

Message:
Logged In: YES 
user_id=290026

The bug is quite obvious when you look at it.
When the character data handler is cleared, the for loop
will do nothing forever. Please check again with xmlparse.c
rev. 1.157. 

However this quick fix is not quite satisfying. There is one
piece of logic that becomes ill-fitted now: the "fail-over"
to the default handler does not work as expected anymore, so
I'll have some more thinking to do.


----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-07-05 19:38

Message:
Logged In: YES 
user_id=3066

The tests no longer complete, but take up all the CPU the
system will let them have.  Hallmarks of an infinite loop,
if you ask me.  :-)

Are you able to run the tests on Windows?  I don't know if a
MSVC++ target was ever built for them, and don't have access
to a Windows development machine most of the time.

One thing that can be done is to document that the character
data handler can't be removed (though it can be replaced),
during parsing, except from some non-character data (and
non-decoding-related) handler.  Then the Python bindings can
use an alternate approach, replacing the character handler
with a completely no-op handler until it can be safely
removed completely.

Brett, are you still paying attention?  I can make the
needed changes to the Python bindings to isolate those from
some of the changes in Expat, hopefully no later than
sometime this weekend.  Not sure what the release schedule
is, though.

Karl, I'm generally inclined to make Expat as safe from
segfaults as possible, so I'd like things to "just work" in
even some of the oddball scenarios that exception-handling
wrappers built to support scripting languages might present,
though I don't object to making them go through a bit of
extra work.  I know our main audience is very
performance-sensitive, so I don't want to pay too high a
cost on that front.  It might be worth taking the discussion
of alternatives to the mailing list, but I vaguely recall
that we've done that before.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-05 06:26

Message:
Logged In: YES 
user_id=290026

Corrected Summary.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-05 06:14

Message:
Logged In: YES 
user_id=290026

Applied the patch for bug # 1515600 which solves this issue
as well. Removed the check for XML_FINISHED/XML_SUSPENDED.
We could discuss special treatment of XML_FINISHED, but if
one is clearing all handlers anyway, then special treatment
of XML_FINISHED is not necessary.

For Fred: I have not re-run the test cases. Please do so and
close the issue if successful.



----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2006-07-04 06:37

Message:
Logged In: YES 
user_id=290026

I am re-opening this issue because in the case of a
suspended parser, breaking out of the inner loop in
XML_TOK_DATA_CHARS means that character call-backs are
missed when resuming the parser. We should let the inner
loop finish reporting all characters.

The documentation already states that after calling
XML_StopParser() there may still be a few call-backs that
would otherwise be missed, so this would not be new
behaviour, but consistent with existing behaviour.

The solution to the problem described is the same as
suggested for bug # 1515600 (Segfault after removing
character data handler). Just put the NULL check for the
character data handler inside the internal loop.

Btw, the same problem exists in the doCdataSection()
function. I'll attach a patch suggestion to bug # 1515600.

We might decide to treat XML_FINISHED different from
XML_SUSPENDED such that no other call-backs will happen, but
in that case we need to review all the other places where
this would need to be done as well (and update the
documentation, of course).

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-07-01 08:32

Message:
Logged In: YES 
user_id=3066

Confirmed that the suspend behavior parallels the abort
behavior Brett's patch fixed; fixed and added a regression
test in lib/xmlparse.c 1.155 and tests/runtests.c 1.66.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-07-01 08:02

Message:
Logged In: YES 
user_id=3066

Added a regression test in tests/runtests.c revision 1.65.

Closing this report.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-06-30 21:00

Message:
Logged In: YES 
user_id=3066

That seems fine, but can be done faster within the Expat
implementation.  I've committed the simplified patch as
lib/xmlparse.c revision 1.154.

I'll have a test case committed tomorrow as well.  Leaving
this report open for now since I need to finish up the test
case.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2006-06-30 11:40

Message:
Logged In: YES 
user_id=3066

The Python folks need this dealt with before Python 2.5, so 
I'll try and take a look at it this weekend if no one beats 
me to it.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1515266&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:38:18 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:38:18 -0800
Subject: [Expat-bugs] [ expat-Bugs-1785430 ] expat build fails on
	linux-amd64 with gcc version>=4.1 -O3
Message-ID: <3V0d300s1CzP7p@mail.python.org>

Bugs item #1785430, was opened at 2007-08-31 01:50
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1785430&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Platform Specific
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Karl Waclawek (kwaclaw)
Summary: expat build fails on linux-amd64 with gcc version>=4.1 -O3

Initial Comment:
Hi,

xmlparse.c contains a cast from const XML_Char* to XML_Char* which newer versions of gcc don't like at all.

cc1: warnings being treated as errors                                                           
src/external/expat-2.0.0/lib/xmlparse.c: In function 'doProlog':                                
src/external/expat-2.0.0/lib/xmlparse.c:3771: warning: passing                                  
argument 1 of 'normalizePublicId' discards qualifiers from pointer target type                  
make: *** [obj/src/external/expat-2.0.0/lib/xmlparse.o] Error 1                                 
 
Thus I devised the attached work-around.

Regards, Thomas Jahns

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-17 08:40

Message:
Is anyone testing this? I'd like to close this issue.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-09-03 09:35

Message:
Logged In: YES 
user_id=290026
Originator: NO

Fixed in xmlparse.c 1.1.62. Please check out from CVS and test!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1785430&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:44:14 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:44:14 -0800
Subject: [Expat-bugs] [ expat-Bugs-2517962 ] Please update libtool scripts
	to support GNU/k*BSD
Message-ID: <3V0d9s0PjFzP88@mail.python.org>

Bugs item #2517962, was opened at 2009-01-18 12:54
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517962&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Platform Specific
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Daniel Leidert (dleidert)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: Please update libtool scripts to support GNU/k*BSD

Initial Comment:
In the Debian bug report #342684 [1] it has been pointed out, that expat 2.0.1 shipped too old libtool scripts to support GNU/k*BSD. Please update them with the next release.

[1] http://bugs.debian.org/342684


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:44

Message:
Modified build system to use autoreconf -fvi instead of buildconf.sh.
(buildconf.sh now just calls autoreconf)
Needs testing

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2010-01-27 06:15

Message:
I spent some time trying to update to the latest libtool, and that's
certainly a losing battle.

Hopefully someone with enough autohell-fu will provide a patch.

----------------------------------------------------------------------

Comment By: Keith Bowes (zooplah)
Date: 2009-12-07 16:33

Message:
This is really imperative to fix (and I saw I patch had been submitted),
because I had to downgrade libtool to 1.5.x to even compile the latest CVS
version.  In my experience with Linux, you need to upgrade at least every
two years or less and less software works, so it's going not going to be
long before the vast majority of users find getting the latest secure
version a major nuisance or perhaps even an impossibility.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-18 16:07

Message:
I have already updated config.sub/guess to the most recent available
version.
I suggest we then use ltmain.sh and libtool.m4 from libtool 1.5.26 for the
next Expat release.

----------------------------------------------------------------------

Comment By: Daniel Leidert (dleidert)
Date: 2009-01-18 14:33

Message:
We updated to the latest version of the 1.x release of libtool. The
following files have been updated: config.sub, config.guess,
conftools/ltmain.sh, conftools/libtool.m4. Then configure has been
recreated.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-18 14:28

Message:
According to the Debian bug report, it might be sufficient to update only
config.sub and config.guess. 

If libtool needs to be updated as well (currently at 1.5.22), it seems
there is a big version jump from 1.5.x to 2.2.x necessary if going to the
most recent version. Any opinions on that?

----------------------------------------------------------------------

Comment By: Daniel Leidert (dleidert)
Date: 2009-01-18 14:21

Message:
This could be a duplicate of item 1983953.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517962&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:45:52 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:45:52 -0800
Subject: [Expat-bugs] [ expat-Bugs-2649838 ] [PATCH] Fix build with libtool
	2.x
Message-ID: <3V0dCk5PMLzNt2@mail.python.org>

Bugs item #2649838, was opened at 2009-02-28 14:14
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2649838&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andreas F?rber (afaerber)
Assigned to: Nobody/Anonymous (nobody)
Summary: [PATCH] Fix build with libtool 2.x

Initial Comment:
libtool 2.x no longer uses a monolithic libtool.m4. buildconf.sh and configure.in need to be updated to copy all relevant files, or autoconf fails to generate configure for CVS HEAD.

Note that the attached patch also contains a local buildconf.sh change (--force in place of --copy) that need not be applied.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:45

Message:
Instead: Modified build system to use autoreconf -fvi instead of
buildconf.sh.
(buildconf.sh now just calls autoreconf)
Needs testing.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2649838&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:46:08 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:46:08 -0800
Subject: [Expat-bugs] [ expat-Bugs-2649838 ] [PATCH] Fix build with libtool
	2.x
Message-ID: <3V0dD22lrtzNt2@mail.python.org>

Bugs item #2649838, was opened at 2009-02-28 14:14
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2649838&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Platform Specific
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andreas F?rber (afaerber)
Assigned to: Nobody/Anonymous (nobody)
Summary: [PATCH] Fix build with libtool 2.x

Initial Comment:
libtool 2.x no longer uses a monolithic libtool.m4. buildconf.sh and configure.in need to be updated to copy all relevant files, or autoconf fails to generate configure for CVS HEAD.

Note that the attached patch also contains a local buildconf.sh change (--force in place of --copy) that need not be applied.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:45

Message:
Instead: Modified build system to use autoreconf -fvi instead of
buildconf.sh.
(buildconf.sh now just calls autoreconf)
Needs testing.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2649838&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:52:03 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:52:03 -0800
Subject: [Expat-bugs] [ expat-Bugs-2517938 ] xmlwf should return non-zero
	exit status if not well-formed
Message-ID: <3V0dLs3z9CzP1T@mail.python.org>

Bugs item #2517938, was opened at 2009-01-18 12:40
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517938&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Daniel Leidert (dleidert)
Assigned to: Karl Waclawek (kwaclaw)
Summary: xmlwf should return non-zero exit status if not well-formed

Initial Comment:
In Debian bug #82763 a user requested to stop at the first not well-formed document with a non-zero exit status. The patch is attached. We apply it for Debian atm.

[1] http://bugs.debian.org/82763

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-18 16:20

Message:
It seems this is indeed a bug.
In other error cases (out of memory, file open error), xmlwf returns 1.
When incorrect parameters are passed, it returns 2. 

The Debian patch returns 2 for a parse error. This seems to be in line with
existing usage, where 1 indicates an exception (execution error), and 2
indicates an actual execution result.

So I applied the patch - see xmlwf.c rev. 1.75.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517938&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:53:17 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:53:17 -0800
Subject: [Expat-bugs] [ expat-Bugs-2517946 ] Wrong statement about XMLDecl
	in xmlwf.1 and xmlwf.sgml
Message-ID: <3V0dNJ0ffGzP1T@mail.python.org>

Bugs item #2517946, was opened at 2009-01-18 12:46
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517946&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Daniel Leidert (dleidert)
Assigned to: Karl Waclawek (kwaclaw)
Summary: Wrong statement about XMLDecl in xmlwf.1 and xmlwf.sgml

Initial Comment:
The manual page (and its source xmlwf.sgml) state in the BUGS section, that an XML document without an XML declaration is not considered well-formed. This is wrong. See

http://www.w3.org/TR/2006/REC-xml-20060816/#NT-prolog

The attached patch should fix this. Please don't forget to recreate the manual page xmlwf.1 as the patch only applies to its source.

This has been reported as Debian bug #412786:
http://bugs.debian.org/412786

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-18 13:55

Message:
Patch applied in xmlwf.sgml rev. 1.5.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2517946&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 20:56:34 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 11:56:34 -0800
Subject: [Expat-bugs] [ expat-Bugs-2894085 ] expat: buffer over-read and
	crash in big2_toUtf8()
Message-ID: <3V0dS61dsqzP1T@mail.python.org>

Bugs item #2894085, was opened at 2009-11-08 03:06
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2894085&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
Private: Yes
Submitted By: Jan Lieskovsky (iankko)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: expat: buffer over-read and crash in big2_toUtf8() 

Initial Comment:
Hello SourceForge expat maintainers,

  originally CVE-2009-3720 was reported in expat:
  [1]  http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2009-3720

  Non-public, original bug report for CVE-2009-3720:
  [2] http://sourceforge.net/tracker/?func=detail&aid=1990430&group_id=10127&atid=110127

  And relevant patch for CVE-2009-3720:
  [3] http://expat.cvs.sourceforge.net/viewvc/expat/expat/lib/xmltok_impl.c?r1=1.13&r2=1.15&view=patch

While the above patch [3] solves the issue in expat itself and in various other packages
(PyXML, 4Suite), which embed expat, or when called via perl-XML-Parser-Expat, it does
 not help,when using the same reproducer via perl-XML-Twig module.  In this case the
 crash (buffer overread) occurs in expat's big2_toUtf8 () routine - more exactly in
 DEFINE_UTF16_TO_UTF8(big2_) macro in lib/xmltok.c:626.

 Have investigated the issue in more detail, and assuming the crash occurs in
 540 E ## toUtf8(const ENCODING *enc, \...) routine, as present in
 expat-2.0.1/lib/xmltok.c (at line 540). 

 Assuming the problematic line of the code is this one (lib/xmltok.c):

545   for (from = *fromP; from != fromLim; from += 2) { \

'from' represents pointer to the start of XML data, we are about to
 parse, 'fromLim' represents upper bound - point, where parsing
 should end. In each pass of the for loop we increment 'from'
 value by two (because on lines:

    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \

we consumed both parts of from). This works perfect,
when addresses of 'from' and 'fromLim' are aligned,
i.e. both are multiple of '2'. But the problem arises,
when 'fromLim' has not value dividable by two
(for example 165218551) - in that case, 'from' value
can't never equal to 'fromLim' value (in last round
== 'fromLim - 1', so we increment it by two, but
now we already 'skipped' it from == fromLim + 1,
and keep incrementing it (in the effort to reach
from == fromLim condition) in an infinite loop,
till the operating system recognizes we tried to
access memory location, which doesn't belong
to us and kills the process.















----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:56

Message:
Testing?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-02-20 18:54

Message:
Updated Resolution.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-27 05:35

Message:
Fixed in xmlparse.c rev. 1.165. Needs regression testing.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-13 04:49

Message:
Karl, Fred,

  could you please add Joe Orton (nickname jorton) to the Cc-list of this
ticket? I would do so, but I don't know how -
- didn't find explicit Cc field to be able to do so. Or the only way how to
make this visible for him is to remove
  the Private checkbox sign (wouldn't like to do so, as since then it would
be visible for everyone :(). Would like
  rather to find the patch for it first.

Thanks && Regards, Jan.
--
Jan iankko Lieskovsky 

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-12 10:37

Message:
I have a hard time reproducing this directly with Expat. It simply works
for me using the two files attached to Bugzilla issue. I don't use Perl at
all, and I am doing my work on Windows. Can you tell me if you use a
specially compiled version of Expat, and how Perl configures Expat when
calling it?

Maybe Fred is better at debugging this, as he is more of a Unix guy.

Btw, my feeling is that this is more related to a bug in the parsing
initialization, as these macros have no safe-guards at all and rely on the
calling code to prevent anomalous situations.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-09 11:41

Message:
The malformed XML file - pythontest1.xml can be downloaded here: 
  
https://bugzilla.redhat.com/attachment.cgi?id=366572

Don't wonder, it really contains only "^@" characters.

From https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2009-3720.

The XML-Twig.pl script code is as follows:

--- Script start ---

use XML::Twig;

my $twig=XML::Twig->new();    # create the twig
$twig->parsefile('pythontest1.xml'); # build it
#my_process( $twig);           # my_process isn't valid XML::Twig routine,
so let this commented out
#$twig->print;                 # output the twig

--- Script end ---

Run it as: perl ./XML-Twig.pl

Apologize for asking you to download the PoC, but three times
tried to attach it here, but was unsuccessful (due *.txt attachment
format and attachment size < 256 K requirement) - changed suffix
of both files to *.txt and both of them are lower in size than 256 K,
but still wasn't successful - maybe I am just doing something wrong.

Thanks, Jan.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-09 11:09

Message:
This is not a showstopper issue that happens all over the place. Priority
reset to default.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-09 09:38

Message:
Can you attach the file that allows us to reproduce this?

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-09 09:10

Message:
Just to make my report complete - this issue is present in all versions of
expat from 1.95.5 up to latest stable
one - 2.0.1

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 07:21

Message:
Grrr, when changing content of pythontest1.xml to contain:

^@space or  ^@spacea

Substitute space for ' '.

the crash is back (pointer are mangled again at the same function :(). 
Now stopping to fuzze with this, because we will never fix it.  

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 06:23

Message:
The following patch seems to fix the issue for me (under assumption patch
for CVE-2009-3720 is also applied):

$ cat expat-toUtf8.patch 
--- expat-2.0.1/lib/xmltok.c.orig	2006-11-26 18:34:46.000000000 +0100
+++ expat-2.0.1/lib/xmltok.c	2009-11-08 15:12:27.000000000 +0100
@@ -543,6 +543,9 @@ E ## toUtf8(const ENCODING *enc, \
 { \
   const char *from; \
   for (from = *fromP; from != fromLim; from += 2) { \
+    /* Stop parsing if from && fromLim addresses aren't aligned */ \
+    if (from == fromLim - 1) \
+      goto after; \
     int plane; \
     unsigned char lo2; \
     unsigned char lo = GET_LO(from); \
@@ -596,6 +599,8 @@ E ## toUtf8(const ENCODING *enc, \
     } \
   } \
   *fromP = from; \
+after: \
+  *fromP = from + 1; \
 }
 
 #define DEFINE_UTF16_TO_UTF16(E) \

The output is then:

# perl XML-Twig.pl 

no element found at line 2, column 1, byte 3 at
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm line
187

But not sure, we shouldn't check also for case, when addresses
of 'to' and 'toLim' aren't aligned (we are doing so in utf8_toUtf16()
routine:

 340 static void PTRCALL
 341 utf8_toUtf16(const ENCODING *enc,

at line:

    358     case BT_LEAD4:
    359       {
    360         unsigned long n;
    361         if (to + 1 == toLim)
    362           goto after;
 ...
    377 after:
    378   *fromP = from;
    379   *toP = to;
    380 }

So the resulting patch would then check both cases from == fromLim -1 || to
== toLim - 1,
will attach it in next comment - opinions appreciated.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:49

Message:
Here is the valgrind output (proving it's buffer over-read) in the moment
of crash:

==28534== Memcheck, a memory error detector.
==28534== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==28534== Using LibVEX rev 1658, a library for dynamic binary translation.
==28534== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==28534== Using valgrind-3.2.1, a dynamic binary instrumentation
framework.
==28534== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==28534== For more details, rerun with: -v
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x457077C: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x4570733: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x4570740: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x660AAB5: Perl_utf8n_to_uvuni (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6600CB1: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6604DB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x66094E0: Perl_regexec_flags (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65AB011: Perl_pp_match (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654703D: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654B79F: Perl_call_sv (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4016E7E: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4564AE8: reportDefault (xmlparse.c:5130)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x6600CB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6604DB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x66094E0: Perl_regexec_flags (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65AB011: Perl_pp_match (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654703D: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654B79F: Perl_call_sv (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4016E7E: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4564AE8: reportDefault (xmlparse.c:5130)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x457076F: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E676A0 is 0 bytes after a block of size 65,536
alloc'd
==28534==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==28534==    by 0x6595C1E: Perl_safesysmalloc (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4015A3C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4566262: XML_GetBuffer (xmlparse.c:1634)
==28534==    by 0x400EE5C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x4570772: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E676A1 is 1 bytes after a block of size 65,536
alloc'd
==28534==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==28534==    by 0x6595C1E: Perl_safesysmalloc (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4015A3C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4566262: XML_GetBuffer (xmlparse.c:1634)
==28534==    by 0x400EE5C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x4570891: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E83005 is not stack'd, malloc'd or (recently) free'd
==28534==
==28534== Invalid read of size 1
==28534==    at 0x45708B0: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E83004 is not stack'd, malloc'd or (recently) free'd
==28534==
==28534== Process terminating with default action of signal 11 (SIGSEGV):
dumping core
==28534==  Access not within mapped region at address 0x5283000
==28534==    at 0x457076F: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== ERROR SUMMARY: 4417417 errors from 9 contexts (suppressed: 30
from 1)
==28534== malloc/free: in use at exit: 4,192,790 bytes in 95,243 blocks.
==28534== malloc/free: 141,904 allocs, 46,661 frees, 12,100,734 bytes
allocated.
==28534== For counts of detected errors, rerun with: -v
==28534== searching for pointers to 95,243 not-freed blocks.
==28534== checked 4,132,360 bytes.
==28534==
==28534== LEAK SUMMARY:
==28534==    definitely lost: 1,415 bytes in 33 blocks.
==28534==      possibly lost: 0 bytes in 0 blocks.
==28534==    still reachable: 4,191,375 bytes in 95,210 blocks.
==28534==         suppressed: 0 bytes in 0 blocks.
==28534== Use --leak-check=full to see details of leaked memory.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:40

Message:
To verify, the issue isn't present in / it isn't fault ot XML-Parser-Expat
create following XML-Parser-Expat.pl file:

use XML::Parser::Expat;

$parser = new XML::Parser::Expat;
$parser->setHandlers('Start' => \&sh,
                      'End'   => \&eh,
                      'Char'  => \&ch);
#open(FOO, 'pythontest1.xml') or die "Couldn't open";
#$parser->parse(*FOO);
$parser->parsefile('pythontest1.xml');
close(FOO);

and run it as:

perl XML-Parser-Expat.pl

This results in:

# perl XML-Parser-Expat.pl 

no element found at line 2, column 1, byte 3 at XML-Parser-Expat.pl line 9

Further note:
-----------------
Even when you modify mentioned 'pythontest1.xml' file, i.e. add one more
character
to it, it's properly parsed by expat (in this case 'from' and 'fromLim'
addresses are
aligned so the parsing ends 'in finite time'):

Added "a" characted at the end of pythontest1.xml (i.e. it looks like ^@a).
This returns:

# perl XML-Twig.pl 

syntax error at line 1, column 0, byte 0 at
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm line
187

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:34

Message:
Reproducer:
=========

The invalid XML file, containing UTF-8 character, the crash occurs on, can
be retrieved from:

      https://bugzilla.redhat.com/attachment.cgi?id=366572

To reproduce the crash, create XML-Twig.pl script in the form of:
===============================================

use XML::Twig;

my $twig=XML::Twig->new();    # create the twig
$twig->parsefile('pythontest1.xml'); # build it
#my_process( $twig);           # my_process isn't valid XML::Twig routine,
so let this commented out
#$twig->print;                 # output the twig

And run the reproducer as:
===================

perl XML-Twig.pl ->
Segmentation fault (core dumped)

Investigating the crash in gdb leads to:
# gdb /usr/bin/perl core.28422
...
Core was generated by `perl XML-Twig.pl'.
Program terminated with signal 11, Segmentation fault.
[New process 28422]
#0  0x00fcd76f in big2_toUtf8 (enc=0xfdf860, fromP=0xbf8c57ac,
fromLim=0x9c8bb4b "", toP=0xbf8c57bc, toLim=0x9868a28 "\005") at
lib/xmltok.c:626
626	DEFINE_UTF16_TO_UTF8(big2_)

(gdb) bt
#0  0x00fcd76f in big2_toUtf8 (enc=0xfdf860, fromP=0xbf8c57ac,
fromLim=0x9c8bb4b "", toP=0xbf8c57bc, toLim=0x9868a28 "\005") at
lib/xmltok.c:626
#1  0x00fc1ac8 in reportDefault (parser=0x982cac8, enc=0xfdf860,
s=0x9cabb3e "", end=0x9c8bb4b "") at lib/xmlparse.c:5128
#2  0x00fc7f2a in doProlog (parser=0x982cac8, enc=0xfdf860, s=0x9c8bb48 "",
end=0x9c8bb4b "", tok=-15, next=0x9c8bb4b "", nextPtr=0x982cae0, 
    haveMore=0 '\0') at lib/xmlparse.c:4497
#3  0x00fc9d05 in prologProcessor (parser=0x982cac8, s=0x9c8bb48 "",
end=0x9c8bb4b "", nextPtr=0x982cae0) at lib/xmlparse.c:3551
#4  0x00fc150b in XML_ParseBuffer (parser=0x982cac8, len=0, isFinal=1) at
lib/xmlparse.c:1562
#5  0x007d1f35 in ?? () from
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so
#6  0x007d2ab4 in XS_XML__Parser__Expat_ParseStream () from
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so
#7  0x065ad51d in Perl_pp_entersub () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#8  0x065a698f in Perl_runops_standard () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#9  0x0654c20e in perl_run () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#10 0x0804921e in main ()


----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:09

Message:
Here is my further issue analysis (some of the information might be
duplicate, but there is also additional one):

While running "perl XML-Parser-Expat.pl" reports error on fixed
CVE-2009-3720 expat packages, running "perl XML-Twig.pl" still crashes:

$ perl XML-Twig.pl 
Segmentation fault (core dumped)

gdb output:
...
Core was generated by `perl XML-Twig.pl'.
Program terminated with signal 11, Segmentation fault.
[New process 23957]
#0  0x009e9cb9 in big2_toUtf8 (enc=0xa00900, fromP=0xbffa17b0,
fromLim=0x8ceca2f "", toP=0xbffa179c, toLim=0x88115f4 "\201") at
lib/xmltok.c:634
634 DEFINE_UTF16_TO_UTF8(big2_)

The problem is present in expat-2.0.1/lib/xmltok.c in toUtf8() macro:

    538 #define DEFINE_UTF16_TO_UTF8(E) \
    539 static void  PTRCALL \
    540 E ## toUtf8(const ENCODING *enc, \
    541             const char **fromP, const char *fromLim, \
    542             char **toP, const char *toLim) \
    543 { \
    544   const char *from; \
    545   for (from = *fromP; from != fromLim; from += 2) { \
    546     int plane; \
    547     unsigned char lo2; \
    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \
    550     switch (hi) { \
    551     case 0: \
    552       if (lo < 0x80) { \
    553         if (*toP == toLim) { \
    554           *fromP = from; \
    555           return; \
    556         } \
    557         *(*toP)++ = lo; \
    558         break; \
    559       } \
    560       /* fall through */ \
    561     case 0x1: case 0x2: case 0x3: \
    562     case 0x4: case 0x5: case 0x6: case 0x7: \
    563       if (toLim -  *toP < 2) { \
    564         *fromP = from; \
    565         return; \
    566       } \
    567       *(*toP)++ = ((lo >> 6) | (hi << 2) |  UTF8_cval2); \
    568       *(*toP)++ = ((lo & 0x3f) | 0x80); \
    569       break; \
    570     default: \
    571       if (toLim -  *toP < 3)  { \
    572         *fromP = from; \
    573         return; \
    574       } \

"from" should point to start of the data and "fromLim" represents upper
bound till above for cycle should loop. In each pass of the for loop,
we increment the "from" value by 2 because we have already eaten its
both parts:

    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \

and can move further. But the problem arises, when the address of
"fromLim"
is not aligned with the address of "from", i.e. it's not multiple of two.
In that case (assume from == fromLim -1) we will increment from value 
(because it != fromLim) but cross the limit value for the "fromLim" and
end up in an infinite loop till the OS recognizes buffer over read and
kills the process.

Running "perl XML-Twig.pl" demonstrates this issue.

Patched expat-2.0.1 to be more verbose which branch the code went through,
and after finding out that by processing "pythontest1.xml" it loops in
"case 0:" for "hi", added functions to print out the values of "from" and
"fromLim" variables. Here is the output:

fromLim (end) has value = 165218551
from has value = 165218548
Went by default branch
fromLim (end) has value = 165218551
from has value = 165218552
fromLim (end) has value = 165218551
from has value = 165218554
...
from has value = 165416942
fromLim (end) has value = 165218551
from has value = 165416944
seg fault

So at startup from < fromLim, we increment from with 2, so the distance
is < 3 -> we go to "default:" break part ("Went by the default branch"),
detect "from" still isn't equal to "fromLim" and increment "from" value
again by two. From now we end up in endless loop, killed by OS.

Further note:
-------------
When you add one more characted (even space) into 'pythontest1.xml',
save it and try to process it again - syntax error by processing XML
file is reported:

$ perl XML-Twig.pl 

syntax error at line 1, column 1, byte 2 at
/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/XML/Parser.pm
line
187
 at XML-Twig.pl line 4
 at XML-Twig.pl line 4  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2894085&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 21:02:29 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 12:02:29 -0800
Subject: [Expat-bugs] [ expat-Bugs-3198241 ] xmlwf does not return an
	appropriate status code when errors
Message-ID: <3V0dZv49qmzP9q@mail.python.org>

Bugs item #3198241, was opened at 2011-03-03 00:43
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3198241&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Olivier Salaun (cru-ct)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: xmlwf does not return an appropriate status code when errors

Initial Comment:
I'm using expat 1.95.8 on Red Hat Enterprise Linux Server release 5.6.
We are using the RPM.

While running xmlwf from a script, I noticed that checking the return code does not allow detecting errors. It seems that xmlwf always return 0 even when error occur while checking a file.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 12:02

Message:
Seems to be a duplicate of bug #2517938, which is fixed now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3198241&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 21:05:24 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 12:05:24 -0800
Subject: [Expat-bugs] [ expat-Bugs-2815947 ] Add support for EXEEXT
Message-ID: <3V0dfG2pwMzNsC@mail.python.org>

Bugs item #2815947, was opened at 2009-07-02 11:58
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2815947&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Paul Green (paulg73)
Assigned to: Karl Waclawek (kwaclaw)
Summary: Add support for EXEEXT

Initial Comment:
The Expat 2.0.1 Makefile.in template file does not contain support for the autoconf EXEEXT macro variable.  I've attached a unidiff that adds this support.  No changes to the configure script are required; the necessary logic is already present. I've tested these changes on a Stratus VOS system, which has a required executable extension. The changes are harmless on an operating system that does not require such an extension. Thanks for your consideration of this patch.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 12:05

Message:
Seems that the more extensive patch for issue #2884086 allows uis to close
this issue and only keep the other one open for feedback.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-02-20 18:51

Message:
Fix applied in Makefile.in rev. 1.60.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2815947&group_id=10127

From noreply at sourceforge.net  Sat Mar  3 21:10:10 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 12:10:10 -0800
Subject: [Expat-bugs] [ expat-Bugs-1810655 ] Cannot compile on solaris 8 /
	gcc-4.2.1
Message-ID: <3V0dlm0WyxzNsT@mail.python.org>

Bugs item #1810655, was opened at 2007-10-10 01:37
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1810655&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: Cannot compile on solaris 8 / gcc-4.2.1

Initial Comment:
% ./configure --prefix=...
% make
...
/bin/ksh ./libtool --silent --mode=link gcc -I./lib -I. -I/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/include -I/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/include -I/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/include -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DHAVE_EXPAT_CONFIG_H -no-undefined -version-info 6:2:5 -rpath /rtools/workspaces/sol28-64/gcc-4.2.1/expat-2.0.1/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/lib -o libexpat.la lib/xmlparse.lo lib/xmltok.lo lib/xmlrole.lo
ld: warning: option -o appears more than once, first setting taken
ld: fatal: file libexpat.so.1: open failed: No such file or directory
ld: fatal: File processing errors. No output written to .libs/libexpat.so.1.5.2
collect2: ld returned 1 exit status
make: *** [libexpat.la] Error 1

when running ksh -x ./libtool --silent ...
...
+ : gcc -shared  lib/.libs/xmlparse.o lib/.libs/xmltok.o lib/.libs/xmlrole.o  -L/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/lib -lc  -Wl,-soname -Wl,libexpat.so.1 -o .libs/libexpat.so.1.5.2
+ eval gcc -shared  lib/.libs/xmlparse.o lib/.libs/xmltok.o lib/.libs/xmlrole.o  -L/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/lib -lc  -Wl,-soname -Wl,libexpat.so.1 -o .libs/libexpat.so.1.5.2
+ gcc -shared lib/.libs/xmlparse.o lib/.libs/xmltok.o lib/.libs/xmlrole.o -L/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/lib -lc -Wl,-soname -Wl,libexpat.so.1 -o .libs/libexpat.so.1.5.2
ld: warning: option -o appears more than once, first setting taken
ld: fatal: file libexpat.so.1: open failed: No such file or directory
ld: fatal: File processing errors. No output written to .libs/libexpat.so.1.5.2
collect2: ld returned 1 exit status
+ lt_exit=1
+ test link = relink
+ exit 1

I removed '-Wl,libexpat.so.1' from the cmd and I ran manually : 
gcc -shared lib/.libs/xmlparse.o lib/.libs/xmltok.o lib/.libs/xmlrole.o -L/rtools/workspaces/sol28-64/gcc-4.2.1/readline-5.2/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/bzip2-1.0.4/lib -L/rtools/workspaces/sol28-64/gcc-4.2.1/zlib-1.2.3/lib -lc -Wl,-soname -o .libs/libexpat.so.1.5.2

which works ...

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 12:10

Message:
The most recent build system changes using autoreconf -fvi may have fixed
this.
In other words, we won't have a specific fix based on the build system used
here.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-17 10:32

Message:
Maybe Fred has time...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1810655&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 00:06:16 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 15:06:16 -0800
Subject: [Expat-bugs] [ expat-Bugs-1796209 ] Build failed on SUSE linux with
	gcc (GCC) version 3.2
Message-ID: <3V0jfy6zLvzP9G@mail.python.org>

Bugs item #1796209, was opened at 2007-09-17 06:22
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1796209&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Platform Specific
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: Build failed on SUSE linux with gcc (GCC)  version 3.2

Initial Comment:
we are looking for a UNICODE build of expat-2.0.1 
our platform details are as follows :

OS name and version details : SUSE Linux 2.4.21-138-default  i686

gcc compiler version: 3.2

we have followed following steps: 

1) ./configure --prefix=/home/eXpat/binaries/debug/ --disable-static CFLAGS="-g -O2 -fshort-wchar" CPPFLAGS="-DXML_UNICODE_WCHAR_T" 

2)make

then it throws following errors : 
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmlparse.lo -c lib/xmlparse.c
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmltok.lo -c lib/xmltok.c
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmlrole.lo -c lib/xmlrole.c
/bin/sh ./libtool --silent --mode=link gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -no-undefined -version-info 6:2:5 -rpath /home/gcuser/venkat/eXpat/binaries/debug//lib  -o libexpat.la lib/xmlparse.lo lib/xmltok.lo lib/xmlrole.lo
gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlwf.o -c xmlwf/xmlwf.c
xmlwf/xmlwf.c: In function `characterData':
xmlwf/xmlwf.c:34: warning: implicit declaration of function `fputws'
xmlwf/xmlwf.c:53: warning: implicit declaration of function `fwprintf'
xmlwf/xmlwf.c:57: warning: implicit declaration of function `putwc'
xmlwf/xmlwf.c: In function `attcmp':
xmlwf/xmlwf.c:117: warning: implicit declaration of function `wcscmp'
xmlwf/xmlwf.c: In function `nsattcmp':
xmlwf/xmlwf.c:159: warning: implicit declaration of function `wcsrchr'
xmlwf/xmlwf.c: In function `startElementNS':
xmlwf/xmlwf.c:176: warning: assignment makes pointer from integer without a cast
xmlwf/xmlwf.c:197: warning: assignment makes pointer from integer without a cast
xmlwf/xmlwf.c: In function `endElementNS':
xmlwf/xmlwf.c:222: warning: assignment makes pointer from integer without a cast
xmlwf/xmlwf.c: In function `metaStartElement':
xmlwf/xmlwf.c:353: warning: implicit declaration of function `wcslen'
xmlwf/xmlwf.c: In function `unknownEncoding':
xmlwf/xmlwf.c:575: warning: implicit declaration of function `wcschr'
xmlwf/xmlwf.c:575: warning: initialization makes pointer from integer without a cast
xmlwf/xmlwf.c: At top level:
xmlwf/xmlwf.c:644: warning: no previous prototype for `wmain'
xmlwf/xmlwf.c: In function `wmain':
xmlwf/xmlwf.c:779: warning: initialization makes pointer from integer without a cast
xmlwf/xmlwf.c:795: warning: implicit declaration of function `wcscpy'
xmlwf/xmlwf.c:796: warning: implicit declaration of function `wcscat'
xmlwf/xmlwf.c:798: warning: implicit declaration of function `_wfopen'
xmlwf/xmlwf.c:798: warning: assignment makes pointer from integer without a cast
xmlwf/xmlwf.c:800: warning: implicit declaration of function `_wperror'
xmlwf/xmlwf.c:853: warning: implicit declaration of function `_wremove'
gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlfile.o -c xmlwf/xmlfile.c
xmlwf/xmlfile.c: In function `reportError':
xmlwf/xmlfile.c:66: warning: implicit declaration of function `fwprintf'
xmlwf/xmlfile.c: In function `resolveSystemId':
xmlwf/xmlfile.c:114: warning: implicit declaration of function `wcslen'
xmlwf/xmlfile.c:118: warning: implicit declaration of function `wcscpy'
xmlwf/xmlfile.c:120: warning: implicit declaration of function `wcsrchr'
xmlwf/xmlfile.c:121: warning: assignment makes pointer from integer without a cast
xmlwf/xmlfile.c: In function `processStream':
xmlwf/xmlfile.c:160: warning: implicit declaration of function `_wopen'
xmlwf/xmlfile.c:162: warning: implicit declaration of function `_wperror'
xmlwf/xmlfile.c:173: warning: pointer type mismatch in conditional expression
xmlwf/xmlfile.c:178: warning: pointer type mismatch in conditional expression
xmlwf/xmlfile.c:184: warning: pointer type mismatch in conditional expression
gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o xmlwf/codepage.o -c xmlwf/codepage.c
gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o xmlwf/unixfilemap.o -c xmlwf/unixfilemap.c
/bin/sh ./libtool --silent --mode=link gcc -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H  -o xmlwf/xmlwf xmlwf/xmlwf.o xmlwf/xmlfile.o xmlwf/codepage.o xmlwf/unixfilemap.o libexpat.la
/usr/lib/gcc-lib/i486-suse-linux/3.2/../../../crt1.o: In function `_start':
/usr/lib/gcc-lib/i486-suse-linux/3.2/../../../crt1.o(.text+0x18): undefined reference to `main'
xmlwf/xmlwf.o: In function `wmain':
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlwf.c:798: undefined reference to `_wfopen'
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlwf.c:853: undefined reference to `_wremove'
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlwf.c:800: undefined reference to `_wperror'
xmlwf/xmlfile.o: In function `processStream':
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlfile.c:160: undefined reference to `_wopen'
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlfile.c:178: undefined reference to `_wperror'
/home/gcuser/venkat/eXpat/expat-2.0.1/xmlwf/xmlfile.c:162: undefined reference to `_wperror'
collect2: ld returned 1 exit status
make: *** [xmlwf/xmlwf] Error 1


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:06

Message:
I added a note to the README file stating that a UTF-16 build of xmlwf and
the tests is not supported.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-17 08:33

Message:
Fred,

can we close this issue, or is there something to do?

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2007-11-09 05:32

Message:
Logged In: YES 
user_id=3066
Originator: NO

Not if I'm actually careful about building it; reconfiguring an existing
build doesn't cause the object files to be re-built, so I was still using
the UTF-8 build.

Doing a "make clean" before reconfiguring yields similar results on my Mac
OS X (Leopard) machine (errors below).  I don't think this ever worked
before, but I've rarely built anything other than the UTF-8 version.

/bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2
-fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmlparse.lo -c
lib/xmlparse.c
/bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2
-fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmltok.lo -c
lib/xmltok.c
/bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2
-fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -o lib/xmlrole.lo -c
lib/xmlrole.c
/bin/sh ./libtool --silent --mode=link gcc -std=gnu99 -I./lib -I. -g -O2
-fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H -no-undefined -version-info
6:2:5 -rpath /usr/local/lib  -o libexpat.la lib/xmlparse.lo lib/xmltok.lo
lib/xmlrole.lo
gcc -std=gnu99 -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes
-Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T
-DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlwf.o -c xmlwf/xmlwf.c
xmlwf/xmlwf.c: In function ?characterData?:
xmlwf/xmlwf.c:34: warning: implicit declaration of function ?fputws?
xmlwf/xmlwf.c:53: warning: implicit declaration of function ?fwprintf?
xmlwf/xmlwf.c:57: warning: implicit declaration of function ?putwc?
xmlwf/xmlwf.c: In function ?attcmp?:
xmlwf/xmlwf.c:117: warning: implicit declaration of function ?wcscmp?
xmlwf/xmlwf.c: In function ?nsattcmp?:
xmlwf/xmlwf.c:159: warning: implicit declaration of function ?wcsrchr?
xmlwf/xmlwf.c: In function ?startElementNS?:
xmlwf/xmlwf.c:176: warning: assignment makes pointer from integer without a
cast
xmlwf/xmlwf.c:197: warning: assignment makes pointer from integer without a
cast
xmlwf/xmlwf.c: In function ?endElementNS?:
xmlwf/xmlwf.c:222: warning: assignment makes pointer from integer without a
cast
xmlwf/xmlwf.c: In function ?metaStartElement?:
xmlwf/xmlwf.c:353: warning: implicit declaration of function ?wcslen?
xmlwf/xmlwf.c: In function ?unknownEncoding?:
xmlwf/xmlwf.c:575: warning: implicit declaration of function ?wcschr?
xmlwf/xmlwf.c:575: warning: initialization makes pointer from integer
without a cast
xmlwf/xmlwf.c: At top level:
xmlwf/xmlwf.c:644: warning: no previous prototype for ?wmain?
xmlwf/xmlwf.c: In function ?wmain?:
xmlwf/xmlwf.c:779: warning: initialization makes pointer from integer
without a cast
xmlwf/xmlwf.c:795: warning: implicit declaration of function ?wcscpy?
xmlwf/xmlwf.c:796: warning: implicit declaration of function ?wcscat?
xmlwf/xmlwf.c:798: warning: implicit declaration of function ?_wfopen?
xmlwf/xmlwf.c:798: warning: assignment makes pointer from integer without a
cast
xmlwf/xmlwf.c:800: warning: implicit declaration of function
?_wperror?
xmlwf/xmlwf.c:853: warning: implicit declaration of function
?_wremove?
gcc -std=gnu99 -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes
-Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T
-DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlfile.o -c xmlwf/xmlfile.c
xmlwf/xmlfile.c: In function ?reportError?:
xmlwf/xmlfile.c:66: warning: implicit declaration of function
?fwprintf?
xmlwf/xmlfile.c: In function ?resolveSystemId?:
xmlwf/xmlfile.c:114: warning: implicit declaration of function
?wcslen?
xmlwf/xmlfile.c:118: warning: implicit declaration of function
?wcscpy?
xmlwf/xmlfile.c:120: warning: implicit declaration of function
?wcsrchr?
xmlwf/xmlfile.c:121: warning: assignment makes pointer from integer without
a cast
xmlwf/xmlfile.c: In function ?processStream?:
xmlwf/xmlfile.c:160: warning: implicit declaration of function
?_wopen?
xmlwf/xmlfile.c:162: warning: implicit declaration of function
?_wperror?
xmlwf/xmlfile.c:173: warning: pointer type mismatch in conditional
expression
xmlwf/xmlfile.c:178: warning: pointer type mismatch in conditional
expression
xmlwf/xmlfile.c:184: warning: pointer type mismatch in conditional
expression
gcc -std=gnu99 -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes
-Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T
-DHAVE_EXPAT_CONFIG_H -o xmlwf/codepage.o -c xmlwf/codepage.c
gcc -std=gnu99 -I./lib -I. -g -O2 -fshort-wchar -Wall -Wmissing-prototypes
-Wstrict-prototypes -fexceptions  -DXML_UNICODE_WCHAR_T
-DHAVE_EXPAT_CONFIG_H -o xmlwf/unixfilemap.o -c xmlwf/unixfilemap.c
/bin/sh ./libtool --silent --mode=link gcc -std=gnu99 -I./lib -I. -g -O2
-fshort-wchar -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions 
-DXML_UNICODE_WCHAR_T -DHAVE_EXPAT_CONFIG_H  -o xmlwf/xmlwf xmlwf/xmlwf.o
xmlwf/xmlfile.o xmlwf/codepage.o xmlwf/unixfilemap.o libexpat.la
Undefined symbols:
  "__wfopen", referenced from:
      _wmain in xmlwf.o
  "_main", referenced from:
      start in crt1.10.5.o
  "__wopen", referenced from:
      _processStream in xmlfile.o
  "__wremove", referenced from:
      _wmain in xmlwf.o
  "__wperror", referenced from:
      _wmain in xmlwf.o
      _processStream in xmlfile.o
      _processStream in xmlfile.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [xmlwf/xmlwf] Error 1


----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-11-09 05:03

Message:
Logged In: YES 
user_id=290026
Originator: NO

I am not even sure xmlwf was ever meant to be built for UTF-16.
The README seems to mention that (under Unix) you should only run "make
buildlib" for UTF-16 output, which builds the Expat library itself, and
nothing else.

Fred, does a complete UTF-16 build really work for you?


----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2007-11-08 20:20

Message:
Logged In: YES 
user_id=3066
Originator: NO

I don't have access to a SUSE linux system at this time; hopefully someone
else will be able to test and reproduce.


I'll see if I can put together a SUSE linux environment if I can squeeze
out some time, but I don't know anything about that distribution, so I may
be stabbing in the dark.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2007-09-17 07:26

Message:
Logged In: YES 
user_id=290026
Originator: NO

It looks like the Expat parser still builds OK, but xmlwf doesn't.
Not sure if any of our Unix experts can step in to resolve this issue ... 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1796209&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 00:07:45 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 15:07:45 -0800
Subject: [Expat-bugs] [ expat-Bugs-2412451 ] may be undefined behaver on
	integer overflow
Message-ID: <3V0jhh31bhzP9G@mail.python.org>

Bugs item #2412451, was opened at 2008-12-09 12:38
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2412451&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Platform Specific
Status: Open
>Resolution: Postponed
Priority: 5
Private: No
Submitted By: NikeL (gelenberg)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: may be undefined behaver on integer overflow

Initial Comment:
Hi,

I think in lib/xmlparse.c Revision 1.162 on line 2572
my occur integer overflow on x86_64 platform, cause int < sizeof(void*). There should be used size_t, i think.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-18 07:54

Message:
In Expat, pointer differences are basically always cast to int.
It is extremely unlikely that this would cause an overflow, as Expat breaks
the input buffer into chunks according to boundaries it recognizes, like
line-breaks, entity references, tag delimiters, etc.
Also, the input functions allow only int for specifying text lengths.

Technically, it might be possible to construct some input that might cause
an overflow, like an element name that is 2^33 characters long.

Not sure if it makes sense to do anything about it, as it would probably
require us to change all ints to (64bit) longs and maybe also break the
interface.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2412451&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 00:10:39 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 15:10:39 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V0jm13rRSzPD1@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 01:09:41 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 16:09:41 -0800
Subject: [Expat-bugs] [ expat-Bugs-3287849 ] make check fails on mingw-w64
Message-ID: <3V0l473wmRzMQp@mail.python.org>

Bugs item #3287849, was opened at 2011-04-15 13:00
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3287849&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Platform Specific
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: rubenvb (rubenvb)
Assigned to: Nobody/Anonymous (nobody)
Summary: make check fails on mingw-w64

Initial Comment:
Due to an incorrect void* to long int conversion, make check fails on mingw-w64 'mingw-w64.sourceforge.net), GNU triplet x86_64-w64-mingw32. The attached patch fixes the issue and allows 100% passed tests.

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 16:09

Message:
Modified patch applied in runtests.c rev 1.68.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3287849&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 01:17:48 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 16:17:48 -0800
Subject: [Expat-bugs] [ expat-Patches-3312568 ] cmake support
Message-ID: <3V0lFV70kBzP0P@mail.python.org>

Patches item #3312568, was opened at 2011-06-06 09:04
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3312568&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build Control
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ralf Habacker (habacker)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: cmake support

Initial Comment:
Hi, 

the append patch adds cmake support to expat original used for building dbus on windows. The cmake support has been introduced in 2010 March and has reached stable state. 

The advantage of cmake is that it covers several compiler environments 

 Borland Makefiles           = Generates Borland makefiles.
 MSYS Makefiles              = Generates MSYS makefiles.
 MinGW Makefiles             = Generates a make file for use with
                               mingw32-make.
 NMake Makefiles             = Generates NMake makefiles.
 NMake Makefiles JOM         = Generates JOM makefiles.
 Unix Makefiles              = Generates standard UNIX makefiles.
 Visual Studio 10            = Generates Visual Studio 10 project files.
 Visual Studio 10 Win64      = Generates Visual Studio 10 Win64 project
                               files.
 Visual Studio 6             = Generates Visual Studio 6 project files.
 Visual Studio 7             = Generates Visual Studio .NET 2002 project
                               files.
 Visual Studio 7 .NET 2003   = Generates Visual Studio .NET 2003 project
                               files.
 Visual Studio 8 2005        = Generates Visual Studio .NET 2005 project
                               files.
 Visual Studio 8 2005 Win64  = Generates Visual Studio .NET 2005 Win64
                               project files.
 Visual Studio 9 2008        = Generates Visual Studio 9 2008 project files.
 Visual Studio 9 2008 Win64  = Generates Visual Studio 9 2008 Win64 project
                               files.
 Watcom WMake                = Generates Watcom WMake makefiles.
 CodeBlocks - MinGW Makefiles= Generates CodeBlocks project files.
 CodeBlocks - NMake Makefiles= Generates CodeBlocks project files.
 CodeBlocks - Unix Makefiles = Generates CodeBlocks project files.
 Eclipse CDT4 - MinGW Makefiles
                             = Generates Eclipse CDT 4.0 project files.
 Eclipse CDT4 - NMake Makefiles
                             = Generates Eclipse CDT 4.0 project files.
 Eclipse CDT4 - Unix Makefiles
                             = Generates Eclipse CDT 4.0 project files.

This makes it possible to reduce the numbers of build systems used by expat and will increase the number of support compilers 



----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 16:17

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3312568&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 01:22:27 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 03 Mar 2012 16:22:27 -0800
Subject: [Expat-bugs] [ expat-Bugs-2978002 ] Pointer cast in tests/runtest.c
Message-ID: <3V0lLs2FbpzP8g@mail.python.org>

Bugs item #2978002, was opened at 2010-03-28 07:16
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2978002&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Platform Specific
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: rubenvb (rubenvb)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: Pointer cast in tests/runtest.c

Initial Comment:
Expat 2.0.1
Under mingw-w64, which uses long long (not equal to long, which is 32-bit under Windows x64) for pointers:

line 1257: illegal cast to long. Should be something like long long or intptr_t for windows x64

I'm not sure about the rest of the line. If the value will not be used as a pointer anymore, the long=1+intptr_t is okay, but if the long value is used as pointer later, it will need to be adapted as well.

Howto build: configure --host=x86_64-w64-mingw32 && make && make check
make check passes all tests.

PS: there's also a lot of warnings about gcc-deprecated casts of string constants to char* in gcc-4.4 and up

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 16:22

Message:
Was fixed with a modified patch included by the same author in a second bug
report for the same issue. Closing this one as a duplicate of #3287849
(although it was the first report).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2978002&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 22:53:26 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 13:53:26 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1J0T5Vh7zPHh@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by pspe2
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Sun Mar  4 23:15:37 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 14:15:37 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1JV66VZ0zPpm@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 00:11:23 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 15:11:23 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1KkR0sLgzPl1@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by pspe2
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:11

Message:
hm, obviously those files need to be distributed as well, I just don't know
how to bring the make-release.sh script to include them - I will try that
out in a minute.
I also forgot to add a CMake.README file, so I will update the patch again,
but that might have to wait until tomorrow (if that is ok for you).
Just in general, building with cmake should be as easy as 'mkdir build &&
cd build && cmake .. && make && make install' (you run the build process in
the subdirectory build)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 00:21:32 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 15:21:32 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1Ky82TKKzPy7@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:21

Message:
The files to include in the distribution archive should be listed in the
MANIFEST file. With the standard build not everything is included (for
instance, buildconf.sh is not distributed).

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:11

Message:
hm, obviously those files need to be distributed as well, I just don't know
how to bring the make-release.sh script to include them - I will try that
out in a minute.
I also forgot to add a CMake.README file, so I will update the patch again,
but that might have to wait until tomorrow (if that is ok for you).
Just in general, building with cmake should be as easy as 'mkdir build &&
cd build && cmake .. && make && make install' (you run the build process in
the subdirectory build)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 00:47:58 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 15:47:58 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1LXd5NcvzPbZ@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by pspe2
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:47

Message:
Ok, I updated the old patch with a readme file:
http://paste.kde.org/~windows/433496/raw/
and I made a new patch with just the changes to MANIFEST:
http://paste.kde.org/~windows/433502/
Also, I cannot do cvs add locally, is this expected?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:21

Message:
The files to include in the distribution archive should be listed in the
MANIFEST file. With the standard build not everything is included (for
instance, buildconf.sh is not distributed).

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:11

Message:
hm, obviously those files need to be distributed as well, I just don't know
how to bring the make-release.sh script to include them - I will try that
out in a minute.
I also forgot to add a CMake.README file, so I will update the patch again,
but that might have to wait until tomorrow (if that is ok for you).
Just in general, building with cmake should be as easy as 'mkdir build &&
cd build && cmake .. && make && make install' (you run the build process in
the subdirectory build)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 00:54:54 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 04 Mar 2012 15:54:54 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1Lhg4PrWzPkP@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:54

Message:
To make it simple:
Just e-mail me the new files and the patch to MANIFEST, and I will do the
rest in CVS. My e-mail is karl  at waclawek.net .

I assume your files go into the root directory.

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:47

Message:
Ok, I updated the old patch with a readme file:
http://paste.kde.org/~windows/433496/raw/
and I made a new patch with just the changes to MANIFEST:
http://paste.kde.org/~windows/433502/
Also, I cannot do cvs add locally, is this expected?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:21

Message:
The files to include in the distribution archive should be listed in the
MANIFEST file. With the standard build not everything is included (for
instance, buildconf.sh is not distributed).

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:11

Message:
hm, obviously those files need to be distributed as well, I just don't know
how to bring the make-release.sh script to include them - I will try that
out in a minute.
I also forgot to add a CMake.README file, so I will update the patch again,
but that might have to wait until tomorrow (if that is ok for you).
Just in general, building with cmake should be as easy as 'mkdir build &&
cd build && cmake .. && make && make install' (you run the build process in
the subdirectory build)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 15:46:45 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 05 Mar 2012 06:46:45 -0800
Subject: [Expat-bugs] [ expat-Bugs-2990652 ] CMake Build system
Message-ID: <3V1kTq5sNMzNsT@mail.python.org>

Bugs item #2990652, was opened at 2010-04-21 17:35
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build control
Group: Feature Request
Status: Open
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: SaroEngels (pspendrin)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CMake Build system

Initial Comment:
I am maintaining the 3rdparty libraries for the KDE Windows port. For this we developed a CMake based build system for expat which is attached. Please have a look and say if you would add it to your repository.
regards,
Patrick


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-05 06:46

Message:
Added CMake build files updated by author to repository. This also
satisfies patch submission #3312568.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:54

Message:
To make it simple:
Just e-mail me the new files and the patch to MANIFEST, and I will do the
rest in CVS. My e-mail is karl  at waclawek.net .

I assume your files go into the root directory.

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:47

Message:
Ok, I updated the old patch with a readme file:
http://paste.kde.org/~windows/433496/raw/
and I made a new patch with just the changes to MANIFEST:
http://paste.kde.org/~windows/433502/
Also, I cannot do cvs add locally, is this expected?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 15:21

Message:
The files to include in the distribution archive should be listed in the
MANIFEST file. With the standard build not everything is included (for
instance, buildconf.sh is not distributed).

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 15:11

Message:
hm, obviously those files need to be distributed as well, I just don't know
how to bring the make-release.sh script to include them - I will try that
out in a minute.
I also forgot to add a CMake.README file, so I will update the patch again,
but that might have to wait until tomorrow (if that is ok for you).
Just in general, building with cmake should be as easy as 'mkdir build &&
cd build && cmake .. && make && make install' (you run the build process in
the subdirectory build)

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-04 14:15

Message:
So, is this all that's needed? 

I know nothing about CMake, so I don't even know if the fact that the
make-release.sh script won't include your files in the tar.gz archive
matters or not.

If you need this included there as well, please update the patch
accordingly.

Thanks,

Karl

----------------------------------------------------------------------

Comment By: Patrick Spendrin (pspe2)
Date: 2012-03-04 13:53

Message:
Sorry, had a hdd crash yesterday, so I had to use a different username and
cannot add files here anymore :-(
Please download an updated patch from here:
http://paste.kde.org/~windows/433382/raw/ 
I found another person having made a similar patch (in the lua project),
but it is essentially the same license & code. I would really love to see
this patch upstream, so thank you very much.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 15:10

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2990652&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 15:49:12 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 05 Mar 2012 06:49:12 -0800
Subject: [Expat-bugs] [ expat-Patches-3312568 ] cmake support
Message-ID: <3V1kXY3BvnzNsT@mail.python.org>

Patches item #3312568, was opened at 2011-06-06 09:04
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3312568&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build Control
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Ralf Habacker (habacker)
Assigned to: Karl Waclawek (kwaclaw)
Summary: cmake support

Initial Comment:
Hi, 

the append patch adds cmake support to expat original used for building dbus on windows. The cmake support has been introduced in 2010 March and has reached stable state. 

The advantage of cmake is that it covers several compiler environments 

 Borland Makefiles           = Generates Borland makefiles.
 MSYS Makefiles              = Generates MSYS makefiles.
 MinGW Makefiles             = Generates a make file for use with
                               mingw32-make.
 NMake Makefiles             = Generates NMake makefiles.
 NMake Makefiles JOM         = Generates JOM makefiles.
 Unix Makefiles              = Generates standard UNIX makefiles.
 Visual Studio 10            = Generates Visual Studio 10 project files.
 Visual Studio 10 Win64      = Generates Visual Studio 10 Win64 project
                               files.
 Visual Studio 6             = Generates Visual Studio 6 project files.
 Visual Studio 7             = Generates Visual Studio .NET 2002 project
                               files.
 Visual Studio 7 .NET 2003   = Generates Visual Studio .NET 2003 project
                               files.
 Visual Studio 8 2005        = Generates Visual Studio .NET 2005 project
                               files.
 Visual Studio 8 2005 Win64  = Generates Visual Studio .NET 2005 Win64
                               project files.
 Visual Studio 9 2008        = Generates Visual Studio 9 2008 project files.
 Visual Studio 9 2008 Win64  = Generates Visual Studio 9 2008 Win64 project
                               files.
 Watcom WMake                = Generates Watcom WMake makefiles.
 CodeBlocks - MinGW Makefiles= Generates CodeBlocks project files.
 CodeBlocks - NMake Makefiles= Generates CodeBlocks project files.
 CodeBlocks - Unix Makefiles = Generates CodeBlocks project files.
 Eclipse CDT4 - MinGW Makefiles
                             = Generates Eclipse CDT 4.0 project files.
 Eclipse CDT4 - NMake Makefiles
                             = Generates Eclipse CDT 4.0 project files.
 Eclipse CDT4 - Unix Makefiles
                             = Generates Eclipse CDT 4.0 project files.

This makes it possible to reduce the numbers of build systems used by expat and will increase the number of support compilers 



----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-05 06:49

Message:
The same patch was also submitted in bug report #2990652.
An updated submission has been accepted and also satisfies this patch
request.
Closing it as we don't want to keep duplicates open.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 16:17

Message:
Seems non-intrusive. If you can update it for current CVS HEAD (release
2.1.0 being prepared), I'll consider it for inclusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=310127&aid=3312568&group_id=10127

From noreply at sourceforge.net  Mon Mar  5 20:00:51 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 05 Mar 2012 11:00:51 -0800
Subject: [Expat-bugs] [ expat-Bugs-2680699 ] Memory leak: outName
Message-ID: <3V1r6y6G5pzP4X@mail.python.org>

Bugs item #2680699, was opened at 2009-03-10 16:46
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2680699&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Not a Bug
>Status: Closed
>Resolution: Rejected
Priority: 5
Private: No
Submitted By: Bill Egert (boogachamp)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: Memory leak: outName

Initial Comment:
Using cppcheck 1.30 against expat 2.0.1 yeilds:

cppcheck -j2 -a -s -q .
[./xmlwf/xmlwf.c:857]: (all) Memory leak: outName

.
.
.
    if (outputDir) {
      if (outputType == 'm')
        metaEndDocument(parser);
      fclose(fp);
      if (!result)
        tremove(outName);
      free(outName);
    }
    XML_ParserFree(parser);
  }
  return 0;
}

shouldn't outName be freed regardless of outputDir's value?



----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-05 11:00

Message:
I looked at the code, and outName is only allocated when outputDir !=
null.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2680699&group_id=10127

From noreply at sourceforge.net  Tue Mar  6 05:37:54 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 05 Mar 2012 20:37:54 -0800
Subject: [Expat-bugs] [ expat-Bugs-3496608 ] CVE-2012-0876 - Hash DOS attack
Message-ID: <3V24wj70k1zP4X@mail.python.org>

Bugs item #3496608, was opened at 2012-03-03 11:01
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 7
>Private: Yes
Submitted By: Karl Waclawek (kwaclaw)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CVE-2012-0876 - Hash DOS attack

Initial Comment:
The hash table implementation in Expat can be attacked by a carefully crafted input document where all identifiers hash to the same value.
This leads to a denial of service scenario by forcing hash table lookups to do linear searching.
CVE-2012-0876 (see http://http://www.cve.mitre.org) has been reserved for this issue.
Also discussed on bugs.python.org/issue13703#msg151870 .

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:20

Message:
Fixed in expat.h rev 1.81 and xmlparse.c rev 1.168.
Thanks to David Malcolm (RedHat) for providing me with the initial version
of the patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

From noreply at sourceforge.net  Tue Mar  6 15:25:39 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 06 Mar 2012 06:25:39 -0800
Subject: [Expat-bugs] [ expat-Bugs-2958794 ] memory leak in poolGrow
Message-ID: <3V2Kyy54BpzPP7@mail.python.org>

Bugs item #2958794, was opened at 2010-02-25 08:17
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2958794&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
>Private: Yes
Submitted By: Tim Boddy (timboddy)
Assigned to: Karl Waclawek (kwaclaw)
Summary: memory leak in poolGrow

Initial Comment:
This bug applies at least to 1.95.8, 2.0.0 and 2.0.1

poolGrow in xml.parse.cpp has the following block of code:
 
  if (pool->blocks && pool->start == pool->blocks->s) {
    int blockSize = (int)(pool->end - pool->start)*2;
    pool->blocks = (BLOCK *)
      pool->mem->realloc_fcn(pool->blocks,
                             (offsetof(BLOCK, s)
                              + blockSize * sizeof(XML_Char)));
    if (pool->blocks == NULL)
      return XML_FALSE;
    pool->blocks->size = blockSize;
    pool->ptr = pool->blocks->s + (pool->ptr - pool->start);
    pool->start = pool->blocks->s;
    pool->end = pool->start + blockSize;
  }

It looks like this will cause a memory leak if realloc_fcn returns NULL because pool->blocks will be overwritten but the old memory area to which it pointed won't be freed.
 
The other places where reallocation is done in this file (via the REALLOC macro) don't have this bug because they correctly store the result in a temporary variable.
 
 
 
This bug allows a serious DOS attack on a server that accepts XML-based requests if a request is seen of the following form:
<!DOCTYPE foo [
<!ENTITY a "1234567890" >
<!ENTITY b "&a;&a;&a;&a;&a;&a;&a;&a;" >
<!ENTITY c "&b;&b;&b;&b;&b;&b;&b;&b;" >
<!ENTITY d "&c;&c;&c;&c;&c;&c;&c;&c;" >
<!ENTITY e "&d;&d;&d;&d;&d;&d;&d;&d;" >
<!ENTITY f "&e;&e;&e;&e;&e;&e;&e;&e;" >
<!ENTITY g "&f;&f;&f;&f;&f;&f;&f;&f;" >
<!ENTITY h "&g;&g;&g;&g;&g;&g;&g;&g;" >
<!ENTITY i "&h;&h;&h;&h;&h;&h;&h;&h;" >
<!ENTITY j "&i;&i;&i;&i;&i;&i;&i;&i;" >
<!ENTITY k "&j;&j;&j;&j;&j;&j;&j;&j;" >
<!ENTITY l "&k;&k;&k;&k;&k;&k;&k;&k;" >
<!ENTITY m "&l;&l;&l;&l;&l;&l;&l;&l;" >
<!ENTITY n "&m;&m;&m;&m;&m;&m;&m;&m;" >
]>
<foo bar="&n;"/>

When the attribute is expanded, the realloc (assuming that it bounds the size of the request) will fail, leaking the buffer from before the realloc.
 


----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2010-02-25 10:38

Message:
Fixed in xmlparse.ca rev. 1.167.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2958794&group_id=10127

From noreply at sourceforge.net  Fri Mar  9 15:22:14 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 09 Mar 2012 06:22:14 -0800
Subject: [Expat-bugs] [ expat-Bugs-3496608 ] CVE-2012-0876 - Hash DOS attack
Message-ID: <3V49lf2WlxzQBj@mail.python.org>

Bugs item #3496608, was opened at 2012-03-03 11:01
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 7
>Private: No
Submitted By: Karl Waclawek (kwaclaw)
Assigned to: Karl Waclawek (kwaclaw)
Summary: CVE-2012-0876 - Hash DOS attack

Initial Comment:
The hash table implementation in Expat can be attacked by a carefully crafted input document where all identifiers hash to the same value.
This leads to a denial of service scenario by forcing hash table lookups to do linear searching.
CVE-2012-0876 (see http://http://www.cve.mitre.org) has been reserved for this issue.
Also discussed on bugs.python.org/issue13703#msg151870 .

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:20

Message:
Fixed in expat.h rev 1.81 and xmlparse.c rev 1.168.
Thanks to David Malcolm (RedHat) for providing me with the initial version
of the patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3496608&group_id=10127

From noreply at sourceforge.net  Fri Mar  9 15:23:10 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 09 Mar 2012 06:23:10 -0800
Subject: [Expat-bugs] [ expat-Bugs-2958794 ] memory leak in poolGrow
Message-ID: <3V49mj0LVdzPYn@mail.python.org>

Bugs item #2958794, was opened at 2010-02-25 08:17
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2958794&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
>Private: No
Submitted By: Tim Boddy (timboddy)
Assigned to: Karl Waclawek (kwaclaw)
Summary: memory leak in poolGrow

Initial Comment:
This bug applies at least to 1.95.8, 2.0.0 and 2.0.1

poolGrow in xml.parse.cpp has the following block of code:
 
  if (pool->blocks && pool->start == pool->blocks->s) {
    int blockSize = (int)(pool->end - pool->start)*2;
    pool->blocks = (BLOCK *)
      pool->mem->realloc_fcn(pool->blocks,
                             (offsetof(BLOCK, s)
                              + blockSize * sizeof(XML_Char)));
    if (pool->blocks == NULL)
      return XML_FALSE;
    pool->blocks->size = blockSize;
    pool->ptr = pool->blocks->s + (pool->ptr - pool->start);
    pool->start = pool->blocks->s;
    pool->end = pool->start + blockSize;
  }

It looks like this will cause a memory leak if realloc_fcn returns NULL because pool->blocks will be overwritten but the old memory area to which it pointed won't be freed.
 
The other places where reallocation is done in this file (via the REALLOC macro) don't have this bug because they correctly store the result in a temporary variable.
 
 
 
This bug allows a serious DOS attack on a server that accepts XML-based requests if a request is seen of the following form:
<!DOCTYPE foo [
<!ENTITY a "1234567890" >
<!ENTITY b "&a;&a;&a;&a;&a;&a;&a;&a;" >
<!ENTITY c "&b;&b;&b;&b;&b;&b;&b;&b;" >
<!ENTITY d "&c;&c;&c;&c;&c;&c;&c;&c;" >
<!ENTITY e "&d;&d;&d;&d;&d;&d;&d;&d;" >
<!ENTITY f "&e;&e;&e;&e;&e;&e;&e;&e;" >
<!ENTITY g "&f;&f;&f;&f;&f;&f;&f;&f;" >
<!ENTITY h "&g;&g;&g;&g;&g;&g;&g;&g;" >
<!ENTITY i "&h;&h;&h;&h;&h;&h;&h;&h;" >
<!ENTITY j "&i;&i;&i;&i;&i;&i;&i;&i;" >
<!ENTITY k "&j;&j;&j;&j;&j;&j;&j;&j;" >
<!ENTITY l "&k;&k;&k;&k;&k;&k;&k;&k;" >
<!ENTITY m "&l;&l;&l;&l;&l;&l;&l;&l;" >
<!ENTITY n "&m;&m;&m;&m;&m;&m;&m;&m;" >
]>
<foo bar="&n;"/>

When the attribute is expanded, the realloc (assuming that it bounds the size of the request) will fail, leaking the buffer from before the realloc.
 


----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2010-02-25 10:38

Message:
Fixed in xmlparse.ca rev. 1.167.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2958794&group_id=10127

From noreply at sourceforge.net  Fri Mar  9 15:24:17 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 09 Mar 2012 06:24:17 -0800
Subject: [Expat-bugs] [ expat-Bugs-2894085 ] expat: buffer over-read and
	crash in big2_toUtf8()
Message-ID: <3V49p01ZDYzPYn@mail.python.org>

Bugs item #2894085, was opened at 2009-11-08 03:06
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2894085&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
>Private: No
Submitted By: Jan Lieskovsky (iankko)
Assigned to: Karl Waclawek (kwaclaw)
Summary: expat: buffer over-read and crash in big2_toUtf8() 

Initial Comment:
Hello SourceForge expat maintainers,

  originally CVE-2009-3720 was reported in expat:
  [1]  http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2009-3720

  Non-public, original bug report for CVE-2009-3720:
  [2] http://sourceforge.net/tracker/?func=detail&aid=1990430&group_id=10127&atid=110127

  And relevant patch for CVE-2009-3720:
  [3] http://expat.cvs.sourceforge.net/viewvc/expat/expat/lib/xmltok_impl.c?r1=1.13&r2=1.15&view=patch

While the above patch [3] solves the issue in expat itself and in various other packages
(PyXML, 4Suite), which embed expat, or when called via perl-XML-Parser-Expat, it does
 not help,when using the same reproducer via perl-XML-Twig module.  In this case the
 crash (buffer overread) occurs in expat's big2_toUtf8 () routine - more exactly in
 DEFINE_UTF16_TO_UTF8(big2_) macro in lib/xmltok.c:626.

 Have investigated the issue in more detail, and assuming the crash occurs in
 540 E ## toUtf8(const ENCODING *enc, \...) routine, as present in
 expat-2.0.1/lib/xmltok.c (at line 540). 

 Assuming the problematic line of the code is this one (lib/xmltok.c):

545   for (from = *fromP; from != fromLim; from += 2) { \

'from' represents pointer to the start of XML data, we are about to
 parse, 'fromLim' represents upper bound - point, where parsing
 should end. In each pass of the for loop we increment 'from'
 value by two (because on lines:

    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \

we consumed both parts of from). This works perfect,
when addresses of 'from' and 'fromLim' are aligned,
i.e. both are multiple of '2'. But the problem arises,
when 'fromLim' has not value dividable by two
(for example 165218551) - in that case, 'from' value
can't never equal to 'fromLim' value (in last round
== 'fromLim - 1', so we increment it by two, but
now we already 'skipped' it from == fromLim + 1,
and keep incrementing it (in the effort to reach
from == fromLim condition) in an infinite loop,
till the operating system recognizes we tried to
access memory location, which doesn't belong
to us and kills the process.















----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-03 11:56

Message:
Testing?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-02-20 18:54

Message:
Updated Resolution.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-27 05:35

Message:
Fixed in xmlparse.c rev. 1.165. Needs regression testing.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-13 04:49

Message:
Karl, Fred,

  could you please add Joe Orton (nickname jorton) to the Cc-list of this
ticket? I would do so, but I don't know how -
- didn't find explicit Cc field to be able to do so. Or the only way how to
make this visible for him is to remove
  the Private checkbox sign (wouldn't like to do so, as since then it would
be visible for everyone :(). Would like
  rather to find the patch for it first.

Thanks && Regards, Jan.
--
Jan iankko Lieskovsky 

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-12 10:37

Message:
I have a hard time reproducing this directly with Expat. It simply works
for me using the two files attached to Bugzilla issue. I don't use Perl at
all, and I am doing my work on Windows. Can you tell me if you use a
specially compiled version of Expat, and how Perl configures Expat when
calling it?

Maybe Fred is better at debugging this, as he is more of a Unix guy.

Btw, my feeling is that this is more related to a bug in the parsing
initialization, as these macros have no safe-guards at all and rely on the
calling code to prevent anomalous situations.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-09 11:41

Message:
The malformed XML file - pythontest1.xml can be downloaded here: 
  
https://bugzilla.redhat.com/attachment.cgi?id=366572

Don't wonder, it really contains only "^@" characters.

From https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2009-3720.

The XML-Twig.pl script code is as follows:

--- Script start ---

use XML::Twig;

my $twig=XML::Twig->new();    # create the twig
$twig->parsefile('pythontest1.xml'); # build it
#my_process( $twig);           # my_process isn't valid XML::Twig routine,
so let this commented out
#$twig->print;                 # output the twig

--- Script end ---

Run it as: perl ./XML-Twig.pl

Apologize for asking you to download the PoC, but three times
tried to attach it here, but was unsuccessful (due *.txt attachment
format and attachment size < 256 K requirement) - changed suffix
of both files to *.txt and both of them are lower in size than 256 K,
but still wasn't successful - maybe I am just doing something wrong.

Thanks, Jan.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-09 11:09

Message:
This is not a showstopper issue that happens all over the place. Priority
reset to default.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-11-09 09:38

Message:
Can you attach the file that allows us to reproduce this?

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-09 09:10

Message:
Just to make my report complete - this issue is present in all versions of
expat from 1.95.5 up to latest stable
one - 2.0.1

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 07:21

Message:
Grrr, when changing content of pythontest1.xml to contain:

^@space or  ^@spacea

Substitute space for ' '.

the crash is back (pointer are mangled again at the same function :(). 
Now stopping to fuzze with this, because we will never fix it.  

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 06:23

Message:
The following patch seems to fix the issue for me (under assumption patch
for CVE-2009-3720 is also applied):

$ cat expat-toUtf8.patch 
--- expat-2.0.1/lib/xmltok.c.orig	2006-11-26 18:34:46.000000000 +0100
+++ expat-2.0.1/lib/xmltok.c	2009-11-08 15:12:27.000000000 +0100
@@ -543,6 +543,9 @@ E ## toUtf8(const ENCODING *enc, \
 { \
   const char *from; \
   for (from = *fromP; from != fromLim; from += 2) { \
+    /* Stop parsing if from && fromLim addresses aren't aligned */ \
+    if (from == fromLim - 1) \
+      goto after; \
     int plane; \
     unsigned char lo2; \
     unsigned char lo = GET_LO(from); \
@@ -596,6 +599,8 @@ E ## toUtf8(const ENCODING *enc, \
     } \
   } \
   *fromP = from; \
+after: \
+  *fromP = from + 1; \
 }
 
 #define DEFINE_UTF16_TO_UTF16(E) \

The output is then:

# perl XML-Twig.pl 

no element found at line 2, column 1, byte 3 at
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm line
187

But not sure, we shouldn't check also for case, when addresses
of 'to' and 'toLim' aren't aligned (we are doing so in utf8_toUtf16()
routine:

 340 static void PTRCALL
 341 utf8_toUtf16(const ENCODING *enc,

at line:

    358     case BT_LEAD4:
    359       {
    360         unsigned long n;
    361         if (to + 1 == toLim)
    362           goto after;
 ...
    377 after:
    378   *fromP = from;
    379   *toP = to;
    380 }

So the resulting patch would then check both cases from == fromLim -1 || to
== toLim - 1,
will attach it in next comment - opinions appreciated.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:49

Message:
Here is the valgrind output (proving it's buffer over-read) in the moment
of crash:

==28534== Memcheck, a memory error detector.
==28534== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==28534== Using LibVEX rev 1658, a library for dynamic binary translation.
==28534== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==28534== Using valgrind-3.2.1, a dynamic binary instrumentation
framework.
==28534== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==28534== For more details, rerun with: -v
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x457077C: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x4570733: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==    
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x4570740: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x660AAB5: Perl_utf8n_to_uvuni (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6600CB1: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6604DB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x66094E0: Perl_regexec_flags (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65AB011: Perl_pp_match (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654703D: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654B79F: Perl_call_sv (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4016E7E: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4564AE8: reportDefault (xmlparse.c:5130)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==
==28534== Conditional jump or move depends on uninitialised value(s)
==28534==    at 0x6600CB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x6604DB4: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x66094E0: Perl_regexec_flags (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65AB011: Perl_pp_match (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654703D: (within
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654B79F: Perl_call_sv (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4016E7E: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4564AE8: reportDefault (xmlparse.c:5130)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x457076F: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E676A0 is 0 bytes after a block of size 65,536
alloc'd
==28534==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==28534==    by 0x6595C1E: Perl_safesysmalloc (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4015A3C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4566262: XML_GetBuffer (xmlparse.c:1634)
==28534==    by 0x400EE5C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x4570772: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E676A1 is 1 bytes after a block of size 65,536
alloc'd
==28534==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==28534==    by 0x6595C1E: Perl_safesysmalloc (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x4015A3C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x4566262: XML_GetBuffer (xmlparse.c:1634)
==28534==    by 0x400EE5C: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== Invalid read of size 1
==28534==    at 0x4570891: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E83005 is not stack'd, malloc'd or (recently) free'd
==28534==
==28534== Invalid read of size 1
==28534==    at 0x45708B0: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==  Address 0x4E83004 is not stack'd, malloc'd or (recently) free'd
==28534==
==28534== Process terminating with default action of signal 11 (SIGSEGV):
dumping core
==28534==  Access not within mapped region at address 0x5283000
==28534==    at 0x457076F: big2_toUtf8 (xmltok.c:626)
==28534==    by 0x4564AC7: reportDefault (xmlparse.c:5128)
==28534==    by 0x456AF29: doProlog (xmlparse.c:4497)
==28534==    by 0x456CD04: prologProcessor (xmlparse.c:3551)
==28534==    by 0x456450A: XML_ParseBuffer (xmlparse.c:1562)
==28534==    by 0x400EF34: (within
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x400FAB3: XS_XML__Parser__Expat_ParseStream (in
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so)
==28534==    by 0x65AD51C: Perl_pp_entersub (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x65A698E: Perl_runops_standard (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x654C20D: perl_run (in
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so)
==28534==    by 0x804921D: main (in /usr/bin/perl)
==28534==
==28534== ERROR SUMMARY: 4417417 errors from 9 contexts (suppressed: 30
from 1)
==28534== malloc/free: in use at exit: 4,192,790 bytes in 95,243 blocks.
==28534== malloc/free: 141,904 allocs, 46,661 frees, 12,100,734 bytes
allocated.
==28534== For counts of detected errors, rerun with: -v
==28534== searching for pointers to 95,243 not-freed blocks.
==28534== checked 4,132,360 bytes.
==28534==
==28534== LEAK SUMMARY:
==28534==    definitely lost: 1,415 bytes in 33 blocks.
==28534==      possibly lost: 0 bytes in 0 blocks.
==28534==    still reachable: 4,191,375 bytes in 95,210 blocks.
==28534==         suppressed: 0 bytes in 0 blocks.
==28534== Use --leak-check=full to see details of leaked memory.

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:40

Message:
To verify, the issue isn't present in / it isn't fault ot XML-Parser-Expat
create following XML-Parser-Expat.pl file:

use XML::Parser::Expat;

$parser = new XML::Parser::Expat;
$parser->setHandlers('Start' => \&sh,
                      'End'   => \&eh,
                      'Char'  => \&ch);
#open(FOO, 'pythontest1.xml') or die "Couldn't open";
#$parser->parse(*FOO);
$parser->parsefile('pythontest1.xml');
close(FOO);

and run it as:

perl XML-Parser-Expat.pl

This results in:

# perl XML-Parser-Expat.pl 

no element found at line 2, column 1, byte 3 at XML-Parser-Expat.pl line 9

Further note:
-----------------
Even when you modify mentioned 'pythontest1.xml' file, i.e. add one more
character
to it, it's properly parsed by expat (in this case 'from' and 'fromLim'
addresses are
aligned so the parsing ends 'in finite time'):

Added "a" characted at the end of pythontest1.xml (i.e. it looks like ^@a).
This returns:

# perl XML-Twig.pl 

syntax error at line 1, column 0, byte 0 at
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm line
187

----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:34

Message:
Reproducer:
=========

The invalid XML file, containing UTF-8 character, the crash occurs on, can
be retrieved from:

      https://bugzilla.redhat.com/attachment.cgi?id=366572

To reproduce the crash, create XML-Twig.pl script in the form of:
===============================================

use XML::Twig;

my $twig=XML::Twig->new();    # create the twig
$twig->parsefile('pythontest1.xml'); # build it
#my_process( $twig);           # my_process isn't valid XML::Twig routine,
so let this commented out
#$twig->print;                 # output the twig

And run the reproducer as:
===================

perl XML-Twig.pl ->
Segmentation fault (core dumped)

Investigating the crash in gdb leads to:
# gdb /usr/bin/perl core.28422
...
Core was generated by `perl XML-Twig.pl'.
Program terminated with signal 11, Segmentation fault.
[New process 28422]
#0  0x00fcd76f in big2_toUtf8 (enc=0xfdf860, fromP=0xbf8c57ac,
fromLim=0x9c8bb4b "", toP=0xbf8c57bc, toLim=0x9868a28 "\005") at
lib/xmltok.c:626
626	DEFINE_UTF16_TO_UTF8(big2_)

(gdb) bt
#0  0x00fcd76f in big2_toUtf8 (enc=0xfdf860, fromP=0xbf8c57ac,
fromLim=0x9c8bb4b "", toP=0xbf8c57bc, toLim=0x9868a28 "\005") at
lib/xmltok.c:626
#1  0x00fc1ac8 in reportDefault (parser=0x982cac8, enc=0xfdf860,
s=0x9cabb3e "", end=0x9c8bb4b "") at lib/xmlparse.c:5128
#2  0x00fc7f2a in doProlog (parser=0x982cac8, enc=0xfdf860, s=0x9c8bb48 "",
end=0x9c8bb4b "", tok=-15, next=0x9c8bb4b "", nextPtr=0x982cae0, 
    haveMore=0 '\0') at lib/xmlparse.c:4497
#3  0x00fc9d05 in prologProcessor (parser=0x982cac8, s=0x9c8bb48 "",
end=0x9c8bb4b "", nextPtr=0x982cae0) at lib/xmlparse.c:3551
#4  0x00fc150b in XML_ParseBuffer (parser=0x982cac8, len=0, isFinal=1) at
lib/xmlparse.c:1562
#5  0x007d1f35 in ?? () from
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so
#6  0x007d2ab4 in XS_XML__Parser__Expat_ParseStream () from
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/auto/XML/Parser/Expat/Expat.so
#7  0x065ad51d in Perl_pp_entersub () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#8  0x065a698f in Perl_runops_standard () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#9  0x0654c20e in perl_run () from
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
#10 0x0804921e in main ()


----------------------------------------------------------------------

Comment By: Jan Lieskovsky (iankko)
Date: 2009-11-08 03:09

Message:
Here is my further issue analysis (some of the information might be
duplicate, but there is also additional one):

While running "perl XML-Parser-Expat.pl" reports error on fixed
CVE-2009-3720 expat packages, running "perl XML-Twig.pl" still crashes:

$ perl XML-Twig.pl 
Segmentation fault (core dumped)

gdb output:
...
Core was generated by `perl XML-Twig.pl'.
Program terminated with signal 11, Segmentation fault.
[New process 23957]
#0  0x009e9cb9 in big2_toUtf8 (enc=0xa00900, fromP=0xbffa17b0,
fromLim=0x8ceca2f "", toP=0xbffa179c, toLim=0x88115f4 "\201") at
lib/xmltok.c:634
634 DEFINE_UTF16_TO_UTF8(big2_)

The problem is present in expat-2.0.1/lib/xmltok.c in toUtf8() macro:

    538 #define DEFINE_UTF16_TO_UTF8(E) \
    539 static void  PTRCALL \
    540 E ## toUtf8(const ENCODING *enc, \
    541             const char **fromP, const char *fromLim, \
    542             char **toP, const char *toLim) \
    543 { \
    544   const char *from; \
    545   for (from = *fromP; from != fromLim; from += 2) { \
    546     int plane; \
    547     unsigned char lo2; \
    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \
    550     switch (hi) { \
    551     case 0: \
    552       if (lo < 0x80) { \
    553         if (*toP == toLim) { \
    554           *fromP = from; \
    555           return; \
    556         } \
    557         *(*toP)++ = lo; \
    558         break; \
    559       } \
    560       /* fall through */ \
    561     case 0x1: case 0x2: case 0x3: \
    562     case 0x4: case 0x5: case 0x6: case 0x7: \
    563       if (toLim -  *toP < 2) { \
    564         *fromP = from; \
    565         return; \
    566       } \
    567       *(*toP)++ = ((lo >> 6) | (hi << 2) |  UTF8_cval2); \
    568       *(*toP)++ = ((lo & 0x3f) | 0x80); \
    569       break; \
    570     default: \
    571       if (toLim -  *toP < 3)  { \
    572         *fromP = from; \
    573         return; \
    574       } \

"from" should point to start of the data and "fromLim" represents upper
bound till above for cycle should loop. In each pass of the for loop,
we increment the "from" value by 2 because we have already eaten its
both parts:

    548     unsigned char lo = GET_LO(from); \
    549     unsigned char hi = GET_HI(from); \

and can move further. But the problem arises, when the address of
"fromLim"
is not aligned with the address of "from", i.e. it's not multiple of two.
In that case (assume from == fromLim -1) we will increment from value 
(because it != fromLim) but cross the limit value for the "fromLim" and
end up in an infinite loop till the OS recognizes buffer over read and
kills the process.

Running "perl XML-Twig.pl" demonstrates this issue.

Patched expat-2.0.1 to be more verbose which branch the code went through,
and after finding out that by processing "pythontest1.xml" it loops in
"case 0:" for "hi", added functions to print out the values of "from" and
"fromLim" variables. Here is the output:

fromLim (end) has value = 165218551
from has value = 165218548
Went by default branch
fromLim (end) has value = 165218551
from has value = 165218552
fromLim (end) has value = 165218551
from has value = 165218554
...
from has value = 165416942
fromLim (end) has value = 165218551
from has value = 165416944
seg fault

So at startup from < fromLim, we increment from with 2, so the distance
is < 3 -> we go to "default:" break part ("Went by the default branch"),
detect "from" still isn't equal to "fromLim" and increment "from" value
again by two. From now we end up in endless loop, killed by OS.

Further note:
-------------
When you add one more characted (even space) into 'pythontest1.xml',
save it and try to process it again - syntax error by processing XML
file is reported:

$ perl XML-Twig.pl 

syntax error at line 1, column 1, byte 2 at
/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/XML/Parser.pm
line
187
 at XML-Twig.pl line 4
 at XML-Twig.pl line 4  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=2894085&group_id=10127

From noreply at sourceforge.net  Fri Mar  9 15:25:44 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 09 Mar 2012 06:25:44 -0800
Subject: [Expat-bugs] [ expat-Bugs-1990430 ] Parser crash with specially
	formatted UTF-8 sequences
Message-ID: <3V49qf3C5ZzPpq@mail.python.org>

Bugs item #1990430, was opened at 2008-06-10 21:45
Message generated for change (Settings changed) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1990430&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: www.libexpat.org
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
>Private: No
Submitted By: Peter Valchev (petervalchev)
Assigned to: Karl Waclawek (kwaclaw)
Summary: Parser crash with specially formatted UTF-8 sequences

Initial Comment:
I have discovered a way to crash libexpat's xml parser with certain specially formatted UTF-8 sequences. All applications that link w/ expat and use it to render user-provided XML files are affected. As far as I see, the issue is not exploitable, just denial of service.

This is the patch that I have come up with, also attached to this email:

+++ lib/xmltok_impl.c 2007-12-21 11:11:42.054417000 -0800
@@ -1745,6 +1745,9 @@
 switch (BYTE_TYPE(enc, ptr)) {
 #define LEAD_CASE(n) \
 case BT_LEAD ## n: \
+ if (end - ptr < n) { \
+   return; \
+ } \
 ptr += n; \
 break;
 LEAD_CASE(2) LEAD_CASE(3) LEAD_CASE(4)

The parser's updatePosition function which keeps track of the current position pointer increments the ptr by {2, 3, 4} to skip past multibyte character ombinations, and this causes ptr in the "while (ptr != end)" loop to jump past the terminating condition, causing the loop to continue reading past 'end' and into out of bounds memory until a crash.

In general this parser does not appear the most robust and could be the source of some security issues.

A fault file is attached. To reproduce, compile examples/outline.c and run against it. This patch may not be 100% complete...

Contact:
Peter Valchev <pvalchev at google.com>

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2009-01-17 08:15

Message:
Has anyone tested current CVS for this issue's fix?

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2008-06-13 10:48

Message:
Logged In: YES 
user_id=290026
Originator: NO

I know what you mean and I had the exact same thoughts. I think at least
for older CPUs doing a "<" comparison might have been slower than doing a
comparison for equality, so I am not sure how that would work out now.
Also, we have to consider that this could modify the logic, with a slight
risk of introducing an error.

Anyway, I think there are other parts of the code (like the encoding
checking) that ensures that pathological situations like the one you found
don't happen. It is also important to observe that the crash seems to
happen only *after* the parser returned with an error, and the code gets
called again to get the error location while the parser is already in an
error state.

Don't forget, Expat is by default installed on most Linux distributions and
it is used by Firefox/Mozilla, so Expat has been given a real good workout
with millions of installations. The bugs you reported are the first bug
reports we received in a long time (except for platform specific build
issues).

Nevertheless, please keep digging, we really want Expat to be resistant to
pathological input.

Thanks for the efforts,

Karl

----------------------------------------------------------------------

Comment By: Peter Valchev (petervalchev)
Date: 2008-06-13 09:41

Message:
Logged In: YES 
user_id=2114255
Originator: YES

Yes, I originally had the same patch (while ptr < end) but chose the latter
to keep in line with the rest of the file, sigh.

Now the next question is, there are many while (ptr != end) loops still
left in that file if you search.. some of the look like they could have the
same problem. Should all of them be changed, to be on the paranoid side?
The original construct is just evil and waiting for bugs like this to
happen...

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2008-06-13 06:24

Message:
Logged In: YES 
user_id=290026
Originator: NO

I applied the original fix I had in mind, which I didn't use because I
thought yours might perform better. It is quite simple: replace "while (ptr
!= end)" with "while (ptr < end)".
Seems to work so far. Fixed in xmltok_impl.c rev. 1.15.

Karl

----------------------------------------------------------------------

Comment By: Peter Valchev (petervalchev)
Date: 2008-06-12 22:15

Message:
Logged In: YES 
user_id=2114255
Originator: YES

Ugh the padding I inserted broke the example. New attached tarball with
expat-fault3.xml crashes both outline and xmlwf

(gdb) r < /tmp/expat-fault3.
Starting program: /usr/src/lib/libexpat/examples/outline <
/tmp/expat-fault3.
/bin/ksh: cannot open /tmp/expat-fault3.: No such file or directory

Program exited with code 01.
You can't do that without a process to debug.
(gdb) r < /tmp/expat-fault3.xml
Starting program: /usr/src/lib/libexpat/examples/outline <
/tmp/expat-fault3.xml

Program received signal SIGSEGV, Segmentation fault.
little2_updatePosition (enc=0x2676dec0, ptr=0x7f4ba000 <Address 0x7f4ba000
out of bounds>, 
    end=0x7f4b9003 "", pos=0x7de87198) at xmltok_impl.c:1748
1748        switch (BYTE_TYPE(enc, ptr)) {
(gdb) bt
#0  little2_updatePosition (enc=0x2676dec0, ptr=0x7f4ba000 <Address
0x7f4ba000 out of bounds>, 
    end=0x7f4b9003 "", pos=0x7de87198) at xmltok_impl.c:1748
#1  0x0676b8a3 in XML_GetCurrentLineNumber (parser=0x7de87000)
    at /usr/src/lib/libexpat/lib/xmlparse.c:1799
#2  0x1c000a24 in main ()

File Added: expat-fault3.tar.gz

----------------------------------------------------------------------

Comment By: Peter Valchev (petervalchev)
Date: 2008-06-12 22:14

Message:
Logged In: YES 
user_id=2114255
Originator: YES

Ugh the padding I inserted broke the example. New attached tarball with
expat-fault3.xml crashes both outline and xmlwf

(gdb) r < /tmp/expat-fault3.
Starting program: /usr/src/lib/libexpat/examples/outline <
/tmp/expat-fault3.
/bin/ksh: cannot open /tmp/expat-fault3.: No such file or directory

Program exited with code 01.
You can't do that without a process to debug.
(gdb) r < /tmp/expat-fault3.xml
Starting program: /usr/src/lib/libexpat/examples/outline <
/tmp/expat-fault3.xml

Program received signal SIGSEGV, Segmentation fault.
little2_updatePosition (enc=0x2676dec0, ptr=0x7f4ba000 <Address 0x7f4ba000
out of bounds>, 
    end=0x7f4b9003 "", pos=0x7de87198) at xmltok_impl.c:1748
1748        switch (BYTE_TYPE(enc, ptr)) {
(gdb) bt
#0  little2_updatePosition (enc=0x2676dec0, ptr=0x7f4ba000 <Address
0x7f4ba000 out of bounds>, 
    end=0x7f4b9003 "", pos=0x7de87198) at xmltok_impl.c:1748
#1  0x0676b8a3 in XML_GetCurrentLineNumber (parser=0x7de87000)
    at /usr/src/lib/libexpat/lib/xmlparse.c:1799
#2  0x1c000a24 in main ()

File Added: expat-fault3.tar.gz

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2008-06-12 19:40

Message:
Logged In: YES 
user_id=290026
Originator: NO

What is supposed to happen with the second test case?
It reports an error, but does not crash with xmlwf or outline.
IE reports the same error, so that is probably OK.

----------------------------------------------------------------------

Comment By: Peter Valchev (petervalchev)
Date: 2008-06-12 15:08

Message:
Logged In: YES 
user_id=2114255
Originator: YES

Thanks.

Actually rechecking in this area again, I think I found another issue,
which seems to not be covered with the patch I provided :( I am attaching
the new test case.

I haven't found many other issues at this point... I stumbled upon this one
fairly quickly and my above observations were more general than anything.
If I do find more I'll be sure to tell you.

File Added: expat-fault2.xml

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2008-06-11 07:46

Message:
Logged In: YES 
user_id=290026
Originator: NO

Can reproduce. The problem is that this code can be called *after* an error
has been found (to report line and column number). Therefore it should not
rely on correct byte counts for multibyte characters.

Patch applied in xmltok_impl.c rev. 1.14.

Would you please also report all the other issues you have found?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=1990430&group_id=10127

From noreply at sourceforge.net  Sat Mar 10 03:37:43 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 09 Mar 2012 18:37:43 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V4V4960ZBzPv1@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Tracker Item Submitted) made by marienz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
Assigned to: Nobody/Anonymous (nobody)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sat Mar 10 18:31:15 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 10 Mar 2012 09:31:15 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V4svC470TzQ4X@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Test Required
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
>Assigned to: Karl Waclawek (kwaclaw)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:31

Message:
Yes, your analysis is correct, the hash function was used before the salt
was set.
I will have to move the setContext() call right after the
generate_hash_secret_salt() call.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sat Mar 10 18:41:24 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 10 Mar 2012 09:41:24 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V4t6w67gGzQLT@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
Assigned to: Karl Waclawek (kwaclaw)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:41

Message:
Fixed in xmlparse.c  rev 1.169. Please test.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:31

Message:
Yes, your analysis is correct, the hash function was used before the salt
was set.
I will have to move the setContext() call right after the
generate_hash_secret_salt() call.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sat Mar 10 18:54:22 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 10 Mar 2012 09:54:22 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V4tPv3Ky4zPhy@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Comment added) made by ssuominen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
Assigned to: Karl Waclawek (kwaclaw)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

Comment By: Samuli Suominen (ssuominen)
Date: 2012-03-10 09:54

Message:
I can verify that pulling in the diff of xmlparse.c 1.168 and 1.169 and
applying it on top of expat-2.1.0-beta2 works for me. Thanks a lot for
quick action!

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:41

Message:
Fixed in xmlparse.c  rev 1.169. Please test.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:31

Message:
Yes, your analysis is correct, the hash function was used before the salt
was set.
I will have to move the setContext() call right after the
generate_hash_secret_salt() call.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sat Mar 10 19:07:37 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 10 Mar 2012 10:07:37 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V4tj91fc2zPJJ@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Comment added) made by marienz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
Assigned to: Karl Waclawek (kwaclaw)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

>Comment By: Marien Zwart (marienz)
Date: 2012-03-10 10:07

Message:
Yes, that fixed it, thanks!

----------------------------------------------------------------------

Comment By: Samuli Suominen (ssuominen)
Date: 2012-03-10 09:54

Message:
I can verify that pulling in the diff of xmlparse.c 1.168 and 1.169 and
applying it on top of expat-2.1.0-beta2 works for me. Thanks a lot for
quick action!

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:41

Message:
Fixed in xmlparse.c  rev 1.169. Please test.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:31

Message:
Yes, your analysis is correct, the hash function was used before the salt
was set.
I will have to move the setContext() call right after the
generate_hash_secret_salt() call.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sun Mar 11 06:15:11 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 10 Mar 2012 21:15:11 -0800
Subject: [Expat-bugs] [ expat-Bugs-3500861 ] fix for CVE-2012-0876 breaks
	"xml" default prefix
Message-ID: <3V59WR4CfjzPfn@mail.python.org>

Bugs item #3500861, was opened at 2012-03-09 18:37
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Test Required
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Marien Zwart (marienz)
Assigned to: Karl Waclawek (kwaclaw)
Summary: fix for CVE-2012-0876 breaks "xml" default prefix

Initial Comment:
expat-2.1.0-beta2 will fail a namespace-aware parse of a document relying on the "xml" being bound by default, like the following test document:

<?xml version="1.0"?>
<root xml:whitespace="preserve"/>

xmlwf -n on that document returns "2:0: unbound prefix", while xmlwf from expat 2.0.1 succeeds.

This seems to be caused by the call to setContext(parser, implicitContext) adding that default prefix happening too early (before hash_secret_salt is initialized).

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 21:15

Message:
It did but make check failed. I realized that setContextimplicitContext)
must not be called for child parsers.
This is now fixed in xmlparse.c rev 1 170.

----------------------------------------------------------------------

Comment By: Marien Zwart (marienz)
Date: 2012-03-10 10:07

Message:
Yes, that fixed it, thanks!

----------------------------------------------------------------------

Comment By: Samuli Suominen (ssuominen)
Date: 2012-03-10 09:54

Message:
I can verify that pulling in the diff of xmlparse.c 1.168 and 1.169 and
applying it on top of expat-2.1.0-beta2 works for me. Thanks a lot for
quick action!

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:41

Message:
Fixed in xmlparse.c  rev 1.169. Please test.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-10 09:31

Message:
Yes, your analysis is correct, the hash function was used before the salt
was set.
I will have to move the setContext() call right after the
generate_hash_secret_salt() call.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=3500861&group_id=10127

From noreply at sourceforge.net  Sun Mar 18 23:31:53 2012
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 18 Mar 2012 15:31:53 -0700
Subject: [Expat-bugs] [ expat-Bugs-569461 ] OASIS XML Test Suite
Message-ID: <3V9wBM55k9zQYj@mail.python.org>

Bugs item #569461, was opened at 2002-06-15 12:34
Message generated for change (Comment added) made by kwaclaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=569461&group_id=10127

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Rolf Ade (pointsman)
Assigned to: Karl Waclawek (kwaclaw)
Summary: OASIS XML Test Suite

Initial Comment:
I've tested expat-1.95.3 (with xmltok.c updated to rev.
1.17, becase of bug 566240, all other files are the
original 1.95.3) against the recently updated OASIS xml
test suite (XML 1.0 (Second Edition) errata 20020320,
W3C Conformance Test Suite 20020606), avaliable via
http://www.w3.org/XML/Test/ and found a few new
problems, that are not triggered by older versions of
this test suite.

As in previous reports, I checked all
not-wellformedness tests (should
all raise error) and all valid tests (should all pass)
of the
test-suites xmltest, ibm, sun and oasis with xmlwf -p.
Especially for the well-formedness tests, I have _not_
throughout checked if the error reason, reported by
expat is the expected error, but checked only
mechanical, if the test has raised an error, regardless
of the exact error reason. This method is clearly not
perfect, and this time we have an example, that
underlines this.

ibm/not-wf/P32/ibm32n09.xml

This is a new test, not included in previous versions.
Problem is, that the standalone document declaration
has the value &quot;yes&quot; and there is an external markup
declaration of an entity (other than amp, lt, gt, apos,
quot). xmlwf -p doesn't report an error. The not
well-formedness problem is, that standalone=&quot;yes&quot;
means, that all informations needed to build the XML
infoset must be found in the document entity
(standalone=&quot;yes&quot; doesn't mean, that the document must
not have an external subset or external PE's, only that
this external entities doesn't change - per attribute
defaults or as in this case, entity declarations -
change the info in the document entity. See the last
sentence of &quot;Well-Formedness Constraint: Entity
Declared&quot; (P68).


ibm/not-wf/P68/ibm68n06.xml

Same reason as the test befor. This test _was_ present
in previous versions of the test suite. But with the
previous version of the external subset of this test,
xmlwf claimed a &quot;syntax error&quot; error in the external
subset, which I plain can't understand (eventually an
other expat bug?), but is clearly not the expected
error. In the new version of the test suite, this
external subset now has an XML declaration with
explicite encoding (the older version had only an XML
declaration without encoding) and is accepted by expat.


xmltest/not-wf/not-sa/010.xml
xmltest/not-wf/not-sa/011.xml

This tests are new in this edition of the test suite.
Unfortunately, this both tests seems to be not
documented, either in the test files isself nor in the
documentation file xmlconf-20020606.htm. As far as I
see, this tests test &quot;Validity Constraint: Proper
Declaration/PE Nesting&quot; (P29).


xmltest/not-wf/not-sa/005.xml

This test raised error with previous expat versions,
but does not anymore due to the changes, discussed in
bug 548690. This is intentional, according to the
548690 discussion. This test is now listed under &quot;XML
Documents with Optional Errors&quot;. The test suite
documentation says:

&quot;Conforming XML 1.0 Processors are permitted to ignore
certain errors, or to report them at user option. In
this section of this test report are found descriptions
of test cases which fit into this category.

Processor behavior on such test cases does not affect
conformance to the XML 1.0 (Second Edition)
Recommendation, except as noted.&quot; 

So, according to this, it's OK, that expat doesn't
report an error for this case. Since both reporting
error and not reporting error are OK, it may be
debatably, which behavior is more convenient for the
expat user. (Karl: ;-))



sun/not-wf/not-sa03.xml

This is a new test in this edition of the test suite.
Unfortunately, this test seems not to be documented. As
far as I see, it tests the same as
xmltest/not-wf/not-sa/005.xml



Tests, that still are wrong, as in previous versions are

ibm/not-wf/misc/432gewf.xml
sun/not-wf/uri01.xml

These are already discussed in the past.

Well, that's all.

rolf


----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2012-03-18 15:31

Message:
The results for expat 2.1.0 are based on XML test suite version 20080827,
but they are the same if tested against version 20020606 - see attached
file. Essentially, the test cases that really can be considered as failed
still are the same as for previous Expat versions:

Expected not well-formed: ibm/not-wf/misc/432gewf.xml
Expected not well-formed: sun/not-wf/uri01.xml

Karl

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-12-05 05:31

Message:
Logged In: NO 

can any one of u tell me why i am getting failed case as 
33...
     I will tell u all the errors i got...can anyone tell 
me how to rectify the same....and why these errors arised..
ibm/valid/P02/ibm02v01.xml
ibm/valid/p28/ibm28v02.xml
         /p29/      02.xml
         /p29/      01.xml
similarly p54, p56   , p58, p57,p70, p82
ibm/invalid/p49../p58/...
xmltest/valid/sa/069.xml...76.xml...90.xml...91.xml....





----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2003-09-24 16:48

Message:
Logged In: YES 
user_id=290026

Just changed the summary to be more generic, as this bug 
will probably stay open permanently, assuming we will never
pass 100% of all test cases.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2003-01-29 13:26

Message:
Logged In: YES 
user_id=290026

This is just to report that the new release Expat 1.95.6
passes the OASIS test suite (same version - 20020606)
with the exact same results as Expat 1.95.5.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2003-01-21 11:21

Message:
Logged In: YES 
user_id=290026

For comparison purposes, I ran the xmltest.sh test script
against release 1.95.5. The results are attached as
TestResults_1_95_5.txt.

Discussion of results:

There are many cases were &quot;output differs&quot;is reported,
but these are due to xmlwf having a different definition
of &quot;canonical XML&quot; than used in the test suite.

Leaving these out, and those that were discussed 
already in this thread, we have the following errors 
reported:
(Note: the two test cases 
          ibm/not-wf/P32/ibm32n09.xml and
          ibm/not-wf/P68/ibm68n06.xml
          are not reported anymore by the script)

* In ibm/invalid/P49/: ibm49i02.xml:7:1: error in 
   processing external entity reference:
   The associated DTD file does not exist - an error
   in the test suite.

The next three documents are not UTF-8 encoded,
and do not have an XML declaration, so Expat rejects 
them, which is correct. An error in the test suite.

* In xmltest/valid/sa/: 049.xml:2:0: not well-formed 
   (invalid token)
* In xmltest/valid/sa/: 050.xml:2:0: not well-formed 
   (invalid token)
* In xmltest/valid/sa/: 051.xml:2:0: not well-formed 
   (invalid token)

The next two documents are classified as invalid,
but well-formed, but they contain faulty UTF-16 
encoding, so they should be classified as not
well-formed. Expat seems correct here.

* In sun/invalid/: utf16b.xml:2:0: not well-formed 
   (invalid token)
* In sun/invalid/: utf16l.xml:1:40: not well-formed 
   (invalid token)

The next three are not marked as why they should fail, 
so the script thinks they are not well-formed, but in
fact they are:

* Well formed: oasis/p06fail1.xml
* Well formed: oasis/p08fail1.xml
* Well formed: oasis/p08fail2.xml

So, no new test case errors have really been added
for release 1.95.5.

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2003-01-19 11:01

Message:
Logged In: YES 
user_id=290026

Just a comment:
This bug report will likely stay open until Expat passes the
OASIS test suite without any problem at all.
Since no parser currently achieves this, there is a good
chance this bug report will stay open for a long time to come..


----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-07-07 08:39

Message:
Logged In: YES 
user_id=290026

Patch #587161 should fix some of the problems,
but I specifically made no attempt to fix the problems
Expat has with:
- xmltest/not-wf/not-sa/010.xml and
- xmltest/not-wf/not-sa/011.xml.

Reason: It turns out, after consulting with the mailing
list for the XML test suite, public-xml-testsuite at w3.org,
that these two violate WFC: PE Between Declarations.
There is no quick and easy fix for this in Expat,
and I would have to spend some time thinking about it,
which I don't have at the moment.

Karl


----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-06-17 20:15

Message:
Logged In: YES 
user_id=290026

Assigned to me, but only for the three test cases
described in my last message.

Karl

----------------------------------------------------------------------

Comment By: Rolf Ade (pointsman)
Date: 2002-06-17 16:21

Message:
Logged In: YES 
user_id=13222

Agreed

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-06-17 12:05

Message:
Logged In: YES 
user_id=290026

Given an improved understanding of section 4.1
in the XML spec, I will try to fix the following
test cases in the next Expat release:

ibm/not-wf/P32/ibm32n09.xml,
ibm/not-wf/P68/ibm68n06.xml and
sun/not-wf/not-sa03.xml 

In my opinion, the third one is not the same type as
xmltest/not-wf/not-sa/005.xml, but the same type
as the other two.

About the test cases
xmltest/not-wf/not-sa/010.xml and
xmltest/not-wf/not-sa/011.xml:

If they really check validity constraint P29, as Rolf
has suggested, then it is OK that Expat does not report
an error.

So, If I am successful, we would be left with only:
ibm/not-wf/misc/432gewf.xml and
sun/not-wf/uri01.xml,
conformance with which does not seem a 100% 
necessity, as previously discussed.

Karl









----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=569461&group_id=10127