[XML-SIG] Sgmlop SAX 2 parser

[Alexandre Fayolle]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

---1463793919-1301454201-995652246=:3530
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hello,

Here's a first version of my attempt at providing a SAX2 parser for
Sgmlop. It still features some debugging prints. I'd be very grateful if
you could scrutinize it hard and tell me what you think of it. I'll be
back online on Monday.

from xml.sax.drivers2 import drv_sgmlop
from xml.dom.ext.reader.Sax2 import Reader
p = drv_sgmlop.SaxHtmlParser()
r = Reader(parser=p)
d = r.fromUri('http://www.slashdot.org/')
#debugging output skipped...
from xml.dom.ext import PrettyPrint
PrettyPrint(d)

Cheers,

Alexandre Fayolle
-- 
LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org
Narval, the first software agent available as free software (GPL).

---1463793919-1301454201-995652246=:3530
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="drv_sgmlop.py"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.21.0107202004060.3530@pisces.logilab.fr>
Content-Description: 
Content-Disposition: attachment; filename="drv_sgmlop.py"

IiIiDQpTQVgyIGRyaXZlciBmb3IgdGhlIHNnbWxvcCBwYXJzZXIuDQoNCiRJ
ZCQNCiIiIg0KDQp2ZXJzaW9uPSIwLjEiDQoNCmZyb20geG1sLnBhcnNlcnMu
c2dtbGxpYiBpbXBvcnQgU0dNTFBhcnNlcg0KZnJvbSB4bWwuc2F4IGltcG9y
dCBzYXhsaWINCmZyb20geG1sLnNheC54bWxyZWFkZXIgaW1wb3J0IEF0dHJp
YnV0ZXNJbXBsLFhNTFJlYWRlcg0KZnJvbSB4bWwuc2F4LnNheHV0aWxzIGlt
cG9ydCBDb250ZW50R2VuZXJhdG9yLCBwcmVwYXJlX2lucHV0X3NvdXJjZQ0K
DQoNCnRyeToNCiAgICBpbXBvcnQgY29kZWNzDQogICAgZGVmIHRvX3htbF9z
dHJpbmcoc3RyLGVuY29kaW5nKToNCiAgICAgICAgdHJ5Og0KICAgICAgICAg
ICAgZGVjb2RlciA9IGNvZGVjcy5sb29rdXAoZW5jb2RpbmcpWzFdDQogICAg
ICAgICAgICBlbmNvZGVyID0gY29kZWNzLmxvb2t1cCgndXRmLTgnKVswXQ0K
ICAgICAgICAgICAgcmV0dXJuIGVuY29kZXIoZGVjb2RlcihzdHIpWzBdKVsw
XQ0KICAgICAgICBleGNlcHQgTG9va3VwRXJyb3I6DQogICAgICAgICAgICBy
ZXR1cm4gc3RyDQpleGNlcHQgSW1wb3J0RXJyb3I6DQogICAgZnJvbSB4bWwu
dW5pY29kZS5pc284ODU5IGltcG9ydCB3c3RyaW5nDQogICAgZGVmIHRvX3ht
bF9zdHJpbmcoc3RyLGVuY29kaW5nKToNCiAgICAgICAgaWYgdXBwZXIoc2Vs
Zi5fZW5jb2RpbmcpID09ICdVVEYtOCc6DQogICAgICAgICAgICByZXR1cm4g
c3RyDQogICAgICAgIGVsc2U6DQogICAgICAgICAgICByZXR1cm4gd3N0cmlu
Zy5kZWNvZGUoZW5jb2Rpbmcsc3RyKS51dGY4KCkNCiAgICAgICAgDQoNCg0K
Y2xhc3MgU2F4UGFyc2VyKFNHTUxQYXJzZXIsWE1MUmVhZGVyKToNCiAgICAi
IiIgSW1wbGVtZW50cyBJbmNyZW1lbnRhbFJlYWRlciAiIiINCg0KICAgIGRl
ZiBfX2luaXRfXyhzZWxmLGJ1ZnNpemUgPSA2NTUzNixlbmNvZGluZz0nVVRG
LTgnKToNCiAgICAgICAgWE1MUmVhZGVyLl9faW5pdF9fKHNlbGYpDQogICAg
ICAgIFNHTUxQYXJzZXIuX19pbml0X18oc2VsZikNCiAgICAgICAgc2VsZi5w
YXJzZXIgPSBTR01MUGFyc2VyKCkNCiAgICAgICAgc2VsZi5fYnVmc2l6ZSA9
IGJ1ZnNpemUNCiAgICAgICAgc2VsZi5fbGV4aWNhbF9oYW5kbGVyID0gTm9u
ZQ0KICAgICAgICBzZWxmLl9lbmNvZGluZyA9IGVuY29kaW5nDQogICAgICAg
IA0KICAgIGRlZiBwYXJzZShzZWxmLCBzb3VyY2UpOg0KICAgICAgICBzb3Vy
Y2UgPSBwcmVwYXJlX2lucHV0X3NvdXJjZShzb3VyY2UpDQoNCiAgICAgICAg
c2VsZi5wcmVwYXJlUGFyc2VyKHNvdXJjZSkNCiAgICAgICAgZmlsZSA9IHNv
dXJjZS5nZXRCeXRlU3RyZWFtKCkNCiAgICAgICAgYnVmZmVyID0gZmlsZS5y
ZWFkKHNlbGYuX2J1ZnNpemUpDQogICAgICAgIHdoaWxlIGJ1ZmZlciAhPSAi
IjoNCiAgICAgICAgICAgIHNlbGYuZmVlZChidWZmZXIpDQogICAgICAgICAg
ICBidWZmZXIgPSBmaWxlLnJlYWQoc2VsZi5fYnVmc2l6ZSkNCiAgICAgICAg
c2VsZi5jbG9zZSgpDQoNCiAgICBkZWYgcHJlcGFyZVBhcnNlcihzZWxmLCBz
b3VyY2UpOg0KICAgICAgICAiIiJUaGlzIG1ldGhvZCBpcyBjYWxsZWQgYnkg
dGhlIHBhcnNlIGltcGxlbWVudGF0aW9uIHRvIGFsbG93DQogICAgICAgIHRo
ZSBTQVggMi4wIGRyaXZlciB0byBwcmVwYXJlIGl0c2VsZiBmb3IgcGFyc2lu
Zy4iIiINCiAgICAgICAgc2VsZi5fY29udF9oYW5kbGVyLnN0YXJ0RG9jdW1l
bnQoKQ0KICAgICAgICANCiAgICBkZWYgY2xvc2Uoc2VsZik6DQogICAgICAg
ICIiIlRoaXMgbWV0aG9kIGlzIGNhbGxlZCB3aGVuIHRoZSBlbnRpcmUgWE1M
IGRvY3VtZW50IGhhcyBiZWVuDQogICAgICAgIHBhc3NlZCB0byB0aGUgcGFy
c2VyIHRocm91Z2ggdGhlIGZlZWQgbWV0aG9kLCB0byBub3RpZnkgdGhlDQog
ICAgICAgIHBhcnNlciB0aGF0IHRoZXJlIGFyZSBubyBtb3JlIGRhdGEuIFRo
aXMgYWxsb3dzIHRoZSBwYXJzZXIgdG8NCiAgICAgICAgZG8gdGhlIGZpbmFs
IGNoZWNrcyBvbiB0aGUgZG9jdW1lbnQgYW5kIGVtcHR5IHRoZSBpbnRlcm5h
bA0KICAgICAgICBkYXRhIGJ1ZmZlci4NCg0KICAgICAgICBUaGUgcGFyc2Vy
IHdpbGwgbm90IGJlIHJlYWR5IHRvIHBhcnNlIGFub3RoZXIgZG9jdW1lbnQg
dW50aWwNCiAgICAgICAgdGhlIHJlc2V0IG1ldGhvZCBoYXMgYmVlbiBjYWxs
ZWQuDQoNCiAgICAgICAgY2xvc2UgbWF5IHJhaXNlIFNBWEV4Y2VwdGlvbi4i
IiINCiAgICAgICAgU0dNTFBhcnNlci5jbG9zZShzZWxmKQ0KICAgICAgICBz
ZWxmLl9jb250X2hhbmRsZXIuZW5kRG9jdW1lbnQoKSAgICAgICAgDQoNCiAg
ICBkZWYgX21ha2VfYXR0cl9kaWN0KHNlbGYsYXR0cl9saXN0KToNCiAgICAg
ICAgZCA9IHt9DQogICAgICAgIGN2cnQgPSBsYW1iZGEgc3RyLGU9c2VsZi5f
ZW5jb2Rpbmc6dG9feG1sX3N0cmluZyhzdHIsZSkNCiAgICAgICAgZm9yIChh
LGIpIGluIGF0dHJfbGlzdDoNCiAgICAgICAgICAgIGRbY3ZydChhKV09Y3Zy
dChiKQ0KICAgICAgICByZXR1cm4gZA0KICAgIA0KICAgIGRlZiB1bmtub3du
X3N0YXJ0dGFnKHNlbGYsdGFnLGF0dHJzKToNCiAgICAgICAgc2VsZi5fY29u
dF9oYW5kbGVyLnN0YXJ0RWxlbWVudCh0b194bWxfc3RyaW5nKHRhZyxzZWxm
Ll9lbmNvZGluZyksDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgQXR0cmlidXRlc0ltcGwoc2VsZi5fbWFrZV9hdHRyX2RpY3Qo
YXR0cnMpKSkNCg0KICAgIGRlZiB1bmtub3duX2VuZHRhZyhzZWxmLHRhZyk6
DQogICAgICAgIHNlbGYuX2NvbnRfaGFuZGxlci5lbmRFbGVtZW50KHRvX3ht
bF9zdHJpbmcodGFnLHNlbGYuX2VuY29kaW5nKSkNCg0KICAgIGRlZiBoYW5k
bGVfZGF0YShzZWxmLGRhdGEpOg0KICAgICAgICBzZWxmLl9jb250X2hhbmRs
ZXIuY2hhcmFjdGVycyh0b194bWxfc3RyaW5nKGRhdGEsc2VsZi5fZW5jb2Rp
bmcpKQ0KDQogICAgZGVmIGhhbmRsZV9jb21tZW50KHNlbGYsZGF0YSk6DQog
ICAgICAgIGlmIHNlbGYuX2xleGljYWxfaGFuZGxlciBpcyBub3QgTm9uZToN
CiAgICAgICAgICAgIHNlbGYuX2xleGljYWxfaGFuZGxlci5jb21tZW50KHRv
X3htbF9zdHJpbmcoZGF0YSxzZWxmLl9lbmNvZGluZykpDQoNCiAgICBkZWYg
c2V0X3Byb3BlcnR5KHNlbGYsbmFtZSx2YWx1ZSk6DQogICAgICAgIGlmIG5h
bWUgPT0gaGFuZGxlci5wcm9wZXJ0eV9sZXhpY2FsX2hhbmRsZXI6DQogICAg
ICAgICAgICBzZWxmLl9sZXhpY2FsX2hhbmRsZXIgPSB2YWx1ZQ0KICAgICAg
ICBlbHNlOg0KICAgICAgICAgICAgcmFpc2UgU0FYTm90UmVjb2duaXplZEV4
Y2VwdGlvbigiUHJvcGVydHkgJyVzJyBub3QgcmVjb2duaXplZCIgJSBuYW1l
KQ0KICAgIGRlZiBnZXRQcm9wZXJ0eShzZWxmLCBuYW1lKToNCiAgICAgICAg
aWYgbmFtZSA9PSBoYW5kbGVyLnByb3BlcnR5X2xleGljYWxfaGFuZGxlcjoN
CiAgICAgICAgICAgIHJldHVybiBzZWxmLl9sZXhpY2FsX2hhbmRsZXINCiAg
ICAgICAgcmFpc2UgU0FYTm90UmVjb2duaXplZEV4Y2VwdGlvbigiUHJvcGVy
dHkgJyVzJyBub3QgcmVjb2duaXplZCIgJSBuYW1lKQ0KDQojIyAgICBkZWYg
Z2V0RmVhdHVyZShzZWxmLCBuYW1lKToNCiMjICAgICAgICBpZiBuYW1lID09
IGhhbmRsZXIuZmVhdHVyZV9uYW1lc3BhY2VzOg0KIyMgICAgICAgICAgICBy
ZXR1cm4gc2VsZi5fbmFtZXNwYWNlcw0KIyMgICAgICAgIHJhaXNlIFNBWE5v
dFJlY29nbml6ZWRFeGNlcHRpb24oIkZlYXR1cmUgJyVzJyBub3QgcmVjb2du
aXplZCIgJSBuYW1lKQ0KDQojIyAgICBkZWYgc2V0RmVhdHVyZShzZWxmLCBu
YW1lLCBzdGF0ZSk6DQojIyAgICAgICAgaWYgc2VsZi5fcGFyc2luZzoNCiMj
ICAgICAgICAgICAgcmFpc2UgU0FYTm90U3VwcG9ydGVkRXhjZXB0aW9uKCJD
YW5ub3Qgc2V0IGZlYXR1cmVzIHdoaWxlIHBhcnNpbmciKQ0KIyMgICAgICAg
IGlmIG5hbWUgPT0gaGFuZGxlci5mZWF0dXJlX25hbWVzcGFjZXM6DQojIyAg
ICAgICAgICAgIHNlbGYuX25hbWVzcGFjZXMgPSBzdGF0ZQ0KIyMgICAgICAg
IGVsc2U6DQojIyAgICAgICAgICAgIHJhaXNlIFNBWE5vdFJlY29nbml6ZWRF
eGNlcHRpb24oIkZlYXR1cmUgJyVzJyBub3QgcmVjb2duaXplZCIgJQ0KIyMg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5h
bWUpDQoNCiAgICAjIC0tLSBFWFBFUklNRU5UQUwgUFlUSE9OIFNBWCBFWFRF
TlNJT05TDQoNCiAgICBkZWYgZ2V0X3BhcnNlcl9uYW1lKHNlbGYpOg0KICAg
ICAgICByZXR1cm4gInNnbWxvcCINCg0KICAgIGRlZiBnZXRfcGFyc2VyX3Zl
cnNpb24oc2VsZik6DQogICAgICAgIHJldHVybiAiVW5rbm93biINCg0KICAg
IGRlZiBnZXRfZHJpdmVyX3ZlcnNpb24oc2VsZik6DQogICAgICAgIHJldHVy
biB2ZXJzaW9uDQogICAgDQogICAgZGVmIGlzX3ZhbGlkYXRpbmcoc2VsZik6
DQogICAgICAgIHJldHVybiAwDQoNCiAgICBkZWYgaXNfZHRkX3JlYWRpbmco
c2VsZik6DQogICAgICAgIHJldHVybiAwDQoNCg0KZnJvbSB4bWwuZG9tLmh0
bWwgaW1wb3J0IEhUTUxfQ0hBUkFDVEVSX0VOVElUSUVTLCBIVE1MX0ZPUkJJ
RERFTl9FTkQsSFRNTF9PUFRfRU5ELEhUTUxfRFREDQpmcm9tIHN0cmluZyBp
bXBvcnQgc3RyaXAsdXBwZXINCg0KY2xhc3MgU2F4SHRtbFBhcnNlcihTYXhQ
YXJzZXIpOg0KDQogICAgZGVmIF9faW5pdF9fKHNlbGYsYnVmc2l6ZSA9IDY1
NTM2LGVuY29kaW5nPSdpc28tODg1OS0xJyk6DQogICAgICAgIFNheFBhcnNl
ci5fX2luaXRfXyhzZWxmLGJ1ZnNpemUsZW5jb2RpbmcpDQogICAgICAgIA0K
ICAgIGRlZiBmaW5pc2hfc3RhcnR0YWcoc2VsZiwgdGFnLCBhdHRycyk6DQog
ICAgICAgICIiInVzZXMgdGhlIEhUTUwgRFREIHRvIGF1dG9tYXRpY2FsbHkg
Z2VuZXJhdGUgZXZlbnRzDQogICAgICAgIGZvciBtaXNzaW5nIHRhZ3MiIiIN
Cg0KICAgICAgICBwcmludCAnc3RhcnQnLHRhZyxzZWxmLnN0YWNrDQogICAg
ICAgICMgZ3Vlc3Mgb21pdHRlZCBjbG9zZSB0YWdzDQogICAgICAgIHdoaWxl
IHNlbGYuc3RhY2sgYW5kIFwNCiAgICAgICAgICAgICAgdXBwZXIoc2VsZi5z
dGFja1stMV0pIGluIEhUTUxfT1BUX0VORCBhbmQgXA0KICAgICAgICAgICAg
ICB0YWcgbm90IGluIEhUTUxfRFRELmdldChzZWxmLnN0YWNrWy0xXSxbXSk6
DQogICAgICAgICAgICBwcmludCAnIyBmb3JjZSBlbmQnLHNlbGYuc3RhY2tb
LTFdDQogICAgICAgICAgICBzZWxmLnVua25vd25fZW5kdGFnKHNlbGYuc3Rh
Y2tbLTFdKQ0KICAgICAgICAgICAgZGVsIHNlbGYuc3RhY2tbLTFdDQoNCiAg
ICAgICAgaWYgc2VsZi5zdGFjayBhbmQgdGFnIG5vdCBpbiBIVE1MX0RURC5n
ZXQoc2VsZi5zdGFja1stMV0sW10pOg0KICAgICAgICAgICAgcHJpbnQgJyon
KjMwDQogICAgICAgICAgICBwcmludCAnV2FybmluZyA6IHRyeWluZyB0byBh
ZGQgJXMgYXMgYSBjaGlsZCBvZiAlcyclXA0KICAgICAgICAgICAgICAgICAg
KHRhZyxzZWxmLnN0YWNrWy0xXSkNCiAgICAgICAgDQogICAgICAgIHNlbGYu
dW5rbm93bl9zdGFydHRhZyh0YWcsYXR0cnMpDQogICAgICAgIGlmIHVwcGVy
KHRhZykgaW4gSFRNTF9GT1JCSURERU5fRU5EOg0KICAgICAgICAgICAgIyBj
bG9zZSBpbW1lZGlhdGx5IHRhZ3MgZm9yIHdoaWNoIHdlIHdvbid0IGdldCBh
bmQgZW5kDQogICAgICAgICAgICBwcmludCAnZW5kJyx0YWcNCiAgICAgICAg
ICAgIHNlbGYudW5rbm93bl9lbmR0YWcodGFnKQ0KICAgICAgICAgICAgcmV0
dXJuIDANCiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIHNlbGYuc3RhY2su
YXBwZW5kKHRhZykNCiAgICAgICAgcmV0dXJuIDENCg0KICAgIGRlZiBmaW5p
c2hfZW5kdGFnKHNlbGYsIHRhZyk6DQogICAgICAgIHByaW50ICdlbmQnLHRh
ZyxzZWxmLnN0YWNrDQogICAgICAgIGlmIHRhZyBpbiBIVE1MX0ZPUkJJRERF
Tl9FTkQgOg0KICAgICAgICAgICAgIyBkbyBub3RoaW5nOiB3ZSd2ZSBhbHJl
YWR5IGNsb3NlZCBpdA0KICAgICAgICAgICAgcmV0dXJuDQogICAgICAgIGlm
IHRhZyBpbiBzZWxmLnN0YWNrOg0KICAgICAgICAgICAgd2hpbGUgc2VsZi5z
dGFjayBhbmQgc2VsZi5zdGFja1stMV0gIT0gdGFnOg0KICAgICAgICAgICAg
ICAgIHByaW50ICcjIGZvcmNlIGVuZCcsc2VsZi5zdGFja1stMV0NCiAgICAg
ICAgICAgICAgICBzZWxmLnVua25vd25fZW5kdGFnKHNlbGYuc3RhY2tbLTFd
KQ0KICAgICAgICAgICAgICAgIGRlbCBzZWxmLnN0YWNrWy0xXQ0KICAgICAg
ICAgICAgc2VsZi51bmtub3duX2VuZHRhZyh0YWcpDQogICAgICAgICAgICBk
ZWwgc2VsZi5zdGFja1stMV0NCiAgICAgICAgZWxzZToNCiAgICAgICAgICAg
IHByaW50ICcqJyozMA0KICAgICAgICAgICAgcHJpbnQgIldhcm5pbmc6IEkg
ZG9uJ3Qgc2VlIHdoZXJlIHRhZyAlcyB3YXMgb3BlbmVkIiV0YWcNCg0KDQog
ICAgZGVmIGhhbmRsZV9kYXRhKHNlbGYsZGF0YSk6DQogICAgICAgIGlmIHNl
bGYuc3RhY2s6DQogICAgICAgICAgICBpZiAnI1BDREFUQScgbm90IGluIEhU
TUxfRFRELmdldChzZWxmLnN0YWNrWy0xXSxbXSkgYW5kIG5vdCBzdHJpcChk
YXRhKToNCiAgICAgICAgICAgICAgICAjIHRoaXMgaXMgcHJvYmFibHkgaWdu
b3JhYmxlIHdoaXRlc3BhY2UNCiAgICAgICAgICAgICAgICBwcmludCAnd2hp
dGVTcGFjZScNCiAgICAgICAgICAgICAgICBzZWxmLl9jb250X2hhbmRsZXIu
aWdub3JhYmxlV2hpdGVzcGFjZShkYXRhKQ0KICAgICAgICAgICAgZWxzZToN
CiAgICAgICAgICAgICAgICBwcmludCAnZGF0YScsc3RyaXAoZGF0YSkNCiAg
ICAgICAgICAgICAgICBzZWxmLl9jb250X2hhbmRsZXIuY2hhcmFjdGVycyh0
b194bWxfc3RyaW5nKGRhdGEsc2VsZi5fZW5jb2RpbmcpKQ0KDQogICAgZGVm
IGNsb3NlKHNlbGYpOg0KICAgICAgICBwcmludCAnZW5kIGRvY3VtZW50Jyxz
ZWxmLnN0YWNrDQogICAgICAgIFNHTUxQYXJzZXIuY2xvc2Uoc2VsZikNCiAg
ICAgICAgc2VsZi5zdGFjay5yZXZlcnNlKCkNCiAgICAgICAgZm9yIHRhZyBp
biBzZWxmLnN0YWNrOg0KICAgICAgICAgICAgcHJpbnQgJyMgZm9yY2UgZW5k
Jyx0YWcNCiAgICAgICAgICAgIHNlbGYudW5rbm93bl9lbmR0YWcodGFnKQ0K
ICAgICAgICBzZWxmLnN0YWNrID0gW10NCiAgICAgICAgc2VsZi5fY29udF9o
YW5kbGVyLmVuZERvY3VtZW50KCkgICAgICAgIA0KDQoNCg0KDQojIC0tLS0N
Cg0KZGVmIGNyZWF0ZV9wYXJzZXIoKToNCiAgICByZXR1cm4gU2F4UGFyc2Vy
KCkNCg0KZGVmIGNyZWF0ZV9odG1sX3BhcnNlcigpOg0KICAgIHJldHVybiBT
YXhIdG1sUGFyc2VyKCkNCg==
---1463793919-1301454201-995652246=:3530--